View Proposal

---

Proposer

Michael Lones

Title

Assessing the reliability of LLM-generated machine learning code

Goal

Description

Large language models (LLMs) are increasingly being used to generate software, including code for machine learning (ML) applications. While these tools can greatly accelerate development, there are growing concerns about the correctness, robustness, and security of the code they produce. Errors in ML code can be particularly difficult to detect, potentially leading to faulty models, unreliable results, or hidden vulnerabilities. One possible explanation is that, compared with mainstream software engineering, there are relatively fewer examples of high-quality ML code available for training LLMs, alongside many examples of poor coding practices found online. However, the extent of this problem has not yet been studied rigorously. This project will investigate the quality of ML code generated by modern LLMs. The student will design experiments to evaluate code correctness, identify common types of mistakes, and assess whether generated code follows established ML best practices. The project may also explore methods for improving code quality, such as prompt engineering or structured guidance. This is an opportunity to work at the intersection of machine learning, software engineering, and AI safety, contributing to an important and rapidly emerging area of research.

Resources

Background

Take a look at this paper which looked at the ability of LLMs to spot mistakes in ML code (spoiler alert: they didn't do well!): https://arxiv.org/abs/2505.18220

Url

Difficulty Level

Variable

Ethical Approval

None

Number Of Students

1

Supervisor

Michael Lones

Keywords

machine learning, llm, code vulnerabilities

Degrees

Bachelor of Science in Computer Science
Bachelor of Science in Computer Systems
Master of Engineering in Software Engineering
Master of Science in Artificial Intelligence
Master of Science in Computer Science for Cyber Security
Master of Science in Computer Systems Management
Master of Science in Computing (2 Years)
Master of Science in Data Science
Master of Science in Software Engineering
Bachelor of Engineering in Robotics
Bachelor of Science in Computer Science (Cyber Security)
MSc Applied Cyber Security