View Proposal

---

Proposer

Zi Hau Chin

Title

Detection of AI-Generated Programming Solutions Using Code-Structure Features

Goal

Investigate whether structural source-code characteristics can distinguish AI-generated solutions from human-written solutions.

Description

This project involves developing and evaluating a machine-learning system that investigates whether structural characteristics of source code can help distinguish AI-generated programming solutions from human-written solutions. The student will construct a dataset containing solutions to the same programming problems written by students or other human programmers and generated by several AI models. The dataset should also include more realistic cases, such as AI-generated code that has been manually edited, human-written code that has been automatically reformatted, and solutions produced using different prompts or programming styles. Rather than relying only on superficial text patterns, the system will extract code-structure features such as abstract syntax tree patterns, cyclomatic complexity, nesting depth, function length, identifier characteristics, control-flow structures, library usage, exception handling, and code duplication. The student will implement and compare several classifiers and investigate which combinations of lexical, structural, and software-metric features provide the most reliable results. The system should produce a probability or confidence level and explain which features contributed to each prediction. A major part of the project will be testing whether the detector generalises beyond the data used for training. It should therefore be evaluated on unseen programming problems, outputs from an AI model excluded from training, and code modified through identifier renaming, comment removal, reformatting, or refactoring. Suitable evaluation measures include precision, recall, F1 score, false-positive rate, calibration, and performance under code transformations. Particular attention should be given to false positives involving genuine human-written solutions. The completed system must be presented as an experimental screening and analysis tool, not as conclusive proof that a student used generative AI or committed academic misconduct. Which structural features are most useful for detecting AI-generated code? -Does the detector generalise to unseen programming tasks? -Does it generalise to unseen AI models? -How robust is it to human editing, reformatting, identifier renaming, and refactoring? -Are its explanations stable and meaningful?

Resources

HumanEval, HumanEval on Hugging Face, Zero-Shot Detection of Machine-Generated Code, Project CodeNet

Background

Python, supervised machine learning, feature engineering and model evaluation, generative AI and code-generation models

Url

Difficulty Level

High

Ethical Approval

Full

Number Of Students

1

Supervisor

Zi Hau Chin

Keywords

ai-generated code detection, code stylometry, code authorship, abstract syntax tree, machine-generated code, explainable classification

Degrees

Bachelor of Science in Computing Science