View Proposal

---

Proposer

John See

Title

Understanding Malaysian Food Through AI: A Data Analytics Study

Goal

The aim of this project is to evaluate the responses of frontier multimodal AI models in tasks like identifying Malaysian dishes, their ingredients, and their dietary/nutritional information. What follows is an investigation into agreements, disagreements and errors across different models and to generate valuable insights into the strengths and limitation of these models.

Description

This project will make use of an existing Malaysian food image dataset (created from a previous project) and use several contemporary frontier multimodal AI models (e.g. GPT, Claude, Gemini) to analyse them. Customised prompts are to be crafted and the models will be asked to identify dishes, infer ingredients, determine dietary characteristics, and estimate nutritional information such as calories and macronutrients. Their responses will be collected, structured and compared against appropriate reference information. The project will focus on analysing the resulting data obtained from these frontier models, and this creates a new meta-dataset that is useful for further research in future. Data analytics techniques will be used to examine model accuracy, agreement, differences between food categories, nutritional estimation errors, and the relationship between AI confidence and actual correctness. The project may also investigate whether particular types of Malaysian food are consistently more challenging for AI to understand.

Resources

There is available School (fund) support for token costs in using the AI models.

Background

Multimodal AI models can analyse food images beyond simply recognising a dish. They can potentially identify ingredients, describe dietary characteristics, estimate nutritional content, and provide information about the food's cultural context. However, the reliability of these AI-generated insights may vary, particularly for culturally diverse foods such as Malaysian cuisine, where dishes can have regional variations, visually similar alternatives, hidden ingredients and different preparation methods. There hasn't been a lot of studies that check on the ability of modern LLMs/VLMs in understanding the complex nature of food images, even more so for the diverse Malaysian cuisine. This provides an opportunity to use data analytics to systematically investigate what AI can and cannot reliably understand about Malaysian food.

Url

Difficulty Level

Easy

Ethical Approval

Full

Number Of Students

1

Supervisor

John See

Keywords

model evaluation, food analysis, data analytics

Degrees

BSc Information Systems with Data Analytics