Details - MACS Project System

Proposer: Phil Bartie
Title: Natural Language Navigation based on images
Goal
Description: For this project computer vision will be used to return a list of objects in view (eg from a Streetview type image) from which a user can input questions that a LLM (large language model) will be able to answer based on the knowledge gained from the image for the task of navigation. The purpose is to create a service that when provided an image (or set of images) can answer questions related to navigation (ie direction, position). For example: user) which way do I head now? system) towards the building with a red door user) the one next to the bins? system) yes, head towards the bins, and the red door, the cafe will be just after that on your left - it has green windows. Topics: Scene graph, visual grounded models, LLMs, navigation tasks, UI, UX, Geographic Information Systems, Location based services
Resources
Background
Url
Difficulty Level: High
Ethical Approval: None
Number Of Students: 1
Supervisor: Phil Bartie
Keywords
Degrees: Bachelor of Science in Computer Science
Bachelor of Science in Computer Systems
Master of Engineering in Software Engineering
Master of Science in Artificial Intelligence
Master of Science in Data Science