View Proposal

---

Proposer

Oliver Lemon

Title

Human-Robot Teamwork with Generative AI

Goal

To use generative AI (LLMs and VLMs) to explore and develop models of teamwork in interactive robotics

Description

This project will explore how humans, robots, and AIs can collaborate together in teams, to coordinate on shared tasks. This will involve generating conversational interaction to understand tasks, agree plans, resolve ambiguities, correct mistakes, and so on. You will use LLMs and/or VLMs (such as LLAMA, LLAVA, GPT, Gemini, Moshi, etc ...) to create a system which can meet and coordinate with a previously unseen other agent (a human or robot) and collaborate with them to complete a shared task -- for example to tidy up a room, make breakfast, or build a lego model. You will use real robots such as Tiago, ARI, Stretch, Furhat and/or simulations of them. You will evaluate the system's effectiveness and efficiency in completing different shared tasks with different people.

Resources

Generative AI models such as LLAMA, LLAVA, GPT, Gemini etc. Possibly use robots such as Tiago, ARI, Stretch, Furhat

Background

AI, Conversational Agents, ideally some NLP

Url

External Link

Difficulty Level

Moderate

Ethical Approval

InterfaceOnly

Number Of Students

2

Supervisor

Oliver Lemon

Keywords

generative ai, llm, human-robot interaction, evaluation

Degrees

Bachelor of Science in Computer Science
Master of Engineering in Software Engineering
Master of Science in Artificial Intelligence
Master of Science in Artificial Intelligence with SMI
Master of Science in Human Robot Interaction
Master of Science in Robotics
Master of Science in Software Engineering
Bachelor of Engineering in Robotics
Master of Science in Robotics with Industrial Application