View Proposal

---

Proposer

Oliver Lemon

Title

Social robotics: HRI for multi-person conversation

Goal

To develop and test a socially-intelligent robot that can correctly handle multiple humans in a conversation

Description

This project will develop a new conversational interface for a robot receptionist for the Robotarium building -- using the robot animated head FurHat: http://www.speech.kth.se/furhat/ which we have in the lab, or using the PAL ARI robot. A key aspect will be to handle multiple people in the conversation (e.g. 2 groups of 2 visitors, one group wants to find the cafe, the other has a meeting in room 3). The project combines audio and visual processing to create a socially intelligent robot. Using e.g. the Furhat SDK and an LLM or VLM, you will develop and test a new interface that performs socially intelligent human-robot interaction for greeting visitors to a building, allowing them to find their meeting location, alerting their contacts, handling deliveries, etc. You will evaluate the system you create with real users. If you use the PAL ARI robot, you could attempt to navigate the robot to lead the human to the right location. This project should handle dialogue with multiple humans in the scene. You should take F21CA Conversational Agents if you do this project.

Resources

Furhat SDK, generative AI system such as GPT4o, LLAVA etc

Background

AI, Conversational agents F21CA

Url

External Link

Difficulty Level

Moderate

Ethical Approval

InterfaceOnly

Number Of Students

2

Supervisor

Oliver Lemon

Keywords

generative ai, llm, human-robot interaction, evaluation

Degrees

Bachelor of Science in Computer Science
Master of Engineering in Software Engineering
Master of Science in Artificial Intelligence
Master of Science in Artificial Intelligence with SMI
Master of Science in Computing (2 Years)
Master of Science in Data Science
Master of Science in Human Robot Interaction
Master of Science in Robotics
Master of Science in Software Engineering
Bachelor of Engineering in Robotics
Master of Science in Robotics with Industrial Application