View Proposal

---

Proposer

Oliver Lemon

Title

Assistive Robots with Generative AI: voice-controlled robot helpers

Goal

To explore and create voice-controlled assistive robots for people who can talk but not manipulate objects

Description

Some people are not able to manipulate objects but they can speak. This project explores the development of voice-controlled assistant robots to help them, for example fetching objects and manipulating them, to enable more independence. You will use vision-language-action models (VLA) such as SmolVLA, Open VLA, or Gemini robotics to create and evaluate such a system, either in simulation or with a robot. You will use a speech recogniser such as Whisper. You should take F20/21 CA- conversational agents to do this project.

Resources

Huggingface Le Robot: https://github.com/huggingface/lerobot SmolVLA: https://huggingface.co/blog/smolvla OpenVLA: https://github.com/openvla/openvla Whisper: https://openai.com/index/whisper/

Background

AI, NLP, conversational agents, robotics

Url

External Link

Difficulty Level

Moderate

Ethical Approval

Full

Number Of Students

2

Supervisor

Oliver Lemon

Keywords

assistive technology, ai, generative ai, llm, speech, conversation, interface, robotics, human-robot interaction, nlp

Degrees

Bachelor of Science in Computer Science
Master of Science in Artificial Intelligence
Master of Science in Artificial Intelligence with SMI
Master of Science in Human Robot Interaction
Master of Science in Robotics
Bachelor of Science in Computing Science
Bachelor of Engineering in Robotics
Master of Science in Robotics with Industrial Application