View Proposal

---

Proposer

Matthew Aylett

Title

Human-like turn-taking

Goal

Build a runtime system that can converse with a user rapidly, smothly and fluidly

Description

Current conversational systems with robots and artificial agents are typically speak-wait systems. The system uses speech synthesis to produce output, waits for a user to say something. When the user stops speaking, the system processes the input speech and generates a response on its content. However, human conversation does not work like this. Human response times to a dialog partner are on average 200ms, they also can interrupt, they can show they are listening by producing backchannels ('yea', 'aha', 'okey etc). Significance progress has been made with incremental speech recognition and with the use of transformer models to create natural conversational content, this project is about putting this all together with previous work analyzing human-turn taking to produce a fluid conversational system.

Resources

https://github.com/ErikEkstedt/VoiceActivityProjection?tab=readme-ov-file Cereproc speech sythessis Azure incremental speech recognition

Background

The project would consist of building a human-like conversational system with some current tools, modify them to work in real time, and evaluate the conversations with human subjects. Any system should be made available through git-hub as an open source project and carefully documented to allow other researchers to run the system.

Url

Difficulty Level

Moderate

Ethical Approval

Full

Number Of Students

1

Supervisor

Matthew Aylett

Keywords

speech technology, conversational agents, dialog systems, human evaluation

Degrees

Bachelor of Science in Computer Science
Bachelor of Science in Computer Systems
Master of Engineering in Software Engineering
Master of Science in Artificial Intelligence
Master of Science in Artificial Intelligence with SMI
Master of Science in Computing (2 Years)
Master of Science in Data Science
Master of Science in Human Robot Interaction
Master of Science in Robotics
Master of Science in Software Engineering
Bachelor of Science in Computing Science
Bachelor of Engineering in Robotics
BSc Data Sciences