View Proposal

---

Proposer

Matthew Aylett

Title

Graphical character for conversational interaction

Goal

Implement a graphical character that is pollable, streamable and interruptible

Description

In human dialog, participants are able to interrupt each other at any point. Once a dialog participant has been interrupted they may cede the floor (let another person speak) or alter what they are saying to show they are actively listening. In this project a graphical character will be implemented using Unity and Python that allows some limited behaviour (e.g. head nods, eye brow raise), that is integrated with a speech synthesis system (CereProc) allowing basic lip syncing and output speech. An API will be designed and built over http that will allow a client Python program to stream content (e.g. text for the character to speak and instructions for behaviour), poll the character (establish within a narrow time window what the character has said or done from the streamed content), and interruptible (will allow the system to stop the character gracefully or to stop the character and begin producing new content).

Resources

CereProc TTS system

Background

Aylett, M. P., & Romeo, M. (2023, July). You Don’t Need to Speak, You Need to Listen: Robot Interaction and Human-Like Turn-Taking. In Proceedings of the 5th International Conference on Conversational User Interfaces (pp. 1-5).

Url

Difficulty Level

Challenging

Ethical Approval

Full

Number Of Students

2

Supervisor

Matthew Aylett

Keywords

face animation, python

Degrees

Bachelor of Science in Computer Science
Master of Design in Games Design and Development
Master of Science in Artificial Intelligence
Master of Science in Human Robot Interaction
Master of Science in Robotics