Research

In my PhD and PostDoc research, I conducted a series of studies on task-oriented interactions between humans and between humans and machines, with a focus on mutual understanding - commonly referred to as grounding in psycholinguistics. My work explored how understanding is established in conversations and the interactional phenomena involved in this process. Addressing gaps in computational models of understanding, my studies utilised multisensory input and were evaluated using statistical and machine-learning models. Given that miscommunication is common in human conversations, my research examined how embodied computer interfaces can assess human responses to determine whether spoken utterances are understood, a central contribution of my dissertation.

Some of our studies investigated how humans incrementally establish understanding and collaboratively resolve misunderstandings in joint-construction tasks. Drawing from interaction paradigms in human-human communication, I examined collaborative interactions between humans and machines, focusing on two key factors: embodiment and conversational failures. These methods explored whether embodiment influences grounding behaviours and the verbal and non-verbal channels used to respond to and recover from miscommunication. To further enhance interaction, I integrated explainability, exploring how tailored explanations can improve user understanding of system decisions and foster alignment in collaborative tasks. For applications in robotics and conversational user interfaces, I have developed failure detection systems capable of predicting user uncertainty in real-time, paving the way for multimodal computer interfaces that can detect and respond to dialogue breakdowns, system failures, and user-specific needs for explanation.