Communicative Humanoids: Abstract

A Computational Model of Psychosocial Dialogue Skills

Submitted to the Program in Media Arts & Sciences,
School of Architecture & Planning
on July 19, 1996
in partial fulfillment of the requirements for the degree of
Doctor of Philosophy
at the
Massachusetts Institute of Technology

Abstract

Face-to-face interaction between people is generally effortless and effective. We exchange glances, take turns speaking and make facial and manual gestures to achieve the goals of the dialogue. Endowing computers with such an interaction style marks the beginning of a new era in our relationship with machines-one that relies on communication, social convention and dialogue skills. This thesis presents a computational model of psychosocial dialogue expertise, bridging between perceptual analysis of multimodal events and multimodal action generation, supporting the creation of interfaces that afford full-duplex, real-time face-to-face interaction between a human and autonomous computer characters. The architecture, called Ymir, has been implemented in software, and a prototype humanoid created. The humanoid, named Gandalf, commands a graphical model of the solar system, and can interact with people using speech, manual and facial gesture. Gandalf has been tested in interaction with users and has been shown capable of fluid face-to-face dialogue. The prototype demonstrates several new ideas in the creation of communicative computer agents, including perceptual integration of multimodal events, distributed processing and decision making, layered input analysis and motor control, and the integration of reactive and reflective perception and action. Applications of the work presented in this thesis can be expected in such diverse fields as education, psychological and social research, work environments, and entertainment.

THESIS COMMITTEE

JUSTINE CASSELL
Assistant Professor of Media Arts & Sciences, MIT Program in Media Arts & Sciences
PATTIE MAES
Associate Professor of Media Arts & Sciences, Sony Corporation Career Development Professor of Media Arts & Sciences
STEPHEN WHITTAKER
Research Scientist, AT&T Labs Research

Key parts of my thesis have been published in peer-reviewed journals and conferences. If you prefer these versions (sometimes more compact and/or expanded) you can get them here.
Full Architecture		A Mind Model for Multimodal Communicative Creatures and Humanoids. International Journal of Applied Artificial Intelligence, 13(4-5): 449-486. Thórisson, K. R. (1999).
Perception		Machine Perception of Multimodal Natural Dialogue. In P. McKevitt, S. Ó Nulláin, C. Mulvihill (Eds.), Language, Vision & Music, 97-115. Amsterdam: John Benjamins. Thórisson, K. R. (2002).
Decision		Real-Time Decision Making in Multimodal Face to Face Communication. Second ACM International Conference on Autonomous Agents, Minneapolis, Minnesota, May 11-13, 16-23. Thórisson, K. R. (1998).
Action		Layered Modular Action Control for Communicative Humanoids. Computer Animation '97, Geneva, Switzerland, June 5-6, 134-143. Thórisson, K. R. (1997).
Turntaking & Dialog		Natural Turn-Taking Needs No Manual: Computational Theory and Model, from Perception to Action. In B. Granström, D. House, I. Karlsson (Eds.), Multimodality in Language and Speech Systems, 173-207. Dordrecht, The Netherlands: Kluwer Academic Publishers. Thórisson, K. R. (2002).

Communicative Humanoids

A Computational Model of Psychosocial Dialogue Skills

Abstract

THESIS COMMITTEE

Downloadable Thesis Chapters

Table of Contents