Coordination and context-dependence in the generation of embodied conversation

Justine Cassell *
* MIT Media Laboratory
E15-315
20 Ames St., Cambridge, MA 02139
justine@media.mit.edu

Matthew Stone **
** Department of Computer Science & Center
for Cognitive Science
Rutgers University
110 Frelinghuysen Rd., Piscataway, NJ 08854
mdstone@cs.rutgers.edu

Hao Yan *
yanhao@media.mit.edu

 

ABSTRACT

We describe the generation of communicatve actions in an implemented embodied comversational agent. Our agent plans each utterance so that multiple communicative goals may be realized opportunistically by a composite action including not only speech but also coverbal gesture that fits the context and the ongoing speecch in ways representative of natural human conversation. We accomplish this by reasoning from a grammar which describes gesture declaratively in terms of its discourse function, semantics and synchrony with speech.