Coordination and context-dependence in the generation of embodied conversation

Justine Cassell *
* MIT Media Laboratory
20 Ames St., Cambridge, MA 02139

Matthew Stone **
** Department of Computer Science & Center
for Cognitive Science
Rutgers University
110 Frelinghuysen Rd., Piscataway, NJ 08854

Hao Yan *



We describe the generation of communicatve actions in an implemented embodied comversational agent. Our agent plans each utterance so that multiple communicative goals may be realized opportunistically by a composite action including not only speech but also coverbal gesture that fits the context and the ongoing speecch in ways representative of natural human conversation. We accomplish this by reasoning from a grammar which describes gesture declaratively in terms of its discourse function, semantics and synchrony with speech.