Modeling Gaze Behavior as a Function of Discourse Structure

Obed E. Torres, Justine Cassell, Scott Prevost
The Media Laboratory
Massachusetts Institute of Technology
20 Ames Street
Cambridge, MA 02139
{obed,justine,prevost}@media.mit.edu

This paper addresses the problem of designing conversational agents that exhibit appropriate gaze behavior during dialogues with human users. Previous research on gaze behavior has concentrated on its relationship to turn-taking phenomena [3,4,5]. Recent work has incorporated some of these findings into the design of autonomous human-like conversational agents and interactive communicative humanoids [1,13]. However, none of this research has examined the relationship between information structure and gaze behavior. In this paper we discuss why turn-taking is not an adequate explanation for gaze behavior in conversation and why information structure should be integrated with turn-taking as an explanation for this behavior. We then examine the relationship of gaze behavior to information structure and turn-taking through an empirical analysis of discourse transcripts for several dyadic conversations. A simple algorithm for assigning gaze behavior is proposed on the basis of the findings of this empirical analysis. We describe work in progress on implementing this algorithm in an existent communicative humanoid agent [14] with the goal of producing more natural gaze behavior related to propositional content in human-computer conversations.