Marilyn A. Walker & Obed E. Torres Mitsubishi Electric Research Laboratories & MIT Media Laboratory 201 Broadway & 20 Ames St. Cambridge, Ma. 02139, USA walker@merl.com, obed@media.mit.edu Linguistic Variation and Character in VIVA VIVA (Virtual Interaction with Virtual Actors) is a virtual theater where human actors can perform roles and interact with synthetic actors in a play. VIVA is built on SPLINE (Scalable Platform for Interactive Environments), a 3D distributed virtual world environments that supports distributed animation and audio (Waters etal. 1994). Human actors in VIVA choose a role to perform in a play; any roles not played by humans can be performed by synthetic actors. Previous work in interactive theater has focused on techniques for allowing users to vary the plot structure (Maes etal. 1994, Weyhrauch and Bates 1995). Our hypothesis is that significantly interesting capabilities for user expression can be provided within a single plot line. This is based on our observation that there are many human activities that are intrinsically ritualistic or depend on predictable content, e.g. re-reading a book, playing an instrument, or participating in a religious ceremony. Given a fully scripted interaction, synthetic actors must be able to perform their roles on cue by making minimal use of speech recognition technology to track progress through a script. However, our research goals are to support conversational interaction and improvisation by both human and synthetic actors. Supporting improvisation by human actors will require innovative use of both natural language and speech technologies. Our approach to improvisation by synthetic actors focuses on expressive variation, which we believe defines a fundamental aspect of character. We are currently developing a framework for varying the expressive quality of the communicative choices that characters make, by varying a small number of cognitive and social parameters, which determine the choices an agent makes about the linguistic form and affective intonational realization of an utterance. For example, consider the various expressive forms shown in (2) to (6) for the proposal communicative act in (1) from a task-oriented dialogue (Walker 1993): (propose Speaker Hearer (put agent:Hearer item:green-couch location:study)) Put the green couch in the study. Let's put the green couch in the study. Would you mind putting the green couch in the study? The green couch could be in the study, couldn't it? The green couch isn't in the study yet, is it? Choices in linguistic form can be combined with choices about expressing affect through intonational realization; any of (2) to (6) can be realized with emotions such as disgust, surprise, annoyance, warmth, sadness, or fear (Cahn, 1990). Not surprisingly, the combination of linguistic form and emotion produces a range of expressive capabilities not realized by either system alone. Annoyance combined with the ultra-indirect form of (5) humorously evokes the character of a fussbudget, while annoyance combined with the direct form in (2) evokes a drill sergeant. Combinations of these choices allow one plot line to be realized in many ways. By the time of the conference, we expect to have a fully implemented version of VIVA that supports scripted interaction by human and synthetic actors. In addition, we will have a initial prototype with improvised interaction by synthetic actors. We also intend to explore the use of VIVA as a language learning environment, by using a single play script in multiple languages, where we expect predictable content to facilitate language learning.