Marilyn A. Walker & Obed E. Torres
Mitsubishi Electric Research Laboratories & MIT Media Laboratory
201 Broadway & 20 Ames St. 
Cambridge, Ma. 02139, USA 
walker@merl.com, obed@media.mit.edu

Linguistic Variation and Character in VIVA

VIVA (Virtual Interaction with Virtual Actors) is a virtual
theater where human actors can perform roles and interact with synthetic
actors in a play.  VIVA is built on SPLINE (Scalable Platform
for Interactive Environments), a 3D distributed virtual world environments
that supports distributed animation and audio (Waters etal. 1994).
Human actors in VIVA choose a role to perform in a play; any roles not
played by humans can be performed by synthetic actors.    

Previous work in interactive theater has focused on techniques for allowing
users to vary the plot structure (Maes etal. 1994, Weyhrauch and Bates 1995).  
Our hypothesis is that significantly interesting capabilities for user 
expression can be provided within a single plot line.  This is based on our 
observation that there are many human activities that are intrinsically 
ritualistic or depend on predictable content, e.g. re-reading a book, 
playing an instrument, or participating in a religious ceremony.  Given a 
fully scripted interaction, synthetic actors must be able to perform their 
roles on cue by making minimal use of speech recognition technology to 
track progress through a script. 

However, our research goals are to support conversational interaction and 
improvisation by both human and synthetic actors.  Supporting improvisation 
by human actors will require innovative use of both natural language and 
speech technologies.  Our approach to improvisation by synthetic actors 
focuses on expressive variation, which we believe defines a fundamental 
aspect of character.

We are currently developing a framework for varying the expressive
quality of the communicative choices that characters make, by varying
a small number of cognitive and social parameters, which determine the 
choices an agent makes about the linguistic form and affective intonational 
realization of an utterance.  For example, consider the various
expressive forms shown in (2) to (6) for the proposal communicative
act in (1) from a task-oriented dialogue (Walker 1993):

(propose Speaker Hearer (put agent:Hearer item:green-couch location:study))
Put the green couch in the study.
Let's put the green couch in the study.
Would you mind putting the green couch in the study?
The green couch could be in the study, couldn't it?
The green couch isn't in the study yet, is it?

Choices in linguistic form can be combined with choices about
expressing affect through intonational realization; any of (2) to (6)
can be realized with emotions such as disgust, surprise, annoyance,
warmth, sadness, or fear (Cahn, 1990). Not surprisingly, the
combination of linguistic form and emotion produces a range of
expressive capabilities not realized by either system alone.
Annoyance combined with the ultra-indirect form of (5) humorously
evokes the character of a fussbudget, while annoyance combined with
the direct form in (2) evokes a drill sergeant. Combinations of these
choices allow one plot line to be realized in many ways.

By the time of the conference, we expect to have a fully implemented 
version of VIVA that supports scripted interaction by human and 
synthetic actors.  In addition, we will have a initial prototype with
improvised interaction by synthetic actors.  We also intend to explore
the use of VIVA as a language learning environment, by using a single
play script in multiple languages, where we expect
predictable content to facilitate language learning.