Deb Roy

Abstract

This chapter presents a physical-computational model of sensory-motor grounded language interpretation for simple speech acts. The model is based on an implemented conversational robot. It combines a cybernetic closed-loop control architecture with structured conceptual schemas. The interpretation of directive and descriptive speech acts consists of translating utterances into updates of memory systems in the controller. The same memory systems also mediate sensory-motor interactions and thus serve as a cross-modal bridge between language, perception, and action. The referential, functional, and connotative meanings of speech acts emerge from the effects of memory updates on the future dynamics of the controller as it physically interacts with its environment.