Stefanie Tellex, Deb Roy

Abstract

We have defined a lexicon of words in terms of spatial routines, and used that lexicon to build a speech controlled vehicle in a simulator. A spatial routine is a script composed from a set of primitive operations on occupancy grids, analogous to Ullman’s visual routines. The vehicle understands the meaning of context-dependent natural language commands such as “Go across the room.” When the system receives a command, it combines definitions from the lexicon according to the parse structure of the command, creating a script that selects a goal for the vehicle. Spatial routines may provide the basis for interpreting spatial language in a broad range of physically situated language understanding systems.