We aim to (1) Create autonomous systems including interactive physical robots and synthetic characters in virtual worlds that learn to communicate in human-like ways; (2) Understand how children learn to communicate through longitudinal in vivo observation and analysis; (3) Develop tools for visualizing, searching, and analyzing large corpora (e.g., video) using task-dependent semantic models. Cutting across these research threads is a theoretical interest in understanding how the cognitive structures and processes underlying social skills such as language use are grounded in interaction with the physical world.