Jason B. Alonso, Angela Chang, Jeff Orkin, Cynthia Breazeal

Abstract

The vision in Human Robot Interaction is that autonomous robots will have the ability to understand common sense behavioral patterns. We present a method of learning these behavior patterns through a multi-modal online game where two players collaborate to create a salad through selecting and discussing available salad items. Our approach relies heavily on the creation of a virtual environment that mimics real-world interactions and the common sense assumptions necessary to create a believable environment. We find that significant resources are required to create realistic scenarios where common sense can be transferred from the real world. The collected data is intended for learning behavior models for autonomous social robots. We summarize four considerations for future game designers who intend to capture common sense using virtual worlds, with particular attention to how the common sense used to create virtual environments influences the scope of the interaction data captured.