 |
||
 |
 |
 |
 |
||
 |
 |
 |
 |
 |
 |
 |
 |
 |
|
Human Speechome Project The Human Speechome Project is an effort to observe and computationally model the longitudinal language development of a single child at an unprecedented scale. To achieve this, we are recording, storing, visualizing, and analyzing communication and behavior patterns in several hundred thousand hours of home video and speech recordings. The tools that are being developed for mining and learning from hundreds of terabytes of multimedia data offer potential for breaking open new opportunities for a broad range of areas - from security to personal memory augmentation. This project is supported in part through donations of hardware and other resources by Zetera Corporation, Bell Microproducts, Seagate Technology, and funding from the National Science Foundation.
|
 |
|
Situated Natural Language Processing for Sports Video The quantity and availability of video content is soaring due to the combination of television networks and the Internet. The aim of this project is to develop more effective means to manage, search, and translate video content. We are developing algorithms that interpret language in video (speech and closed caption text) by exploiting aspects of the non-linguistic context, or situation, conveyed by the accompanying video. We model situations by automatically finding patterns within low-level audio/video features that represent events. Event patterns are then mapped to words spoken in the video in order to create a "grounded" dictionary of word meanings. Our research focuses on sports video, in particular, on Major League Baseball games. We are exploring applications in multimedia search and video-based machine translation.
|
 |
Trisk: A Conversational Robot Trisk is a humanoid robot that integrates speech input, visual perception, and active touch in order to interact with humans and its environment. It can understand and obey natural language commands, and will soon be able to answer questions. The robot is a platform for designing new algorithms and multimodal knowledge representations for sensory-motor grounded language use. This research takes steps towards social robots that can coordinate activities with human partners using natural language and gesture.
|
 |
Behavior Capture from Thousands of People Online The Restaurant Game is a research project that will algorithmically combine the gameplay experiences of thousands of players to create a new game. We will apply machine learning algorithms to data collected through the multiplayer Restaurant Game, and produce a new single-player game that we will enter into the 2008 Independent Games Festival. Everyone who plays The Restaurant Game will be credited as a Game Designer.
|
 |
