Journal, Conference, and Workshop Papers
Kai-yuh Hsiao, Soroush Vosoughi, Stefanie Tellex, Rony Kubat, and Deb Roy. (2008). Object Schemas for Responsive Robotic Language Use. To appear in Proceedings of the 3rd ACM/IEEE International Conference on Human-Robot Interaction. pdf (269K)
Jeff Orkin and Deb Roy. (2007). The Restaurant Game: Learning Social Behavior and Language from Thousands of Players Online. Journal of Game Development, 3(1), 39-60. pdf (3.9MB)
Rony Kubat, Philip DeCamp, Brandon Roy, and Deb Roy. (2007). TotalRecall: Visualization and Semi-Automatic Annotation of Very Large Audio-Visual Corpora. Ninth International Conference on Multimodal Interfaces (ICMI 2007). pdf (491K)
Michael Fleischman and Deb Roy. (2007) Unsupervised Content-Based Indexing of Sports Video Retrieval. 9th ACM Workshop on Multimedia Information Retrieval (MIR). Augsburg, Germany. pdf (264K)
Michael Fleischman, Brandon Roy, and Deb Roy. (2007) Temporal Feature Induction for Baseball Highlight Classification. ACM Multimedia Conference. Augsburg, Germany. pdf (317K)
Peter Gorniak and Deb Roy. (2007). Situated Language Understanding as Filtering Perceived Affordances. Cognitive Science, 31(2), 197-231. pdf (1.7MB)
Michael Fleischman and Deb Roy. (2007). Situated Models of Meaning for Sports Video Retrieval. HLT/ACL 2007, Rochester, NY. pdf (293K)
Stefanie Tellex and Deb Roy. (2007). Grounding Language in Spatial Routines. AAAI 2007 Spring Symposia on Control Mechanisms for Spatial Knowledge Processing in Cognitive / Intelligent Systems, Stanford University, Palo Alto CA. pdf (116K)
Michael Levit and Deb Roy. (2007). Interpretation of Spatial Language in a Map Navigation Task. IEEE Transactions on Systems, Man, and Cybernetics, Part B, 37(3), 667-679. pdf (386K)
Michael Fleischman, Philip DeCamp, and Deb Roy. (2006). Mining Temporal Patterns of Movement for Video Content Classification. Proceedings of the 8th ACM SIGMM International Workshop on Multimedia Information Retrieval. pdf (323K)
Deb Roy, Rupal Patel, Philip DeCamp, Rony Kubat, Michael Fleischman, Brandon Roy, Nikolaos Mavridis, Stefanie Tellex, Alexia Salata, Jethran Guinness, Michael Levit, Peter Gorniak. (2006). The Human Speechome Project. Proceedings of the 28th Annual Cognitive Science Conference. pdf (756K)
Peter Gorniak and Deb Roy. (2006). Perceived Affordances as a Substrate for Linguistic Concepts. Twenty-eighth Annual Meeting of the Cognitive Science Society, 6 pages. pdf (3,318K)
Peter Gorniak, Jeff Orkin, and Deb Roy. (2006). Speech, Space and Purpose: Situated Language Understanding in Computer Games. Twenty-eighth Annual Meeting of the Cognitive Science Society Workshop on Computer Games. pdf (313K)
Nikolaos Mavridis and Deb Roy. (2006). Grounded Situation Models for Robots: Where Words and Percepts Meet. Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). pdf (598K)
Dong Zhang, Daniel Gatica-Perez, Deb Roy, Samy Bengio. (2006). Modeling Interactions from Email Communication. IEEE International Conference on Multimedia & Expo (ICME). pdf (208K)
Stefanie Tellex and Deb Roy. (2006). Spatial Routines for a Simulated Speech-Controlled Vehicle. Proceedings of Human Robot Interaction Conference 2006 (HRI-2006). pdf (200K)
Philip DeCamp, Amber Frid-Jimenez, Jethran Guiness and Deb Roy. (2005). Gist Icons: Seeing Meaning in Large Bodies of Literature. IEEE Info Visualization 2005 Conference. pdf (1.5MB)
Kai-yuh Hsiao and Deb Roy. (2005). A Habit System for an Interactive Robot. AAAI Fall Symposium 2005:
From Reactive to Anticipatory Cognitive Embodied
Systems. pdf (981K)
Peter Gorniak and Roy (2005). Probabilistic Grounding of Situated Speech using Plan Recognition and Reference Resolution. Seventh International Conference on Multimodal Interfaces (ICMI 2005). Best Paper Award. pdf (312K)
Michael Fleischman and Deb Roy. (2005). Intentional Context in Situated Language Learning. Ninth Conference on Computational Natural Language Learning. pdf (224K)
Nick Mavridis and Deb Roy. (2005). Grounded Situation Models for Robots: Bridging language, Perception, and Action. AAAI-05 Workshop on Modular Construction of Human-Like Intelligence. pdf (544K)
Deb Roy. (2005). Semiotic Schemas: A Framework for Grounding Language in Action and Perception. Artificial Intelligence, 167(1-2):170-205. pdf (1 MB)
Michael Fleischman and Deb Roy. (2005). Why are verbs harder to learner than nouns? Initial insights from a computational model of situated word learning. 27th Annual Meeting of the Cognitive Science Society. pdf (584K)
Deb Roy. (2005). Grounding words in perception and action: computational insights. Trends in Cognitive Science, 9(8), 389-396.
pdf (272K)
Kai-yuh Hsiao, Peter Gorniak, and Deb Roy. NetP: A Network API for Building Heterogeneous Modular Intelligent Systems. Proceedings of AAAI 2005 Workshop in Modular Construction of Human-Like Intelligence, pdf (667K)
Peter Gorniak and Deb Roy. (2005). Speaking with your Sidekick: Understanding Situated Speech in Computer Role Playing Games. Proceedings of Artificial Intelligence and Interactive Digital Entertainment, 2005. pdf (624K)
Deb Roy and Niloy Mukherjee. (2005). Towards Situated Speech Understanding: Visual Context Priming of Language Models. Computer Speech and Language, 19(2), pages 227-248. pdf (567K)
Joshua Juster and Deb Roy. (2004). Elvis: Situated Speech and Gesture Understanding for a Robotic Chandelier. Proc. Int. Conf. Multimodal Interfaces. pdf (372K)
Deb Roy, Yair Ghitza, Jeff Bartelma, and Charlie Kehoe. (2004). Visual Memory Augmentation: Using Eye Gaze as an Attention Filter. Proceedings of the IEEE International Symposium on Wearable Computers. pdf (8 MB)
Deb Roy, Kai-Yuh Hsiao, and Nikolaos Mavridis. (2004). Mental
Imagery for a Conversational Robot. IEEE Transactions on Systems, Man,
and Cybernetics, Part B, 34(3), 1374-1383. pdf (488K)
Peter Gorniak and Deb Roy. (2004). Grounded Semantic Composition for Visual Scenes, Journal of Artificial Intelligence Research, Volume 21, pages 429-470. pdf (1.2MB)
Peter Gorniak and Deb Roy. (2003). Augmenting User Interfaces with Adaptive Speech Commands. In Proceedings of the International Conference for Multimodal Interfaces. pdf (355K)
Peter Gorniak and Deb Roy. (2003). A Visually Grounded Natural Language Interface for Reference to Spatial Scenes. In Proceedings of the International Conference for Multimodal Interfaces. pdf (562K)
Brian Whitman, Deb Roy, Barry Vercoe. (2003). Learning Word Meanings and Descriptive Parameter Spaces from Music. In Proceedings of the HLT-NAACL03 workshop on Learning Word Meaning from Non-Linguistic Data. pdf (570K)
Kai-yuh Hsiao, Nikolaos Mavridis, Deb Roy. Coupling Perception and Simulation: Steps Towards Conversational Robotics. (2003). The Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems. pdf (206K)
Deb Roy, Kai-yuh Hsiao, Nikolaos Mavridis. (2003). Conversational Robots: Building Blocks for Grounding Word Meanings. In Proceedings of the HLT-NAACL03 Workshop on Learning Word Meaning from Non-Linguistic Data. pdf (364K)
Deb Roy. (2003). Grounded Spoken Language Acquisition: Experiments in Word Learning. IEEE Transactions on Multimedia, 5(2): 197-209. pdf (1.1MB)
Deb Roy, Peter Gorniak, Niloy Mukherjee, and Josh Juster. (2002). A Trainable Spoken Language Understanding System for Visual Object Selection. In Proceedings of the International Conference of Spoken Language Processing. pdf (86K)
Deb Roy. (2002). A Trainable Visually-Grounded Spoken Language Generation System. In Proceedings of the International Conference of Spoken Language Processing. pdf (177K)
Deb Roy. (2002). Learning Words and Syntax for a Visual Description Task. Computer Speech and Language. pdf (513K)
Deb Roy. (2001/2002). Learning Visually Grounded Words and Syntax of Natural Spoken Language. Evolution of Communication. 4(1). pdf (829K)
Deb Roy and Alex Pentland. (2002) Learning Words from Sights and Sounds: A Computational Model. Cognitive Science, 26(1), 113-146. pdf (689K)
Ewa Dominowska, Deb Roy and Rupal Patel. (2002) An Adaptive Context-Sensitive Communication Aid. Proceedings for the 17th Annual International Conference "Technology and Persons with Disabilities".
Deb Roy. (2000) Integration of Speech and Vision using Mutual Information. Int. Conf. Acoustics, Speech and Signal Processing. pdf (626K)
Ph.D. Theses in Media Arts and Sciences
Kai-yuh Hsiao. (2007) Embodied Object Schemas for Grounding Language Use. Ph.D. in Media Arts and Sciences Thesis. pdf (7.6M)
Peter Gorniak. (2005) The Affordance-Based Concept. Ph.D. in Media Arts and Sciences Thesis. pdf (5.8M)
Masters Theses in Media Arts and Sciences
Philip DeCamp. (2007) HeadLock: Wide-Range Head Pose Estimation for Low Resolution Video. M.Sc. in Media Arts and Sciences Thesis. pdf (24.4M)
Jeff Orkin. (2007) Learning Plan Networks in Conversational Video Games. M.Sc. in Media Arts and Sciences Thesis. pdf (6.9M)
Philipp Robbel. (2007) Exploiting Object Dynamics for Recognition and Control. M.Sc. in Media Arts and Sciences Thesis. pdf (6.1M)
Brandon Roy. (2007) Human-Machine Collaboration for Rapid Speech Transcription. M.Sc. in Media Arts and Sciences Thesis. pdf (13.1M)
Stefanie Tellex. (2006) Grounding Language in Spatial Routines. M.Sc. in Media Arts and Sciences Thesis. pdf (1.3M)
Andre Ribeiro. (2005) Graph Dynamics: Learning and Representation. M.Sc. in Media Arts and Sciences Thesis. pdf (1.1M)
Niloy Mukherjee. (2003) Spontenous Speech Recognition Using Visual Context-Aware Language Models. M.Sc. in Media Arts and Sciences Thesis. pdf (1360K)
Sheel Sanjay Dhande. (2003) A Computational Model to Connect Gestalt Perception and Natural Language. M.Sc. in Media Arts and Sciences Thesis. pdf (736K)
M.Eng. Theses in EECS
Charles Kehoe. (2005) Indexical Grounding for a Mobile Robot. M.Eng. EECS Thesis. pdf (255K)
Jeffrey Bartelma. (2004) Flycatcher: Fusion of Gaze with Hierarchical Image Segmentation for Robust Object Detection. M.Eng. EECS Thesis. pdf (949K)
Joshua Juster. (2004) Speech and Gesture Understanding in a Homeostatic Control Framework for a Robotic Chandelier. M.Eng. EECS Thesis. pdf (539K)
Christopher Lucas. (2004) Patent Semantics: Analysis, Search, and Visualization of Large Text Corpora. M.Eng. EECS Thesis. pdf (918K)
Ewa Dominowska. (2002) A Communication Aid with Context-Aware Vocabulary Prediction. M.Eng. EECS Thesis. pdf (1.5M)
Norimasa Yoshida. (2002) Automatic Utterance Segmentation in Spontaneous Speech. M.Eng. EECS Thesis. pdf (591K)
Ben Yoder. (2001) Spontaneous Speech Recognition Using Hidden Markov Models. M.Eng. EECS Thesis. pdf (591K)
Technical Reports
Peter Gorniak. (2003) Meaning "I". General Exams Paper. pdf (2.8M)
|