T. Starner, S. Mann, B. Rhodes, J. Levine, Jennifer Healey, D. Kirsch, Rosalind W. Picard, A. Pentland

Abstract

Wearable computing moves computation from the desktop to the user. We are forming a community of networked wearable computer users to explore, over a long period, the augmented realities that these systems can provide. By adapting its behavior to the user's changing environment, a body-worn computer can assist the user more intelligently, consistently, and continuously than a desktop system. A text-based augmented reality, the Remembrance Agent, is presented to illustrate this approach. Video cameras are used both to warp the visual input (mediated reality) and to sense the user's world for graphical overlay. With a camera, the computer tracks the user's finger, which acts as the system's mouse; performs face recognition; and detects passive objects to overlay 2.5D and 3D graphics onto the real world. Additional apparatus such as audio systems, infrared beacons for sensing location, and biosensors for learning about the wearer's affect are described. Using the input from these interface devices and sensors, a long term goal of this project is to model the user's actions, anticipate his or her needs, and perform a seemless interaction between the virtual and physical environments.