Responsive Environments

We have developed an extremely low-power and low-cost wireless sensor network aimed at applications in asset tracking and ubiquitous activity monitoring. In addition to identifying an object, these nodes (termed active radio-frequency identification [RFID] tags) employ sensors to monitor its state, enabling new applications in fields like security, home automation, and supply-chain management. Although they contain a battery, these tags are not limited by it: by minimizing power consumption and quasi-passively waking on diverse stimuli (changes in light, RF carrier presence, shock or acceleration, and sound), they can last for years. Furthermore, their low cost and small size make them a good candidate for large-scale experiments at the intersection of RFID and wireless sensor networks.

We are constructing a configurable infrastructure to protect users' dynamic levels of privacy in a pervasive sensor network. The work is based around a configurable badge that can alert the user to the presence of participating sensor networks, plus emit an RF beacon with which the network can gauge the level of privacy desired. Badges either periodically emit an "opt out" signal, blocking sensing within their RF (and sensor perceptual) range, or allow the users' desired level of privacy to be configured in a Web interface. This privacy level depends on user location, and can eliminate or "blur" the data and calculated features available from various sensors and sensor nodes. The privacy level can also be dependent on the status of the client browsing the sensor network—the badge user can assign different levels of privacy to different groups of people. Physical means of providing immediate privacy are also afforded (e.g., physically obstructing the sensors), and users can also scrub (or selectively "blur") any archived data.

We can use sensor networks as a way of building a comprehensive picture of a physical space to transmit a sense of that space into a virtual environment. In this project, we use the Spinner sensor nodes to capture video, audio, and environmental information. This information is used to build an interactive three-dimensional space in Second Life that makes it easy to explore the history of any sensor node, as well as interact live using text and audio with people at any of the nodes. Avatars’ virtual presence near nodes is also shown on the physical nodes, so visitors to the physical space know when there are virtual visitors nearby.

Funk2 is a novel process-description language that keeps track of everything that it does. Remembering these causal execution traces allows parallel threads to reflect, recognize, and react to the history and status of other threads. Novel forms of complex, adaptive, nonlinear control algorithms can be written in the Funk2 programming language. Currently, Funk2 is implemented to take advantage of distributed grid processors consisting of a heterogeneous network of computers, so that hundreds of thousands of parallel threads can be run concurrently, each using many gigabytes of memory. Funk2 is inspired by Marvin Minsky's Critic-Selector theory of human cognitive reflection.

This is a suite of devices and protocols to support applications in wearable human/social sensing linked to a distributed camera and vision system. The current system includes a sensate wristwatch with biological and gestural sensors, a lapel-pin device with motion and audio-affect sensing, and a wall-mounted device with a high-resolution camera, environmental sensors, and a localization system for all devices in the network. All devices record data and audio in sync with the recorded video. A full-spec Zigbee network supports device synchronization and mesh networking. All devices have enough on-board power to extract features from the data.

Building on well-developed data visualization techniques, audio is used to enhance understanding by improving the brain's ability to parse information in the scientific process. Sonification (aka auditory display) is the use of non-speech audio to convey information. In collaboration with the Media Lab, NASA is interested in using the human auditory system's powers of organizing and deconstructing sound for the purposes of scientific research and exploratory data analysis. Faced with increasingly voluminous and complex multimodal data and computations, researchers are seeking novel ways to optimize the conversion of information into knowledge. Several sonifications are under study, including spectral data from Mars, sensor data from the Media Lab, space-suit-helmet heads-up displays, new augmentations of the traditional orrery, and enhanced exploration of the Mandelbrot set.

This project explores the wide-scale distribution of low-power, low-cost sensor nodes that can measure temperature, humidity, CO2 content, light levels, and human presence. These sensor nodes will enable buildings to react quickly and effectively to the changing needs of their inhabitants, automatically controlling, for example, heating/air conditioning, windows (opening and shades), and lighting control. Total building power consumption can be reduced, and repair requests can be made automatically.

Spinner is a Lab-wide sensor network platform designed to detect and capture fragmented events of human behavior that can be collected and sequenced into a cohesive narrative conveying a larger overall meaning. This project also looks at the development of parametric models of narrative that can be mapped on to sensor-detectable elements of human activity.

This project is a system of compact, wearable, wireless sensor nodes, equipped with full six-degree-of-freedom inertial measurement units and node-to-node capacitive proximity sensing. A high-bandwidth, channel-shared RF protocol has been developed to acquire data from many (e.g., 25) of these sensors at 100 Hz full-state update rates, and software is being developed to fuse this data into a compact set of descriptive parameters in real time. A base station and central computer clock the network and process received data. We aim to capture and analyze the physical movements of multiple people in real time, using unobtrusive sensors worn on the body. Applications abound in biomotion analysis, sports medicine, health monitoring, interactive exercise, immersive gaming, and interactive dance ensemble performance.