How do we leverage people to make systems more intelligent, efficient, and successful? Is it worthwhile to involve people heavily in the goals of a system? How does a system most effectively coax stories out of people? To investigate these questions, a robot was built that facilitated interaction and documentary gathering within an ubiquitous media framework. We then let the robot roam freely, with the goal of capturing stories about its environment. This was done by leveraging human mobility and intelligence, as the robot relied upon people to move long distances and achieve its goals. The end products were a study of how people related to a robot asking for assistance and interaction in various ways, and a set of movies showing the robot navigating the resulting "thread" of a narrative.

We are working with sponsor Schneider Electric to deploying a dense, low-power wireless sensor network aimed at environmental monitoring for smart energy profiling. This distributed sensor system measures temperature, humidity, and 3D airflow, and transmits this information through a wireless Zigbee protocol. These sensing units are currently deployed in the lower atrium of E14. The data is being used to inform CFD models of airflow in buildings, explore and retrieve valuable information regarding the efficiency of commercial building HVAC systems, energy efficiency of different building materials, and lighting choices in novel architectural designs.

Homes and offices are being filled with sensor networks to answer specific queries and solve pre-determined problems, but no comprehensive visualization tools exist for fusing these disparate data to examine relationships across spaces and sensing modalities. DoppelLab is an immersive, cross-reality virtual environment that serves as an active repository of the multimodal sensor data produced by a building and its inhabitants. We transform architectural models into browsing environments for real-time sensor data visualization and sonification, as well as open-ended platforms for building audiovisual applications atop those data. These applications in turn become sensor-driven interfaces to physical world actuation and control. DoppelLab encompasses a set of tools for parsing, visualization, sonification, and application development, and by organizing data by the space from which they originate, DoppelLab provides a platform to make both broad and specific queries about the activities, systems, and relationships in a complex, sensor-rich environment.

At present, luminous efficacy and cost remain the greatest barriers to broad adoption of LED lighting. However, it is anticipated that within several years, these challenges will be overcome. While we may think our basic lighting needs have been met, this technology offers many more opportunities than just energy efficiency: this research attempts to alter our expectations for lighting and cast aside our assumptions about control and performance. We will introduce new, low-cost sensing modalities that are attuned to human factors such as user context, circadian rhythms, or productivity, and integrate these data with atypical environmental factors to move beyond traditional lux measurements. To research and study these themes, we are focusing on the development of superior color-rendering systems, new power topologies for LED control, and low-cost multimodal sensor networks to monitor the lighting network as well as the environment.

Funk2 is a novel process-description language that keeps track of everything that it does. Remembering these causal execution traces allows parallel threads to reflect, recognize, and react to the history and status of other threads. Novel forms of complex, adaptive, nonlinear control algorithms can be written in the Funk2 programming language. Currently, Funk2 is implemented to take advantage of distributed grid processors consisting of a heterogeneous network of computers, so that hundreds of thousands of parallel threads can be run concurrently, each using many gigabytes of memory. Funk2 is inspired by Marvin Minsky's Critic-Selector theory of human cognitive reflection.

We are developing an opt-in camera network, in which users carrying wearable tags are visible to the network and everyone else is invisible. Existing systems for configurable dynamic privacy in this context are opt-out and catch-all; users desiring privacy carry pre-registered tags that disable sensing and networked media services for everyone in the room. To address these issues, we separate video into layers of flexible sprites representing each person in the field of view, and transmit video of only those who opt-in. Our system can also define groups of users who can be dialed in and out of the video stream dynamically. For cross-reality applications, these dynamic layers achieve a new level of video granularity, allowing users and groups to uncover correspondences between their activities across spaces.

We are developing a system for inferring safety context on construction sites by fusing data from wearable devices, distributed sensing infrastructure, and video. Wearable sensors stream real-time levels of dangerous gases, dust, noise, light quality, precise altitude, and motion to base stations that synchronize the mobile devices, monitor the environment, and capture video. Context mined from these data is used to highlight salient elements in the video stream for monitoring and decision support in a control room. We tested our system in a initial user study on a construction site, instrumenting a small number of steel workers and collecting data. A recently completed hardware revision will be followed by further user testing and interface development.