Research Group Projects and Descriptions

The Responsive Environments group explores how sensor networks augment and mediate human experience, interaction and perception, while developing new sensing modalities and enabling technologies that create new forms of interactive experience and expression. Our current research encompasses the development and application of various types of sensor networks, energy harvesting and power management, and the technical foundation of ubiquitous computing. Our work is highlighted in diverse application areas, which have included automotive systems, smart highways, medical instrumentation, RFID, wearable computing, and interactive media.

We are developing an extremely low-power and low-cost wireless sensor network aimed at applications in asset tracking and ubiquitous activity monitoring. In addition to identifying an object, these nodes (termed active radio-frequency identification [RFID] tags) employ sensors to monitor its state, enabling new applications in fields like security, home automation, and supply-chain management. Although they contain a battery, these tags are not limited by it: by minimizing power consumption and quasi-passively waking on diverse stimuli (changes in light, RF carrier presence, shock or acceleration, and sound), they can last for years. Furthermore, their low cost and small size make them a good candidate for large-scale experiments at the intersection of RFID and wireless sensor networks.

Alumni Contributor(s): Mateusz Malinowski

This research focuses on developing a smart material capable of sensing, distilling, and interpreting environmental stimuli while offering mechanical flexibility. The device itself is composed of a series of small, interconnectable nodes. Each node has its own embedded processing and a host of multi-modal sensors. Physically, the system's nodular design allows for scalability as well as customize-ability. Computationally, the design allows researchers to experiment with resource allocation, information exchange, power management, and differing scopes of processing in sensor networks.

Alumni Contributor(s): Gerardo Barroeta Perez

Funk2 is a novel process description language that keeps track of everything that it does. Remembering these causal execution traces allows parallel threads to reflect, recognize, and react to the history and status of other threads. Novel forms of complex, adaptive, nonlinear control algorithms can be written in the Funk2 programming language. Currently, Funk2 is implemented to take advantage of distributed grid processors consisting of a heterogeneous network of computers, so that hundreds of thousands of parallel threads can be run concurrently, each using many gigabytes of memory. Funk2 is inspired by Marvin Minsky's Critic-Selector theory of human cognitive reflection, and is the foundation for the Neural Models of Mind project.

This is a suite of devices and protocols to support applications in wearable human/social sensing linked to a distributed camera and vision system. The current system includes a sensate wristwatch with biological and gestural sensors, a lapel-pin device with motion and audio affect sensing, and a wall-mounted device with a high-resolution camera, environmental sensors, and a localization system for all devices in the network. All devices record data and audio in sync with the recorded video. A full-spec Zigbee network supports device synchronization and mesh networking. All devices have enough on-board power to extract features from the data.

This project addresses human cognitive models of reflective problem solving in terms of psychology, neuroscience, and artificial intelligence. A programming language describing reflective human thought processes is being developed for the purpose of understanding the biological process of thought. This description language allows distributed reflective monitoring and control of parallel threads. In addition to being a novel method for the robust control of distributed computer programs, this technology is directed toward consumer HCI and medical cures for neuropsychological problems, and has applications for neural-interface computer gaming peripherals, aging population cognitive evaluation, and training.

This project focuses on the development of a low-cost system to track the precise 3-D position of large numbers of objects tagged with passive microwave RF transponders at short-range (3-100m) and in real time. The proposed system will allow selection, identification, tracking, and encoding of data from connected sensors to be relayed by the passive tags. These tags can address a multitude of applications, including new human-computer interfaces, robot control, animation and virtual reality, gaming, low-cost tracking of machinery and animals, search and rescue, and warehouse monitoring.

Roboverse is a physical artificial intelligence simulation. The environment supports rigid body physical simulation of wheeled and legged robots with default algorithms for handling the basic reactive control layers to move from one location to another by straight line, to observe objects in the local neighborhood of the robot's 2D position, to pick up, translate and rotate objects by using a servo interface. Higher level cognitive functions, such as speech and social problem solving were developed as part of Push Singh's cognitive architecture design, the Emotion Machine v1.0 (EM-1).

Alumni Contributor(s): Push Singh

Wide-scale distribution of low-power, low-cost sensor nodes that can measure temperature, humidity, CO2 content, light levels, and human presence enables buildings to react quickly and effectively to the changing needs of their inhabitants. Total building power consumption can be reduced, and repair requests can be made automatically.

Spinner is a Lab-wide sensor network platform designed to detect and capture fragmented events of human behavior that can be collected and sequenced into a cohesive narrative conveying a larger overall meaning. This project also looks at the development of parametric models of narrative that can be mapped on to sensor-detectable elements of human activity.

Tricorder Net is a handheld sensor-network interface inspired by Star Trek's tricorder, a device that provided Science Officer Spock with information about the environment by simply pointing the device and twiddling its knobs. However, unlike Star Trek's tricorder, which had sensors embedded in the handheld device, Tricorder Net's intelligence comes from querying a sensor network distributed throughout the environment.

Alumni Contributor(s): Cameron Lewis and Joshua Lifton

This project is a system of compact, wearable, wireless sensor nodes, equipped with full six degree-of-freedom inertial measurement units and node-to-node capacitive proximity sensing. A high-bandwidth, channel-shared RF protocol has been developed to acquire data from many (e.g., 25) of these sensors at 100 Hz full-state update rates, and software is being developed to fuse this data into a compact set of descriptive parameters in real time. A base station and central computer clock the network and process received data. We aim to capture and analyze the physical movements of multiple people in real time, using unobtrusive sensors worn on the body. Applications abound in biomotion analysis, sports medicine, health monitoring, interactive exercise, immersive gaming, and interactive dance ensemble performance.

Alumni Contributor(s): Ryan Aylward and Mathew Laibowitz