Blossom is a multiperson awareness system that uses ioMaterials-based techniques to connect distant friends and family. It provides an awareness medium that does not rely on the attention- and reciprocity-demanding interfaces that are provided by digital communication media such as mobile phones, SMS, and email. Combining touch-based input with visual, haptic, and motile feedback, Blossoms are created as pairs that can communicate over the network, echoing the conditions of each other and forming an implicit, always-there link that physically express awareness, while retaining the instantaneous capabilities that define digital communication.

Cornucopia is a concept design for a personal food factory, bringing the versatility of the digital world to the realm of cooking. In essence, it is a 3D printer for food that works by storing, precisely mixing, depositing, and cooking layers of ingredients. Cornucopia's cooking process starts with an array of food canisters that refrigerate and store a user's favorite ingredients. These are piped into a mixer and extruder head that can accurately deposit elaborate combinations of food; while this takes place, the food is heated or cooled. This fabrication process not only allows for the creation of flavors and textures that would be completely unimaginable through other cooking techniques, but it also allows the user to have ultimate control over the origin, quality, nutritional value, and taste of every meal.

DepthJS is a framework that allows any web page to interact with Microsoft Kinect via Javascript. Navigating the web is only one application of the framework - we envision all sorts of applications that run in the browser, from games to specific utilities for specific sites. Using DepthJS, web developers who specialize in Javascript can work with Kinect without having to learn any special languages or code. We hope this will allow a new set of interactions beyond what was first developed.

We believe that in the near future many portable devices will have resizable displays. This will allow for devices with a very compact form factor, which can unfold into a large display when needed. In this project, we design and study novel interaction techniques for devices with flexible, rollable, and foldable displays. We explore a number of scenarios, including personal and collaborative uses.

Despite the advances and advantages of computer-aided design tools, the traditional pencil and paper continue to exist as the most important tools in the early stages of design. Inktuitive is an intuitive physical design workspace that aims to bring together conventional design tools such as paper and pencil with the power and convenience of digital tools for design. Inktuitive also extends the natural work-practice of using physical paper by giving the pen the ability to control the design in physical, 3-D, freeing it from its tie to the paper. The intuition of pen and paper are still present, but lines are captured and translated into shapes in the digital world. The physical paper is augmented with overlaid digital strokes. Furthermore, the platform provides a novel interaction mechanism for drawing and designing using above the surface pen movements.

ioMaterials is a project encompassing a variety of collocated sensing-actuation platforms. The project explores various aspects of dense sensing for humane communication, memory, and remote awareness. Using dense collocated sensing actuation and sensing, we can change common objects into an interface capable of hiding unobtrusively in plain sight. Relational Pillow and Blossom are instantiations of this ideal.

We are experimenting with systems that blur the boundary between urban lighting and digital displays in public spaces. These systems consist of liberated pixels, which are not confined to rigid frames as are typical urban screens. Liberated pixels can be applied to existing horizontal and vertical surfaces in any configuration, and communicate with each other to enable a different repertoire of lighting and display patterns. We have developed Urban Pixels a wireless infrastructure for liberated pixels. Composed of autonomous units, the system presents a programmable and distributed interface that is flexible and easy to deploy. Each unit includes an on-board battery, RF transceiver unit, and microprocessor. The goal is to incorporate renewable energy sources in future versions.

LuminAR reinvents the traditional incandescent bulb and desk lamp, evolving them into a new category of robotic, digital information devices. The LuminAR Bulb combines a Pico-projector, camera, and wireless computer in a compact form factor. This self-contained system enables users with just-in-time projected information and a gestural user interface, and it can be screwed into standard light fixtures everywhere. The LuminAR Lamp is an articulated robotic arm, designed to interface with the LuminAR Bulb. Both LuminAR form factors dynamically augment their environments with media and information, while seamlessly connecting with laptops, mobile phones, and other electronic devices. LuminAR transforms surfaces and objects into interactive spaces that blend digital media and information with the physical space. The project radically rethinks the design of traditional lighting objects, and explores how we can endow them with novel augmented-reality interfaces.

Mouseless is an invisible computer mouse that provides the familiarity of interaction of a physical mouse without actually needing a real hardware mouse. Despite the advances in computing hardware technologies, the two-button computer mouse has remained the predominant means to interact with a computer. Mouseless removes the requirement of having a physical mouse altogether, but still provides the intuitive interaction of a physical mouse with which users are familiar.

MTM "Little John" is a multi-purpose, mid-size, rapid prototyping machine with the goal of being a personal fabricator capable of performing a variety of tasks (3D printing, milling, scanning, vinyl cutting) at a price point in the hundreds rather than thousands of dollars. The machine was designed and built in collaboration with the MTM—Machines that Make Project at MIT Center for Bits and Atoms.

PoCoMo is an implementation of a vision in future-projected social interfaces. In this project we try to capture the playfulness of collaborative gaming and apply it to projected interfaces. The maturing of handheld micro-projector technology, in conjunction with advanced mobile environments, enable this novel type of interaction. Our system is made of a micro-projector mobile device with a specially designed case that turns it into a first-of-a-kind handheld mini-projector-camera system. Computer Vision algorithms support collaborative interaction between multiple users of the system. Through PoCoMo, we wish to explore the social nature of projected interfaces. To accommodate this we designed the projection to be of human cartoon-like characters that play out a personal interaction. Following their human controllers, they recognize each other, wave hello, shake hands, and exchange presents.

Pulp-Based Computing is a series of explorations that combine smart materials, papermaking, and printing. By integrating electrically active inks and fibers during the papermaking process, it is possible to create sensors and actuators that behave, look, and feel like paper. These composite materials not only leverage the physical and tactile qualities of paper, but can also convey digital information, spawning new and unexpected application domains in ubiquitous and pervasive computing at extremely affordable costs.

ReflectOns are objects that help people think about their actions and change their behavior based on subtle, ambient nudges delivered at the moment of action. Certain tasks—such as figuring out the number of calories consumed, or amount of money spent eating out—are generally difficult for the human mind to grapple with. By using in-place sensing combined with gentle feedback and understanding of users' goals, we can recognize behaviors and trends, and provide a reflection of their own actions tailored to enable both better understanding of the repercussions of those actions, and changes to their behaviors to help them better match their own goals.

Sensei is a mobile interface for language learning (words, sentences, pronunciation). It combines techniques from computer vision, augmented reality, speech recognition, and commonsense knowledge. In the current prototype, the user points his cell phone at an object and then sees the word and hears it pronounced in the language of his choice. The system also shows more information pulled from a commonsense knowledge base. The interface is primarily designed to be used as an interactive and fun language-learning tool for children. Future versions will be applied to other contexts such as real-time language translation for face-to-face communication and assistance to travelers for reading information displays in foreign languages; in addition, future versions will provide feedback to users about whether they are pronouncing words correctly. The project is implemented on a Samsung Galaxy phone running Android, donated by Samsung Corporation.

Siftables are compact electronic devices with motion sensing, graphical display, and wireless communication. One or more Siftables may be physically manipulated to interact with digital information and media. A group of Siftables can thus act in concert to form a physical, distributed, gesture-sensitive, human-computer interface. Each Siftable object is stand-alone (battery-powered and wireless); Siftables do not require installed infrastructure such as large displays, instrumented tables, or cameras in order to be used. Siftables' key innovation is to give direct physical embodiment to information items and digital media content, allowing people to use their hands and bodies to manipulate these data instead of relying on virtual cursors and windows. By leveraging people’s ability to manipulate physical objects, Siftables radically simplify the way we interact with information and media.

Information is often confined to paper or computer screens. SixthSense frees data from these confines and seamlessly integrates information and reality. With the miniaturization of computing devices, we are always connected to the digital world, but there is no link between our interactions with these digital devices and our interactions with the physical world. SixthSense bridges this gap by augmenting the physical world with digital information, bringing intangible information into the tangible world. Using a projector and camera worn as a pendant around the neck, SixthSense sees what you see and visually augments surfaces or objects with which you interact. It projects information onto any surface or object, and allows users to interact with the information through natural hand gestures, arm movements, or with the object itself. SixthSense makes the entire world your computer.

Spotlight is about an artist's ability to create a new meaning using the combination of interactive portraits and diptych or polyptych layouts. The mere placement of two or more portraits near each other is a known technique to create a new meaning in the viewer's mind. Spotlight takes this concept into the interactive domain, creating interactive portraits that are aware of each other's state and gesture. So not only the visual layout, but also the interaction with others creates a new meaning for the viewer. Using a combination of interaction techniques, Spotlight engages the viewer at two levels. At the group level, the viewer influences the portrait's "social dynamics." At the individual level, a portrait's "temporal gestures" expose much about the subject's personality.

Surflex is a programmable surface for the design and visualization of physical objects and spaces. It combines the different memory and elasticity states of its materials to deform and gain new shapes, providing a novel alternative for 3-D fabrication and the design of physically adaptable interfaces.

TaPuMa is a digital, tangible, public map that allows people to use everyday objects they carry to access relevant, just-in-time information and to find locations of places or people. TaPuMa envisions that conventional maps can be augmented with the unique identities and affordances of the objects. TaPuMa uses an environment where map and dynamic content is projected on a tabletop. A camera mounted above the table identifies and tracks the locations of the objects on the surface, and a software program identifies and registers the location of objects. After identifying the objects, the software provides relevant information visualizations directly on the table. The projector augments both object and table with projected digital information. TaPuMa explores a novel interaction mechanism where physical objects are used as interfaces to digital information. It allows users to acquire information through tangible media, the things they carry.

thirdEye is a new technique that enables multiple viewers to see different things on a same display screen at the same time. With thirdEye: a public sign board can show a Japanese tourist instructions in Japanese and an American in English; games won't need a split screen anymore—each player can see his or her personal view of the game on the screen; two people watching TV can watch their favorite channel on a single screen; a public display can show secret messages or patterns; and in the same movie theater, people can see different ends of a suspense movie.

VisionPlay is a framework to support the development of augmented play experiences for children. We are interested in exploring mixed reality applications enabled by web cameras, computer vision techniques, and animation that are more socially oriented and physically engaging. These include using physical toys to control digital characters, augmenting physical play environments with projection, and merging representations of the physical world with virtual play spaces.