Research Group Projects and Descriptions

We live between two worlds: our physical environment and cyberspace. The Tangible Media group's focus is on the design of seamless interfaces between humans, digital information, and the physical environment. People have developed sophisticated skills for sensing and manipulating our physical environments. However, most of these skills are not employed by traditional GUIs (Graphical User Interfaces). The Tangible Media group is designing a variety of "tangible interfaces" based on these skills by giving physical form to digital information, seamlessly coupling the dual worlds of bits and atoms. The goal is to change the "painted bits" of GUIs to "tangible bits," taking advantage of the richness of multimodal human senses and skills developed through our lifetime of interaction with the physical world.

Dewy is a display surface of ‘pixelized’ condensation, working like a spatially controlled fogged window, one that can communicate back to you with words and patterns. Slow and subtle in behavior, Dewy offers an alternative method of display which can blend seamlessly into varying environments in its materiality and temporality. Inspired by natural state change processes of matter, Dewy attempts to challenge the visual pollution of many existing systems of public media display, opening a dialogue on the physicality of spatial media design and digital control.

GrabIt allows users to collaboratively explore the principles of power generation, transfer, and consumption via tangible wooden sticks. Pushing on one stick generates power that is transfered to other sticks in real time allowing users to see, and feel, the effects of losses and electrical loading. All energy in the GrabIt system is derived from its users and therefore makes the mapping between input, output, and losses transparent. Variable-color light bulbs can also be connected to the system to serve as simplified voltage indicators during use.

Tangible User Interfaces are well-suited to handling three-dimensional data sets by direct manipulation of real objects in space, but current interfaces can make it difficult to look inside dense volumes of information. The HandSaw system detects a virtual cut-plane projected by an outstretched hand or laser-line directly on an object or space and reveals sectional data on an adjacent display. By leaving the hands free and using a remote display, these techniques can be shared among multiple users and integrated into everyday practice. HandSaw has been prototyped for scientific visualizations in medicine, engineering, and urban design. User evaluations suggest that using a hand is more intuitive, while projected light is more precise than keyboard and mouse control, and the HandSaw system has the potential to be used more effectively by novices and in groups.

We have created a system that allows a person's heart rate to be displayed on an ambient orb color display. This system uses a heart-rate sensor developed at the Media Lab. This system has many applications, including baby monitors, home health-care monitoring, company meeting tools, and novel user interfaces or games.

Jabberstamp is the first tool that allows children to synthesize drawings and voices. To use Jabberstamp, children create drawings, collages, or paintings on normal paper, and then press a special rubber stamp onto the page to record sounds into their drawings. When they touch the marks of the stamp with a small trumpet, the sounds play back, retelling the stories they have created. Jabberstamp can be used to embed names, narratives, characters’ voices, and environmental sound effects into drawings, helping children to communicate their stories with peers and adults, and allowing them to record and situate stories in personally meaningful contexts to share with others—before they have mastered writing.

Kinetic Sketch-Up is a system for prototyping motion and kinetic transformability into interfaces. A series of actuated modules allow users to embed programmable actuation into tangible interfaces, dynamically coupling digital information with self-reconfigurable structures. Modules in the systems combine varied materials and technologies for actuation (electro-mechanical, shape memory) creating different forms of motion. This provides a method to prototype perceptual qualities which motion can elicit in the design of an interface: drawing our attention, providing physical feedback, and conveying information through physical change. As interaction design begins to incorporate many of the interaction principles of robotics, the system provides a new design tool and vocabulary for motion construction in the emerging areas of transformable products, environments, and architecture.

Meshotic is a meshed surface with kinetic memory, the ability to record and play back physical motion. Based on the record and play technology of Topobo, servomotor links move a series of nodes to define a set of points in space, whose positions are recorded over time with a distributed network of on-board circuitry. The system also links to an on-screen model to allow for actuated feedback while sculpting a 3-D model.

Picture This! allows algorithmic film assembly using toy gestures. It is a new input device for video capturing and editing. Designed for young children, ages five and up, it allows them to craft compelling movies through the motion analysis of their interaction with toys. Children’s favorite props alternate between characters and cameramen in a film. As children play with the toys to act out a story, they conduct algorithmic film assembly. A new genre of Gesture Object Interfaces as exemplified by Picture This relies on the analysis of gestures coupled with objects to represent bits.

Piezing is an outfit which generates power using the natural gestures of the human body in motion. Around the joints of the elbows and hips, the garment is embedded with piezoelectric material elements which generate an electric potential in response to applied mechanical stress. The electric potential is then stored as voltage in a centralized small battery and later can be discharged into a device.

Psychohaptics explores a diverse set of simulations of touch modalities to assist in the treatment of specific mental disorders. Based on the most promising touch-therapy protocols, we have developed haptic interfaces for psychotherapy that we aim to test in a local hospital.

SandScape is a tangible interface for designing and understanding landscapes through a variety of computational simulations using sand. The simulations are projected on the surface of a sand model representing the terrain; users can choose from a variety of different simulations highlighting height, slope, contours, shadows, drainage, or aspect of the landscape model, and alter its form by manipulating sand while seeing the resulting effects of computational analysis generated and projected on the surface of sand in real time. SandScape demonstrates an alternative form of computer interface (tangible user interface) that takes advantage of our natural abilities to understand and manipulate physical forms while still harnessing the power of computational simulation to help in our understanding of a model representation.

Alumni Contributor(s): Yao Wang and Ben Piper

Sensetable is a system that wirelessly, quickly, and accurately tracks the positions of multiple objects on a flat display surface. The tracked objects have a digital state, which can be controlled by physically modifying them using dials or tokens. We have developed several new interaction techniques and applications on top of this platform. Our current work focuses on business supply-chain visualization using system-dynamics simulation.

Alumni Contributor(s): Dan Chak, Gian Antonio Pangaro and Matt Reynolds

Architects and video game designers use increasingly similar tools to design immersive environments; and there is a growing need for physical environments to have an analogous digital representation – an on-line ‘shadow.’ How can physical and virtual design merge into a seamless practice? We present the Shadowbox, a physical-digital workbench for the design of physical environments and their digital analogues that builds on architectural practice and virtual environment design. We have developed this tool for the design of museum exhibits that exist both on-line and off-. We introduce a novel interaction technique that places elements and textures in a physical workbench through laser-pointer gestures common to presentations. Interviews and pilot evaluations of the Shadowbox techniques suggest that direct embedding of digital characteristics in physical design models can be more intuitive and rapid than GUI-based design.

Slurp is a tangible interface for manipulating abstract digital information as if it were water. Taking the form of an eyedropper, Slurp can extract (slurp up) and inject (squirt out) pointers to digital objects. We have created Slurp to explore the use of physical metaphor, feedback, and affordances in tangible interface design when working with abstract digital media types. Our goal is to privilege spatial relationships between devices and people while providing new physical manipulation techniques for ubiquitous computing environments. Current applications include tangible/graphical mediation, locative-media, multi-display reaching, and physicalization for online retail.

Modern technology is made possible by vast global supply chains with profound environmental impacts—but society remains uninformed about the provenance of products and materials. Sourcemap is a research and visualization technique to document the global reach of production utilizing Web-based tools. Based on Design for the Environment, the system encourages the disclosure of constituent materials in order to estimate global environmental impact through life cycle assessment. We present the technique in the hope of engaging designers in transparent practices toward sustainable industrial production.

SpeakCup is a digital voice recorder. Rather than being controlled via buttons and blinking lights, SpeakCup uses shape to proscribe function. It takes the form of a small rubber disc with holes in its center on one side. When the holes are pressed in, forming a small cup, SpeakCup absorbs and contains sound. When the holes are pressed out, the stored sounds (embodied as fluxuating lights) are released.

Tangible Bits is our vision of Human Computer Interaction (HCI). People have developed sophisticated skills for sensing and manipulating our physical environments, but traditional GUIs (Graphical User Interfaces) do not employ most of these skills. Tangible Bits builds upon these skills by giving physical form to digital information, seamlessly coupling the dual worlds of bits and atoms. We are designing "tangible user interfaces" that employ physical objects, surfaces, and spaces as tangible embodiments of digital information. These include foreground interactions with graspable objects and augmented surfaces, exploiting the human senses of touch and kinesthesia. We are also exploring background information displays that use "ambient media": ambient light, sound, airflow, and water movement. Here, we seek to communicate digitally mediated senses of activity and presence at the periphery of human awareness. The goal is to change the "painted bits" of GUIs to "tangible bits," taking advantage of the richness of multimodal human senses and skills developed through our lifetimes of interaction with the physical world.

Alumni Contributor(s): Yao Wang, Mike Ananny, Scott Brave, Dan Chak, Angela Chang, Seung-Ho Choo, Andrew Dahley, Philipp Frei, Matthew G. Gorbet, Vincent Leclerc, Jae-Chol Lee, Dan Maynes-Aminzade, Ali Mazalek, Gian Antonio Pangaro, Ben Piper, Sandia Ren, Kimiko Ryokai, Victor Su, Brygg Ullmer, Craig Wisneski and Paul Yarin

Topobo is a 3-D constructive assembly system embedded with kinetic memory—the ability to record and play back physical motion. Unique among modeling systems is Topobo’s coincident physical input and output behaviors. By snapping together a combination of passive (static) and active (motorized) components, users can quickly assemble dynamic, biomorphic forms such as animals and skeletons, animate those forms by pushing, pulling, and twisting them, and observe the system repeatedly playing back those motions. For example, a dog can be constructed and then taught to gesture and walk by twisting its body and legs. The dog will then repeat those movements.

A work of art rarely reveals the history of creation and interpretation that has given it meaning and value. Wetpaint is a gallery interface based on a large touch screen that allows curators and museumgoers to investigate the hidden layers of a painting, and in the process contribute to the pluralistic interpretation of the piece, both locally and online. Inspired by traditional restoration and curation methods, we have designed a touch-based user interface for exhibition spaces to allow "virtual restoration" by scraping through the multi-spectral scans of a painting, and "collaborative curation" by leaving voice annotations within the artwork. The system functions through an online social image network for both flexibility and to support rich and collaborative commentary for local and remote visitors.