Tangible Media
How to design seamless interfaces between humans, digital information, and the physical environment.

We live between two worlds: our physical environment and cyberspace. The Tangible Media group's focus is on the design of seamless interfaces between humans, digital information, and the physical environment. People have developed sophisticated skills for sensing and manipulating our physical environments. However, most of these skills are not employed by traditional GUIs (Graphical User Interfaces). The Tangible Media group is designing a variety of "tangible interfaces" based on these skills by giving physical form to digital information, seamlessly coupling the dual worlds of bits and atoms. The goal is to change the "painted bits" of GUIs to "tangible bits," taking advantage of the richness of multimodal human senses and skills developed through our lifetime of interaction with the physical world.

Research Projects

  • Beyond: A Collapsible Input Device for 3D Direct Manipulation

    Jinha Lee and Hiroshi Ishii

    Beyond is a collapsible input device for direct 3D manipulation. When pressed against a screen, Beyond collapses in the physical world and extends into the digital space of the screen, so that users have an illusion that they are inserting the tool into the virtual space. Beyond allows users to interact directly with 3D media without having to wear special glasses, avoiding inconsistencies of input and output. Users can select, draw, and sculpt in 3D virtual space, and seamlessly transition between 2D and 3D manipulation.

  • FocalSpace

    Hiroshi Ishii, Anthony DeVincenzi and Lining Yao

    FocalSpace is a system for focused collaboration utilizing spatial depth and directional audio. We present a space where participants, tools, and other physical objects within the space are treated as interactive objects that can be detected, selected, and augmented with metadata. Further, we demonstrate several scenarios of interaction as concrete examples. By utilizing diminishing reality to remove unwanted background surroundings through synthetic blur, the system aims to attract participant attention to foreground activity.

  • IdeaGarden

    Hiroshi Ishii, David Lakatos, and Lining Yao

    The IdeaGarden allows participants of creative activities to collectively capture, select, and share (CCSS) the stories, sketches, and ideas they produce in physical and digital spaces. The iGarden attempts to optimize the CCSS loop and to bring it from hours to seconds in order to turn asynchronous collaborative thought processes into synchronous real-time cognitive flows. The iGarden system is composed of a tangible capturing system including recording devices always "at-hand", of a selection workflow that allows the group to reflect and reduce the complexity of captured data in real-time and of a sharing module that connects socially selected information to the cloud.

  • Kinected Conference

    Anthony DeVincenzi, Lining Yao, Hiroshi Ishii and Ramesh Raskar

    How could we enhance the experience of video-conference by utilizing an interactive display? With a Kinect camera and sound sensors, we explore how expanding a system's understanding of spatially calibrated depth and audio alongside a live video stream can generate semantically rich three-dimensional pixels, containing information regarding their material properties and location. Four features have been implemented: Talking to Focus, Freezing Former Frames, Privacy Zone, and Spacial Augmenting Reality.

  • Meld

    Hiroshi Ishii, Andy Lippman, Matthew Blackshaw, Anthony DeVincenzi and David Lakatos

    Meld provides a new perspective on your social life. By presenting your social graph as a moving picture, Meld breaks free from the text-centric interfaces of today's social networks, offering a fresh, holistic perspective. Unseen trends, before lost in mountains of text, can be better understood, providing an organic and evolving view of your relationships. Meld is a semi-finalist in the MIT 100K Entrepreneurship Competition.

  • MirrorFugue

    Xiao Xiao and Hiroshi Ishii

    MirrorFugue is an interface for the piano that bridges the gap of location in music playing by connecting pianists in a virtual shared space reflected on the piano. Built on a previous design that only showed the hands, our new prototype displays both the hands and upper body of the pianist. MirrorFugue may be used for watching a remote or recorded performance, taking a remote lesson, and remote duet playing.

  • Peddl

    Andy Lippman, Hiroshi Ishii, Matthew Blackshaw, Anthony DeVincenzi and David Lakatos

    Today, marketplaces like Craigslist and eBay connect people who buy and sell goods and services without leveraging the power of mobility, localized posting, and community. With GPS- and Internet-enabled mobile devices in almost every pocket, we see a space for a new type of marketplace which takes into account your physical location, availability, and means of negotiation. What new and exciting interactions await for mobile marketplaces in the age of the app? How do you redesign the fundamentals of buying and selling goods or services with the people around you, by focusing on simplicity, trust, and discovery? With Peddl we explore this possibility by creating one of the first broadcast marketplaces.

  • PingPongPlusPlus

    Hiroshi Ishii, Xiao Xiao, Michael Bernstein, Lining Yao, Dávid Lakatos, Kojo Acquah, Jeff Chan, Sean Follmer and Daniel Leithinger

    PingPong++ (PingPongPlusPlus) builds on PingPongPlus (1998), a ping pong table that could sense ball hits, and reuse that data to control visualizations projected on the table. We have redesigned the system using open-source hardware and software platforms so that anyone in the world can build their own reactive table. We are exploring ways that people can customize their ping pong game experience. This kiosk allows players to create their own visualizations based on a set of templates. For more control of custom visualizations, we have released a software API based on the popular Processing language to enable users to write their own visualizations. We are always looking for collaborators! Visit pppp.media.mit.edu to learn more.

  • Radical Atoms

    Hiroshi Ishii, Leonardo Bonanni, Keywon Chung, Sean Follmer, Jinha Lee, Daniel Leithinger and Xiao Xiao

    Radical Atoms is our vision of interactions with future material.

  • Recompose

    Matthew Blackshaw, Anthony DeVincenzi, David Lakatos, and Hiroshi Ishii

    Human beings have long shaped the physical environment to reflect designs of form and function. As an instrument of control, the human hand remains the most fundamental interface for affecting the material world. In the wake of the digital revolution, this is changing, bringing us to reexamine tangible interfaces. What if we could now dynamically reshape, redesign, and restructure our environment using the functional nature of digital tools? To address this, we present Recompose, a framework allowing direct and gestural manipulation of our physical environment. Recompose complements the highly precise, yet concentrated affordance of direct manipulation with a set of gestures, allowing functional manipulation of an actuated surface.

  • Relief

    Hiroshi Ishii and Daniel Leithinger

    Relief is an actuated tabletop display, able to render and animate 3D shapes with a malleable surface. It allows users to experience and form digital models such as geographical terrain in an intuitive manner. The tabletop surface is actuated by an array of motorized pins, which can be addressed individually and sense user input like pulling and pushing. Our current research focuses on utilizing freehand gestures for interacting with content on Relief.

  • RopeRevolution

    Jason Spingarn-Koff (MIT), Hiroshi Ishii, Sayamindu Dasgupta, Lining Yao, Nadia Cheng (MIT Mechanical Engineering) and Ostap Rudakevych (Harvard University Graduate School of Design)

    Rope Revolution is a rope-based gaming system for collaborative play. After identifying popular rope games and activities from around the world, we developed a generalized tangible rope interface that includes a compact motion-sensing and force-feedback module that can be used for a variety of rope-based games, such as rope jumping, kite flying, and horseback riding. Rope Revolution is designed to foster both co-located and remote collaborative experiences by using actual rope to connect players in physical activities across virtual spaces.

  • SandScape

    Carlo Ratti, Assaf Biderman and Hiroshi Ishii

    SandScape is a tangible interface for designing and understanding landscapes through a variety of computational simulations using sand. The simulations are projected on the surface of a sand model representing the terrain; users can choose from a variety of different simulations highlighting height, slope, contours, shadows, drainage, or aspect of the landscape model, and alter its form by manipulating sand while seeing the resulting effects of computational analysis generated and projected on the surface of sand in real time. SandScape demonstrates an alternative form of computer interface (tangible user interface) that takes advantage of our natural abilities to understand and manipulate physical forms while still harnessing the power of computational simulation to help in our understanding of a model representation.

  • Sensetable

    James Patten, Jason Alonso and Hiroshi Ishii

    Sensetable is a system that wirelessly, quickly, and accurately tracks the positions of multiple objects on a flat display surface. The tracked objects have a digital state, which can be controlled by physically modifying them using dials or tokens. We have developed several new interaction techniques and applications on top of this platform. Our current work focuses on business supply-chain visualization using system-dynamics simulation.

  • Sourcemap

    Hiroshi Ishii, Leonardo Bonanni and Matthew Hockenberry

    Sourcemap is a crowd-sourced directory of product supply chains and carbon footprints. The free and open-source website offers a suite of tools to visualize where products come from, to calculate their social and environmental impact, and to share this information across social media. Since 2009, the website has gathered thousands of user-generated sourcemaps of foods, furniture, clothing, electronics and more. The Sourcemap team partners with for- and non-profit organizations to bring visibility into their supply chains and to connect consumers with the source of products. Sourcemap has been featured by the BBC, NPR, the Globe and Mail and the Huffington Post and has received awards from Scientific American, Ars Electronica and ID magazine.

  • T(ether)

    Hiroshi Ishii, Andy Lippman, Matthew Blackshaw and David Lakatos

    T(ether) is a novel spatially aware display that supports intuitive interaction with volumetric data. The display acts as a window affording users a perspective view of three- dimensional data through tracking of head position and orientation. T(ether) creates a 1:1 mapping between real and virtual coordinate space allowing immersive exploration of the joint domain. Our system creates a shared workspace in which co-located or remote users can collaborate in both the real and virtual worlds. The system allows input through capacitive touch on the display and a motion-tracked glove. When placed behind the display, the user’s hand extends into the virtual world, enabling the user to interact with objects directly.

  • Tangible Bits

    Hiroshi Ishii, Sean Follmer, Jinha Lee, Daniel Leithinger and Xiao Xiao

    People have developed sophisticated skills for sensing and manipulating our physical environments, but traditional GUIs (Graphical User Interfaces) do not employ most of them. Tangible Bits builds upon these skills by giving physical form to digital information, seamlessly coupling the worlds of bits and atoms. We are designing "tangible user interfaces" that employ physical objects, surfaces, and spaces as tangible embodiments of digital information. These include foreground interactions with graspable objects and augmented surfaces, exploiting the human senses of touch and kinesthesia. We also explore background information displays that use "ambient media"—light, sound, airflow, and water movement—to communicate digitally mediated senses of activity and presence at the periphery of human awareness. We aim to change the "painted bits" of GUIs to "tangible bits," taking advantage of the richness of multimodal human senses and skills developed through our lifetimes of interaction with the physical world.

  • Topobo

    Hayes Raffle, Amanda Parkes and Hiroshi Ishii

    Topobo is a 3-D constructive assembly system embedded with kinetic memory—the ability to record and play back physical motion. Unique among modeling systems is Topobo’s coincident physical input and output behaviors. By snapping together a combination of passive (static) and active (motorized) components, users can quickly assemble dynamic, biomorphic forms such as animals and skeletons, animate those forms by pushing, pulling, and twisting them, and observe the system repeatedly playing back those motions. For example, a dog can be constructed and then taught to gesture and walk by twisting its body and legs. The dog will then repeat those movements.

  • Video Play

    Sean Follmer, Hayes Raffle and Hiroshi Ishii

    Long-distance families are increasingly staying connected with free video conferencing tools. However, the tools themselves are not designed to accommodate children's or families' needs. We explore how play can be a means for communication at a distance. Our Video Play prototypes are simple video-conferencing applications built with play in mind, creating opportunities for silliness and open-ended play between adults and young children. They include simple games, such as Find It, but also shared activities like book reading, where users' videos are displayed as characters in a story book.

  • Wetpaint

    Leonardo Bonanni, Xiao Xiao, Bianca Costanzo, Andrew Shum, Matthew Hockenberry, Maurizio Seracini and Hiroshi Ishii

    The Wetpaint project investigates new interfaces for exploring the history of a work of visual art. We are seeking intuitive metaphors for touch-screen interaction, including virtually scraping through the multispectral scans of an ancient painting, and pulling, stretching, and tearing through a virtual canvas. The interaction techniques refined through Wetpaint are being built into a Web-based tool for leveraging collective intelligence toward the pursuit of art diagnostics.