Tangible Media
Seamlessly coupling the worlds of bits and atoms by giving dynamic physical form to digital information and computation.
We live between two worlds: our physical environment and cyberspace. The Tangible Media group's focus is on the design of seamless interfaces between humans, digital information, and the physical environment. People have developed sophisticated skills for sensing and manipulating our physical environments. However, most of these skills are not employed by traditional GUIs (Graphical User Interfaces). The Tangible Media group is designing a variety of "tangible interfaces" based on these skills by giving physical form to digital information, seamlessly coupling the dual worlds of bits and atoms. The goal is to change the "painted bits" of GUIs to "tangible bits," taking advantage of the richness of multimodal human senses and skills developed through our lifetime of interaction with the physical world.

Research Projects

  • Andante

    Xiao Xiao and Hiroshi Ishii

    Andante is a representation of music as animated characters walking along the piano keyboard that appear to play the physical keys with each step. Based on a view of music pedagogy that emphasizes expressive, full-body communication early in the learning process, Andante promotes an understanding of the music rooted in the body, taking advantage of walking as one of the most fundamental human rhythms.

  • inFORM

    Special Interest group(s): 
    Hiroshi Ishii, Alex Olwal, Daniel Leithinger and Sean Follmer

    Shape displays can be used to render both 3D physical content and user interface elements. We propose to use shape displays in three different ways to mediate interaction: facilitate, providing dynamic physical affordances through shape change; restrict, guiding users through dynamic physical constraints; and manipulate, actuating passive physical objects on the interface surface. We demonstrate this on a new, high-resolution shape display.

  • jamSheets: Interacting with Thin Stiffness-Changing Material

    Jifei Ou, Lining Yao, Daniel Tauber, Juergen Steimle, Ryuma Niiyama, Hiroshi Ishii

    This project introduces layer jamming as an enabling technology for designing deformable, stiffness-tunable, thin sheet interfaces. Interfaces that exhibit tunable stiffness properties can yield dynamic haptic feedback and shape deformation capabilities. In contrast to particle jamming, layer jamming allows for constructing thin and lightweight form factors of an interface. We propose five-layer structure designs and an approach that composites multiple materials to control the deformability of the interfaces. We also present methods to embed different types of sensing and pneumatic actuation layers on the layer-jamming unit. Through three application prototypes we demonstrate the benefits of using layer jamming in interface design. Finally, we provide a survey of materials that have proven successful for layer jamming.

  • MirrorFugue

    Xiao Xiao and Hiroshi Ishii

    MirrorFugue is an installation for a player piano that evokes the impression that the "reflection" of a disembodied pianist is playing the physically moving keys. Live music emanates from a grand piano, whose keys move under the supple touch of a pianist's hands reflected on the lacquered surface of the instrument. The pianist's face is displayed on the music stand, with subtle expressions projecting the emotions of the music. MirrorFugue recreates the feeling of a live performance, but no one is actually there. The pianist is an illusion of light and mirrors, a ghost both present and absent. Viewing MirrorFugue evokes the sense of walking into a memory, where the pianist plays without awareness of the viewer's presence; or, it is as if viewers were ghosts in another's dream, able to sit down in place of the performing pianist and play along.

  • Physical Telepresence

    Special Interest group(s): 
    Daniel Leithinger, Sean Follmer, Alex Olwal and Hiroshi Ishii

    We propose a new approach to physical telepresence, based on shared workspaces with the ability to capture and remotely render the shapes of people and objects. In the research paper, we describe the concept of shape transmission, and propose interaction techniques to manipulate remote physical objects and physical renderings of shared digital content. We investigate how the representation of users' body parts can be altered to amplify their capabilities for teleoperation. A preliminary evaluation found that users were able to manipulate simple objects remotely, and found many different techniques for manipulation that highlight the expressive nature of our system.

  • Pneumatic Shape-Changing Interfaces

    Jifei Ou, Lining Yao, Ryuma Niiyama, Sean Follmer and Hiroshi Ishii

    An enabling technology to build shape-changing interfaces through pneumatically driven, soft-composite materials. The composite materials integrate the capabilities of both input sensing and active shape output. We explore four applications: a multi-shape mobile device, table-top shape-changing tangibles, dynamically programmable texture for gaming, and a shape-shifting lighting apparatus.

  • Radical Atoms

    Hiroshi Ishii
    Radical Atoms is our vision of interactions with future material. Radical Atoms takes a leap beyond Tangible Bits by assuming a hypothetical generation of materials that can change form and appearance dynamically, becoming as reconfigurable as pixels on a screen. Radical Atoms is a computationally transformable and reconfigurable material that is bidirectionally coupled with an underlying digital model (bits) so that dynamic changes of physical form can be reflected in digital states in real time, and vice versa.
  • TRANSFORM

    Special Interest group(s): 
    Hiroshi Ishii, Sean Follmer, Daniel Leithinger, Philipp Schoessler, Amit Zoran and LEXUS International

    TRANSFORM fuses technology and design to celebrate its transformation from still furniture to a dynamic machine driven by a stream of data and energy. TRANSFORM aims to inspire viewers with unexpected transformations and the aesthetics of the complex machine in motion. First exhibited at LEXUS DESIGN AMAZING MILAN (April 2014), the work comprises three dynamic shape displays that move over one thousand pins up and down in real time to transform the tabletop into a dynamic tangible display. The kinetic energy of the viewers, captured by a sensor, drives the wave motion represented by the dynamic pins. The motion design is inspired by dynamic interactions among wind, water, and sand in nature, Escher's representations of perpetual motion, and the attributes of sand castles built at the seashore. TRANSFORM tells of the conflict between nature and machine, and its reconciliation, through the ever-changing tabletop landscape.