Tangible Media
Seamlessly coupling the worlds of bits and atoms by giving dynamic physical form to digital information and computation.
We live between two worlds: our physical environment and cyberspace. The Tangible Media group's focus is on the design of seamless interfaces between humans, digital information, and the physical environment. People have developed sophisticated skills for sensing and manipulating our physical environments. However, most of these skills are not employed by traditional GUIs (Graphical User Interfaces). The Tangible Media group is designing a variety of "tangible interfaces" based on these skills by giving physical form to digital information, seamlessly coupling the dual worlds of bits and atoms. The goal is to change the "painted bits" of GUIs to "tangible bits," taking advantage of the richness of multimodal human senses and skills developed through our lifetime of interaction with the physical world.

Research Projects

  • Andante

    Xiao Xiao and Hiroshi Ishii

    Andante is a representation of music as animated characters walking along the piano keyboard that appear to play the physical keys with each step. Based on a view of music pedagogy that emphasizes expressive, full-body communication early in the learning process, Andante promotes an understanding of the music rooted in the body, taking advantage of walking as one of the most fundamental human rhythms.

  • bioLogic

    Lining Yao

    This research introduces bacterial endospores as microscale biological actuators to build stimuli-responsive, shape-changing interfaces. We demonstrate the unique programmability of spore actuators for achieving transformations on human scale surfaces through precise spore deposition and geometric pattern designs. We describe the process from wet-lab spore cultivation to machine-shop printing spore solutions, as well as exemplifying their applications. This research intends to contribute to the understanding of the control and programming of natural materials. We hope to encourage interface designers to consider adapting living organisms to build actuation mechanisms and to help grow the nascent field of applying biological technologies to HCI.

  • inFORM

    Special Interest group(s): 
    Hiroshi Ishii, Alex Olwal, Daniel Leithinger and Sean Follmer

    Shape displays can be used to render both 3D physical content and user interface elements. We propose to use shape displays in three different ways to mediate interaction: facilitate, providing dynamic physical affordances through shape change; restrict, guiding users through dynamic physical constraints; and manipulate, actuating passive physical objects on the interface surface. We demonstrate this on a new, high-resolution shape display.

  • jamSheets: Interacting with Thin Stiffness-Changing Material

    Jifei Ou, Lining Yao, Daniel Tauber, Juergen Steimle, Ryuma Niiyama, Hiroshi Ishii

    This project introduces layer jamming as an enabling technology for designing deformable, stiffness-tunable, thin sheet interfaces. Interfaces that exhibit tunable stiffness properties can yield dynamic haptic feedback and shape deformation capabilities. In contrast to particle jamming, layer jamming allows for constructing thin and lightweight form factors of an interface. We propose five-layer structure designs and an approach that composites multiple materials to control the deformability of the interfaces. We also present methods to embed different types of sensing and pneumatic actuation layers on the layer-jamming unit. Through three application prototypes we demonstrate the benefits of using layer jamming in interface design. Finally, we provide a survey of materials that have proven successful for layer jamming.

  • LineFORM

    Ken Nakagaki, Sean Follmer and Hiroshi Ishii

    We propose a novel shape-changing interface that consists of a single line. Lines have several interesting characteristics from the perspective of interaction design: abstractness of data representation; a variety of inherent interactions/affordances; and constraints such as boundaries or borderlines. By utilizing such aspects of lines together with the added transformation capability, we present various applications in different scenarios such as shape-changing cords, mobiles, body constraints, and data manipulation to investigate the design space of line-based shape-changing interfaces.

  • MirrorFugue

    Xiao Xiao and Hiroshi Ishii

    MirrorFugue is an installation for a player piano that evokes the impression that the "reflection" of a disembodied pianist is playing the physically moving keys. Live music emanates from a grand piano, whose keys move under the supple touch of a pianist's hands reflected on the lacquered surface of the instrument. The pianist's face is displayed on the music stand, with subtle expressions projecting the emotions of the music. MirrorFugue recreates the feeling of a live performance, but no one is actually there. The pianist is an illusion of light and mirrors, a ghost both present and absent. Viewing MirrorFugue evokes the sense of walking into a memory, where the pianist plays without awareness of the viewer's presence; or, it is as if viewers were ghosts in another's dream, able to sit down in place of the performing pianist and play along.

  • MMODM: Massively Multiplayer Online Drum Machine

    Joseph A. Paradiso, Tod Machover, Donald Derek H. and Basheer Tome

    MMODM is an online drum machine based on the Twitter streaming API, using tweets from around the world to create and perform musical sequences together in real time. Users anywhere can express 16-beat note sequences across 26 different instruments, using plain-text tweets from any device. Meanwhile, users on the site itself can use the graphical interface to locally DJ the rhythm, filters, and sequence blending. By harnessing this duo of website and Twitter network, MMODM enables a whole new scale of synchronous musical collaboration between users locally, remotely, across a wide variety of computing devices, and across a variety of cultures.

  • Physical Telepresence

    Special Interest group(s): 
    Daniel Leithinger, Sean Follmer, Alex Olwal and Hiroshi Ishii

    We propose a new approach to physical telepresence, based on shared workspaces with the ability to capture and remotely render the shapes of people and objects. In the research paper, we describe the concept of shape transmission, and propose interaction techniques to manipulate remote physical objects and physical renderings of shared digital content. We investigate how the representation of users' body parts can be altered to amplify their capabilities for teleoperation. A preliminary evaluation found that users were able to manipulate simple objects remotely, and found many different techniques for manipulation that highlight the expressive nature of our system.

  • Pneumatic Shape-Changing Interfaces

    Jifei Ou, Lining Yao, Ryuma Niiyama, Sean Follmer and Hiroshi Ishii

    An enabling technology to build shape-changing interfaces through pneumatically driven, soft-composite materials. The composite materials integrate the capabilities of both input sensing and active shape output. We explore four applications: a multi-shape mobile device, table-top shape-changing tangibles, dynamically programmable texture for gaming, and a shape-shifting lighting apparatus.

  • Porobits: Conversations in the Clouds

    Hiroshi Ishii, Dhruv Jain and Viirj Kan

    A Porobit is a murmuring interactive block that enables construction of 3D semi-permeable structures. "Conversations in the Clouds" utilizes Porobits to create an interactive art installation that evokes a physical articulation of data and cloud computing. The installation invites individuals to utter their queries into the structures to inflate the Porobits, contributing data by breathing life into the cloud.

  • Radical Atoms

    Hiroshi Ishii
    Radical Atoms is our vision of interactions with future materials. Radical Atoms takes a leap beyond Tangible Bits by assuming a hypothetical generation of materials that can change form and appearance dynamically, becoming as reconfigurable as pixels on a screen. Radical Atoms is a computationally transformable and reconfigurable material that is bidirectionally coupled with an underlying digital model (bits) so that dynamic changes of physical form can be reflected in digital states in real time, and vice versa.
  • Social Textiles

    Hiroshi Ishii, Pattie Maes, Judith Amores Fernandez, Katsuya Fujii, Viirj Kan and Chang Long Zhu Jin

    Social Textiles embodies who you are and dynamically reflects your shared interests with people nearby. It enables you to gain access to communities of people in the physical world and enhances social affordances and icebreaking interactions through wearable social messaging. Social Textiles can serve to connect community members with niche interests, philosophical beliefs, personalities, emotional statuses, and ethical views. It has the potential to enable members to bypass superficial or generic interests through “filtering” individuals, to tune social experiences toward people who are more compatible.


    Special Interest group(s): 
    Hiroshi Ishii, Sean Follmer, Daniel Leithinger, Philipp Schoessler, Amit Zoran and LEXUS International

    TRANSFORM fuses technology and design to celebrate its transformation from still furniture to a dynamic machine driven by a stream of data and energy. TRANSFORM aims to inspire viewers with unexpected transformations and the aesthetics of the complex machine in motion. First exhibited at LEXUS DESIGN AMAZING MILAN (April 2014), the work comprises three dynamic shape displays that move over one thousand pins up and down in real time to transform the tabletop into a dynamic tangible display. The kinetic energy of the viewers, captured by a sensor, drives the wave motion represented by the dynamic pins. The motion design is inspired by dynamic interactions among wind, water, and sand in nature, Escher's representations of perpetual motion, and the attributes of sand castles built at the seashore. TRANSFORM tells of the conflict between nature and machine, and its reconciliation, through the ever-changing tabletop landscape.

  • TRANSFORM: Adaptive and Dynamic Furniture

    Luke Vink, Viirj Kan, Ken Nakagaki, Daniel Leithinger, Sean Follmer, Philipp Schoessler, Amit Zoran, Hiroshi Ishii

    Introducing TRANSFORM, a shape-changing desk. TRANSFORM is an exploration of how shape display technology can be integrated into our everyday lives as interactive, transforming furniture. These interfaces not only serve as traditional computing devices, but also support a variety of physical activities. By creating shapes on demand or by moving objects around, TRANSFORM changes the ergonomics and aesthetic dimensions of furniture, supporting a variety of use cases at home and work: it holds and moves objects like fruit, game tokens, office supplies, and tablets, creates dividers on demand, and generates interactive sculptures to convey messages and audio.