Beyond is a collapsible input device for direct 3D manipulation. When pressed against a screen, Beyond collapses in the physical world and extends into the digital space of the screen, so that users have an illusion that they are inserting the tool into the virtual space. Beyond allows users to interact directly with 3D media without having to wear special glasses, avoiding inconsistencies of input and output. Users can select, draw, and sculpt in 3D virtual space, and seamlessly transition between 2D and 3D manipulation.

An open publishing platform for visualization, social sharing, and data analysis of geospatial data.

Malleable user interfaces have the potential to enable radically new forms of interactions and expressiveness through flexible, free-form and computationally controlled shapes and displays. This work, specifically focuses on particle jamming as a simple, effective method for flexible, shape-changing user interfaces where programmatic control of material stiffness enables haptic feedback, deformation, tunable affordances and control gain. We introduce a compact, low-power pneumatic jamming system suitable for mobile devices, and a new hydraulic-based technique with fast, silent actuation and optical shape sensing. We enable jamming structures to sense input and function as interaction devices through two contributed methods for high-resolution shape sensing using: 1) index-matched particles and fluids, and 2) capacitive and electric field sensing. We explore the design space of malleable and organic user interfaces enabled by jamming through four motivational prototypes that highlight jamming’s potential in HCI, including applications for tabletops, tablets and for portable shape-changing mobile devices.

MirrorFugue is an interface for the piano that bridges the gap of location in music playing by connecting pianists in a virtual shared space reflected on the piano. Built on a previous design that only showed the hands, our new prototype displays both the hands and upper body of the pianist. MirrorFugue may be used for watching a remote or recorded performance, taking a remote lesson, and remote duet playing.

Peddl creates a localized, perfect market. All offers are broadcasts, allowing users to spot trends, bargains, and opportunities. With GPS- and Internet-enabled mobile devices in almost every pocket, we see an opportunity for a new type of marketplace which takes into account your physical location, availability, and open negotiation. Like other real-time activities, we are exploring transactions as an organizing principle among people that, like Barter, may be strong, rich, and long-lived.

An enabling technology to build shape-changing
interfaces through pneumatically-driven soft
composite materials. The composite materials integrate the
capabilities of both input sensing and active shape output. We explore four applications: a multi-shape mobile device, table-top shape-changing tangibles, dynamically programmable texture for gaming, and shape-shifting lighting apparatus.

Human beings have long shaped the physical environment to reflect designs of form and function. As an instrument of control, the human hand remains the most fundamental interface for affecting the material world. In the wake of the digital revolution, this is changing, bringing us to reexamine tangible interfaces. What if we could now dynamically reshape, redesign, and restructure our environment using the functional nature of digital tools? To address this, we present Recompose, a framework allowing direct and gestural manipulation of our physical environment. Recompose complements the highly precise, yet concentrated affordance of direct manipulation with a set of gestures, allowing functional manipulation of an actuated surface.

Rope Revolution is a rope-based gaming system for collaborative play. After identifying popular rope games and activities from around the world, we developed a generalized tangible rope interface that includes a compact motion-sensing and force-feedback module that can be used for a variety of rope-based games, such as rope jumping, kite flying, and horseback riding. Rope Revolution is designed to foster both co-located and remote collaborative experiences by using actual rope to connect players in physical activities across virtual spaces.

An environment for creative collaboration is significant for enhancing human communication and expressive activities, and many researchers have explored different collaborative spatial interaction technologies. However, most of these systems require special equipment and cannot adapt to everyday environments. We introduce Second Surface, a novel multi-user augmented reality system that fosters real-time interactions for user-generated content on top of the physical environment. This interaction takes place in the physical surroundings of everyday objects such as trees or houses. Our system allows users to place 3D drawings, texts, and photos relative to such objects and to share this expression with any other person who uses the same software at the same spot. Second Surface explores a vision that integrates collaborative virtual spaces into the physical space. Our system can provide an alternate reality that generates a playful and natural interaction in an everyday setup.

Sourcemap.com is the open directory of supply chains and environmental footprints. Consumers use the site to learn about where products come from, what they’re made of, and how they impact people and the environment. Companies use Sourcemap to communicate transparently with consumers and tell the story of how products are made. Thousands of maps have already been created for food, furniture, clothing, electronics, and more. Behind the website is a revolutionary social network for supply-chain reporting. The real-time platform gathers information from every stakeholder so that–one day soon–you’ll be able to scan a product on a store shelf and know exactly who made it.

People have developed sophisticated skills for sensing and manipulating our physical environments, but traditional GUIs (Graphical User Interfaces) do not employ most of them. Tangible Bits builds upon these skills by giving physical form to digital information, seamlessly coupling the worlds of bits and atoms. We are designing "tangible user interfaces" that employ physical objects, surfaces, and spaces as tangible embodiments of digital information. These include foreground interactions with graspable objects and augmented surfaces, exploiting the human senses of touch and kinesthesia. We also explore background information displays that use "ambient media"—light, sound, airflow, and water movement—to communicate digitally mediated senses of activity and presence at the periphery of human awareness. We aim to change the "painted bits" of GUIs to "tangible bits," taking advantage of the richness of multimodal human senses and skills developed through our lifetimes of interaction with the physical world.