Object-Based Media
How sensing, understanding, and new interface technologies can change everyday life, the ways in which we communicate with one another, storytelling, and entertainment.
Our group explores how the distribution of computational intelligence throughout video and audio communication systems can make a richer connection between the people at each end. We conduct research into the future of electronic visual communication and expression, and ways to make a richer connection among the people at the ends of the system, whether a broadcast system or a peer-to-peer environment. We also develop hardware and software technologies to support the requirements of such a scenario, with particular focus on new input and output technologies, advanced interfaces for consumer electronics, and self-organization among smart devices.

Research Projects

  • 3D Telepresence Chair

    V. Michael Bove Jr.

    An autostereoscopic (no glasses) 3D display engine is combined with a "Pepper's Ghost" setup to create an office chair that appears to contain a remote meeting participant. The system geometry is also suitable for other applications such as tabletop displays or automotive heads-up displays.

  • Calliope

    Edwina Portocarrero

    Calliope is the follow-up to the NeverEnding Drawing Machine. A portable, paper-based platform for interactive story making, it allows physical editing of shared digital media at a distance. The system is composed of a network of creation stations that seamlessly blend analog and digital media. Calliope documents and displays the creative process with no need to interact directly with a computer. By using human-readable tags and allowing any object to be used as material for creation, it offers opportunities for cross-cultural and cross-generational collaboration among peers with expertise in different media.

  • Consumer Holo-Video

    V. Michael Bove Jr., James D. Barabas, Sundeep Jolly and Daniel E. Smalley
    The goal of this project, building upon work begun by Stephen Benton and the Spatial Imaging group, is to create an inexpensive desktop monitor for a PC or game console that displays holographic video images in real time, suitable for entertainment, engineering, or medical imaging. To date, we have demonstrated the fast rendering of holo-video images (including stereographic images that unlike ordinary stereograms have focusing consistent with depth information) from OpenGL databases on off-the-shelf PC graphics cards; current research addresses new optoelectronic architectures to reduce the size and manufacturing cost of the display system.
  • Direct Fringe Writing of Computer-Generated Holograms

    V. Michael Bove Jr., Sundeep Jolly and University of Arizona College of Optical Sciences

    Photorefractive polymer has many attractive properties for dynamic holographic displays; however, the current display systems based around its use involve generating holograms by optical interference methods that complicate the optical and computational architectures of the systems and limit the kinds of holograms that can be displayed. We are developing a system to write computer-generated diffraction fringes directly from spatial light modulators to photorefractive polymers, resulting in displays with reduced footprint and cost, and potentially higher perceptual quality.

  • Everything Tells a Story

    V. Michael Bove Jr., David Cranor and Edwina Portocarrero
    Following upon work begun in the Graspables project, we are exploring what happens when a wide range of everyday consumer products can sense, interpret into human terms (using pattern recognition methods), and retain memories, such that users can construct a narrative with the aid of the recollections of the "diaries" of their sporting equipment, luggage, furniture, toys, and other items with which they interact.
  • Guided-Wave Light Modulator

    V. Michael Bove Jr., Daniel Smalley and Quinn Smithwick
    We are developing inexpensive, efficient, high-bandwidth light modulators based on lithium niobate guided-wave technology. These modulators are suitable for demanding, specialized applications such as holographic video displays, as well as other light modulation uses such as compact video projectors.
  • Infinity-by-Nine

    V. Michael Bove Jr. and Daniel Novy

    We expand the home-video viewing experience by generating imagery to extend the TV screen and give the impression that the scene wraps completely around the viewer. Optical flow, color analysis, and heuristics extrapolate beyond the screen edge, where projectors provide the viewer's perceptual vision with low-detail dynamic patterns that are perceptually consistent with the video imagery and increase the sense of immersive presence and participation. We perform this processing in real time using standard microprocessors and GPUs.

  • Living Observatory: Arboreal Telepresence

    Joseph A. Paradiso, V. Michael Bove, Gershon Dublon, Edwina Portocarrero and Glorianna Davenport

    Extending the Living Observatory installation, we have instrumented the roots of several trees outside of E15 with vibratory transducers that excite the trees with live streaming sound from a forest near Plymouth, MA. Walking though the trees just outside the Lab, you won't notice anything, but press your ear up against one of them and you'll feel vibrations and hear sound from a tree 60 miles away. Visit at any time from dawn till dusk and again after midnight; if you’re lucky you might just catch an April storm, a flock of birds, or an army of frogs.

  • Narratarium

    V. Michael Bove Jr., Catherine Havasi, Katherine (Kasia) Hayden, Daniel Novy, Jie Qi and Robert H. Speer

    Remember telling scary stories in the dark with flashlights? Narratarium is an immersive storytelling environment to augment creative play using texture, color, and image. We are using natural language processing to listen to and understand stories being told, and thematically augment the environment using color and images. As a child tells stories about a jungle, the room is filled with greens and browns and foliage comes into view. A traveling parent can tell a story to a child and fill to room with images, color, and presence.

  • Pillow-Talk

    V. Michael Bove Jr., Edwina Portocarrero, David Cranor

    Pillow-Talk is the first of a series of objects designed to aid creative endeavors through the unobtrusive acquisition of unconscious self-generated content to permit reflexive self-knowledge. Composed of a seamless recording device embedded in a pillow, and a playback and visualization system in a jar, Pillow-Talk crystallizes that which we normally forget. This allows users to capture their dreams in a less mediated way, aiding recollection by priming the experience and providing no distraction for recall and capture through embodied interaction.

  • ProtoTouch: Multitouch Interfaces to Everyday Objects

    V. Michael Bove Jr. and David Cranor

    An assortment of everyday objects is given the ability to understand multitouch gestures of the sort used in mobile-device user interfaces, enabling people to use such increasingly familiar gestures to control a variety of objects, and to "copy" and "paste" configurations and other information among them.

  • ShakeOnIt

    V. Michael Bove Jr. and David Cranor

    We are exploring ways to encode information exchange into preexisting natural interaction patterns, both between people and between a single user and objects with which he or she interacts on a regular basis. Two devices are presented to provoke thoughts regarding these information interchange modalities: a pair of gloves that requires two users to complete a "secret handshake" in order to gain shared access to restricted information, and a doorknob that recognizes the grasp of a user and becomes operational if the person attempting to use it is authorized to do so.

  • Simple Spectral Sensing

    V. Michael Bove Jr.

    The availability of cheap LEDs and diode lasers in a variety of wavelengths enables creation of simple and cheap spectroscopic sensors for specific tasks such as food shopping and preparation, healthcare sensing, material identification, and detection of contaminants or adulterants.

  • Slam Force Net

    Santiago Alfaro, V. Michael Bove Jr. and Daniel Novy

    A basketball net incorporates segments of conductive fiber whose resistance changes with degree of stretch. By measuring this resistance over time, hardware associated with this net can calculate force and speed of a basketball traveling through the net. Applications include training, toys that indicate the force and speed on a display, “dunk competitions,” and augmented reality effects on television broadcasts. This net is far less expensive and more robust than other approaches to measuring data about the ball (e.g., photosensors or ultrasonic sensors) and doesn’t require a physical change to the hoop or backboard other than providing electrical connections to the net. Another application of the material is a flat net that can measure velocity of a ball hit or pitched into it (as in baseball or tennis), and can measure position as well (e.g., for determining whether a practice baseball pitch would have been a strike).

  • SurroundVision

    V. Michael Bove Jr. and Santiago Alfaro
    Adding augmented reality to the living room TV, we are exploring the technical and creative implications of using a mobile phone or tablet (and possibly also dedicated devices like toys) as a controllable "second screen" for enhancing television viewing. Thus, a viewer could use the phone to look beyond the edges of the television to see the audience for a studio-based program, to pan around a sporting event, to take snapshots for a scavenger hunt, or to simulate binoculars to zoom in on a part of the scene. Recent developments include the creation of a mobile device app for Apple products and user studies involving several genres of broadcast television programming.
  • The "Bar of Soap": Grasp-Based Interfaces

    V. Michael Bove Jr. and Brandon Taylor
    We have built several handheld devices that combine grasp and orientation sensing with pattern recognition in order to provide highly intelligent user interfaces. The Bar of Soap is a handheld device that senses the pattern of touch and orientation when it is held, and reconfigures to become one of a variety of devices, such as phone, camera, remote control, PDA, or game machine. Pattern-recognition techniques allow the device to infer the user's intention based on grasp. Another example is a baseball that determines a user's pitching style as an input to a video game.
  • Vision-Based Interfaces for Mobile Devices

    V. Michael Bove Jr. and Santiago Alfaro
    Mobile devices with cameras have enough processing power to do simple machine-vision tasks, and we are exploring how this capability can enable new user interfaces to applications. Examples include dialing someone by pointing the camera at the person's photograph, or using the camera as an input to allow navigating virtual spaces larger than the device's screen.