Camera Culture

With over a billion people carrying camera-phones worldwide, we have a new opportunity to upgrade the classic bar code to encourage a flexible interface between the machine world and the human world. Current bar codes must be read within a short range and the codes occupy valuable space on products. We present a new, low-cost, passive optical design so that bar codes can be shrunk to fewer than 3mm and can be read by unmodified ordinary cameras several meters away.

Computational photography is an emerging multi-disciplinary field that is at the intersection of optics, signal processing, computer graphics and vision, electronics, art, and online sharing in social networks. The first phase of computational photography was about building a super-camera that has enhanced performance in terms of the traditional parameters, such as dynamic range, field of view, or depth of field. We call this 'Epsilon Photography.' The next phase of computational photography is building tools that go beyond the capabilities of this super-camera. We call this 'Coded Photography.' We can code exposure, aperture, motion, wavelength, and illumination. By blocking light over time or space, we can preserve more details about the scene in the recorded single photograph.

Our goal is to exploit the finite speed of light to improve image capture and scene understanding. New theoretical analysis, coupled with emerging ultra-high-speed imaging techniques, can lead to a new source of computational visual perception. We are developing the theoretical foundation for sensing and reasoning using transient light transport, and experimenting with scenarios in which transient reasoning exposes scene properties that are beyond the reach of traditional machine vision.

The goal is to build a wearable fabric that supports millimeter-accurate location and bio-parameter tracking at thousands of points on the body. Such a fabric can compute and predict 3-D representations of human activity and use them with closed-loop tactile feedback to augment human performance. This will be able to provide a detailed analysis and control of higher-level human activity. The basic technology uses a new, optical motion-capture method we have recently developed.

We present a new method for scanning 3-D objects in a single shot, shadow-based method. We decouple 3-D occluders from 4-D illumination using shield fields: the 4-D attenuation function which acts on any light field incident on an occluder. We then analyze occluder reconstruction from cast shadows, leading to a single-shot light field camera for visual hull reconstruction.

This work focuses on bringing powerful concepts from wave optics to the creation of new algorithms and applications for computer vision and graphics. Specifically, ray-based, 4-D lightfield representation, based on simple 3-D geometric principles, has led to a range of new applications that include digital refocusing, depth estimation, synthetic aperture, and glare reduction within a camera or using an array of cameras. The lightfield representation, however, is inadequate to describe interactions with diffractive or phase-sensitive optical elements. Therefore we use Fourier optics principles to represent wavefronts with additional phase information. We introduce a key modification to the ray-based model to support modeling of wave phenomenon. The two key ideas are "negative radiance" and a "virtual light projector." This involves exploiting higher dimensional representation of light transport.

Computer vision is a class of technologies that lets computers use cameras to automatically stitch together panoramas, reconstruct 3-D geometry from multiple photographs, and even tell you when the water's boiling. For decades, this technology has been advancing mostly within the confines of academic institutions and research labs. Vision on Tap is our attempt to bring computer vision to the masses.