Project

Turning Objects into Cameras

Copyright

Tristan Swedish

Tristan Swedish

Groups

Consider a small object sitting on a desk in your living room. The object is illuminated by light sources from all directions—this includes direct sources such as the sun or overhead lights, but also indirect sources, like the foliage outside that scatters sunlight through your window. The appearance of the object and the surface that it rests upon results from the complex interaction between the incident illumination and the geometry and material properties of the object and the desk. In this paper we ask the question—if the geometry and material properties of the observed scene are known, how well can we reconstruct the incident illumination pattern?

In our work we primarily make use of shadows cast by an object onto nearby surfaces. Cast shadows are particularly easy to interpret when an object is illuminated from a single direction. For example, one can immediately determine the position of the sun by looking at a sundial. Estimating the illumination incident from all directions simultaneously is more challenging, and is a linear but ill-posed inverse problem. 

Copyright

Tristan Swedish

Copyright

Tristan Swedish

 Our paper was published in the proceedings of the Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV).

Title: Objects As Cameras: Estimating High-Frequency Illumination From Shadows

Abstract: We recover high-frequency information encoded in the shadows cast by an object to estimate a hemispherical photograph from the viewpoint of the object, effectively turning objects into cameras. Estimating environment maps is useful for advanced image editing tasks such as relighting, object insertion or removal, and material parameter estimation. Because the problem is ill-posed, recent works in illumination recovery have tackled the problem of low-frequency lighting for object insertion, rely upon specular surface materials, or make use of data-driven methods that are susceptible to hallucination without physically plausible constraints. We incorporate an optimization scheme to update scene parameters that could enable practical capture of real-world scenes. Furthermore, we develop a methodology for evaluating expected recovery performance for different types and shapes of objects.

Full text is accessible at: https://openaccess.thecvf.com/content/ICCV2021/html/Swedish_Objects_As_Cameras_Estimating_High-Frequency_Illumination_From_Shadows_ICCV_2021_paper.html 

Research Topics
#computer vision #imaging