Refraction - Using computation to control caustics

Pixels to Lux

The process to convert an image  into a 3-dimensional refractive surface is informed by  Yue et al.'s 2014 paper on goal-based caustics. At a high level, the process involves creating a mesh that achieves a specific refracted image when illuminated. Each mesh cell directs an incoming ray of light to the appropriate position in the refracted image such that the total light energy entering the surface is conserved while the brightness values of the refracted image match the original input image as closely as possible. 

An equivalence is made between the total pixel brightness in the image and the total area of the refracted surface mesh, such that each mesh cell can be morphed to approximate the brightness of its corresponding pixel. Starting with a mesh constructed from a square grid with no variation on the z-axis, there is a loss value associated with each grid cell based on its area relative to the brightness of it's corresponding pixel.

The ingenuity in the algorithm described by Yue et al.  is to use  Poisson's equation to reframe the initially unconverging problem of loss minimization across the entire mesh,  into a problem of pressure relaxation over time, which is an approached informed by computational fluid dynamics. The result is a mesh in which each cell's area approximates the brightness value of its corresponding image pixel. Subsequently, Poisson's equation is used again in conjunction with Snell's law to ensure that each cell is correctly oriented to refract incoming light towards the appropriate part of the refracted image.

Initial Result

The first tests surfaces that  I created were simple sketches I made as part of my project Cloud Light. I'll continue to experiment with this process as a means for making art.