Work for a Member organization and need a Member Portal account? Register here with your official email address.
October in Cambridge means the annual colorful show of trees reabsorbing their nutrients in preparation for the winter. The fancy word for this is "senescence", and I spent some time exploring it computationally.
I wanted to cover three things: its colors, the leaf structure, and how the colors diffuse over that structure over time. But it all comes back to looking at the thing itself, so let's start by taking a closer look at some leaves!
Creating colors computationally based on real-world phenomena is a game of telephone. Compounds in the leaves are absorbing and reflecting wavelengths from sunlight. Our human vision system gathers information about this: our lenses filter some wavelengths and we trade information about the intensities of specific wavelengths for broader bands of wavelengths that the rods and three types of cones can sense. We can make computer screens trigger the same experience by using three well-chosen components, and our favorites are red, green, and blue. (To us, these images look picture-perfect, but to a different visual system, it would look off.)
The palette for the fall leaf palette consists of the green chlorophyll, yellow/orange carotenoids, red anthocyanin, and brown tannins. Roughly, the leaf breaks down its green chlorophyll, which reveals the yellow/orange carotenoids (that were there the whole time!) Some species, like red maples, produce bright red anthocyanins. Finally when the leaf is drained and dead, all that's left is the light brown from the tannins.
You can run a chromatography experiment in your kitchen to see these components for yourself! I grounded up some spinach and let the compounds travel up a strip of coffee filter to see the individual sources of color.
Now let's get back to computation. In a fragment shaders the amount of red-green-blue-alpha in a given pixel, but you can just as easily work in components of chlorophyll, carotenoids, and anthocyanin. So instead of "red" meaning "show red" it would mean "the cell at this pixel has this amount of anthocyanin." At this point I'd use *artistic license* to covert to the colors shown on the screen (I know better than to try to model exact wavelengths if I want to successfully evoke leaf-color sensations.)
With color out of the way, let's build the structure. There are the thicker veins, including the large one down the center (a midrib), which supports the leaf, as well as smaller veins that run around these colorful cells. (When I backlit the leaf in the close-up images above, the veins glowed more. They are the tubes for transporting water/nutrients, compared to the other cells that are doing a lot of the work of the cell like photosynthesis.
Reunion et al. explains a way to replicate how leaf veins grow in a paper, which works by sprinkling "auxin" (a plant hormone) across the 2d screen, then having a the closest point of the current vein structure grow in the direction of the auxin. So if there is a pair auxin off in some direction, the vein will grow to the midpoint, and then the vein will split and start growing towards the two auxin.
One thing is replicating those little cells in a leaf. I've tried to with the extension of the Reunion paper, which is very cool looking, but doesn't quite make all of the individual cells I'm looking for.
So this time, I wanted to try a hybrid approach. The plan was:
- Use Reunions to draw the veins.
- Distribute points evenly on the remaining screen.
- Use something Voronoi-flavored to turn it into cells.
The path of least resistance was to do this in shaders. First, I created a distance map from the veins, so that I wouldn't place cells right on top of the veins. Then I upscale a tiled blue noise texture I usually use for dithering, which roughly places points. Because the blue noise texture is tiled, it can have ugly repetitions, but I thought I'd cross that road when I got there: sometimes it turns out you have enough variation that you don't notice!
But actually, about that distance map. A distance map takes a texture of "seed points" and plots the distance from all of your pixel to the nearest seed point. It's useful for a things like creating nice smooth outlines.
I've implemented this before by repeatedly blurring my image, but this doesn't scale well: it either can't compute distances very far, or you need to run it a ton of times. There are other tricks, like downsampling the image, blurring, and upsampling (like the effect used to make things glow)... but it was time to try something new. And conveniently, the algorithm to make a faster distance map is actually the same as the one I needed to make Voronoi! It's the amazing, Jump Flooding Algorithm!
In short, given a texture of "seed points" (like the points representing my cells) this outputs a texture where the color for each pixel are the _coordinates_ of the nearest "seed point" (shaders always take me a while to wrap my head around.) From there, you can then compute the distance to a seed to get a distance map from all of your seeds. There's a lot you can do from here, like using shaders to store data in the seeds.
After this successful coding session, I reflected and realized my next step of networking the cells together would be a lot easier in CPU-land. I was also noticing some artifacts.
On the GPU, I reused a blue noise dither layer I already had. For the IAP74 poster, I had done a hybrid using void-cluster algorithm (put a dot on a texture, blur the texture, then compute the lightest point on the texture, and put your next dot there.) For this project, I wanted to try doing CPU-based distributed noise using Poisson Disk Sampling. You set the minimum distance you want your points to be. You put your seed points into a queue, and then take the first point, and sample positions on a disc that is as far away as that distance. If it's far enough away from all of the other points, you add that point to your seeds, add it to the queue, and keep going!
Here's the first step of just spacing out the initial seeds along the veins. That turns out to make a big difference in how the cells look!
Next I used the spade library to compute the Voronoi cells. So we have a way to make leaf structures! Now it's a matter of exploring different parameters.
So we know about the colors, we have a way to compute the structure, next would be to try to tie it together and show colors/compounds move between cell neighbors and the veins. For this I need to know the neighbor of each cell. One way is to use Delaunay triangulation, which is actually already a byproduct of computing Voronoi!
It works like this: each frame, we compute how much of each compound flows between neighboring cells. If a cell has lower concentration of a compound, it'll pull in from neighboring cell with higher concentration. Now if you just keep running this for a while, colors should seep through the system.
Well then, easier said than done. Tuning a system apparently takes some work. Increasing the velocity so change was visible led to a disco flashing light leaf.
The way I got it to work was to crank down the amount that could move each iteration, and run a bunch of iterations each frame.
Phew. So now to control things, I can set the initial amount of a compound, how much every cell loses from decay (I'm using Murrelet, so I can have this be a function of the cell's position and its current compounds). I compute how much should be fed or taken from the root of the leaf, to simulate the leaf pulling nutrients out, or how the leaf is a little greener near the vein. Then I diffuse the compounds through its neighbors, with a higher rate for cells near veins. I apply the color scheme from the first part, and we get something like this:
I'm still exploring what I can do with this, and occasionally hit on something I like aesthetically. It does hit on something I tend to like: diffusing gradients of colors, and playing on a computer screen's grid-based noise compared to and the natural pixelation through colors in discrete cells or scales or feather barbs and so on. I think it could be interesting to use interactively: I used a light sensor to control the global decay. A color sensor could introduce the colors sensed and let it diffuse through the system.