An Adaptive Cartography: creative parameter space exploration

Because these systems are particular about their parameters, I find this makes parameter exploration techniques more jarring to use. So can we update the tools or system to create more bird-looking birds?

Out of the box, MurreletGen let's me set bounds on parameters one-by-one. I could use this to restrict the parameter ranges to the safe area that is valid. But by doing so, I might miss out on interesting variations! You could think of this as drawing a valid box when the actual shape of valid variations is more of a blob. A box that only contains valid birds would need to be extra conservative to not fall outside of the blob, and might turn out a bit boring.

Adaptive Cartography

Instead, I draw an extra-large box that contains the entire blob, and use machine learning techniques (which can be non-linear and blob-like) to have us spend more time in the blob area. I do this by testing a bunch of values from within the bounds and gathering data on which inputs produce valid birds (within the blob) or invalid birds (outside the blob). I use machine learning to learn a function that gives more space to the interesting results and less to the buggy places. In other words, I want to use ML to crumple up the map so I can skim over the buggy/boring things, but not miss out on the really interesting places.

Automated Checks and User Feedback

First, we need to find the region that is interesting and not buggy. I did this with a combination of automated checks and user feedback. The tool begins by generating random examples. Getting user feedback on every example would be time-consuming. 

One insight was to program checks that are easy to check, but have a complex non-linear relationship with parameters. One example of this type of check was "is the hip within the body area?" It would be a pain to constrain the parameters directly! But it is trivial to verify after we start computing the bird shape, and can even be estimated before the bird is drawn. 

Next are the slightly slower, but very interesting post-render checks. For example, I added one that made sure "at least 5% of pixels are beak-colored." This helped catch some issues that were difficult to detect before the pixels are drawn. Pixel-based tests could be really interesting for making art too! ("only show me when 20% of the image is red").

At this point, I also tested automated tests like YOLO. Out of the box, it wasn't particularly good at distinguishing invalid birds from valid birds, and also had a tendency to mark my graphical birds as umbrellas. Perhaps it would be useful if it were to be refined!

The most expensive and valuable test is user feedback. I would score birds that passed the rest of the tests. This way I could promote interesting birds. I could use this to down-rank rare edge cases where it wasn't worth it to add an automatic test.

An Aside on Sampling

I tried to speed this up by trying different ways of sampling.
By default, we sample the entire space. Once we have birds that pass a few checks, we can start to use tricks from genetic algorithms, like mixing valid birds, introducing mutations to valid birds, and encouraging diversity by prioritizing birds that are different from each other. 

Since running the full checks took time, I also tried training a simple neural network to predict the score straight from the input vector. That can be computed much faster than running the graphics code! But qualitatively, it seemed like the most interesting birds came from the totally random selection. But something to keep exploring!

Results

After running for a while, it found some very interesting birds! I enjoy them, and I don't think I would have created these on my own.