Jay Silver, Eric Rosenbaum

Abstract

Twinkle allows anyone to program using colors in the real world. Twinkle uses a color sensor to read colors from arrangements of objects, drawings, or collages. Those colors are then mapped to certain outputs, like sounds, graphics, or robotic movements. Color patterns can even be used to control the color sensor itself, closing the loop. The result is that you can program a computer or a robot, or compose a musical score, just by drawing on a piece of paper with crayons. Of course it's not limited to crayons. You could build your program with Lego bricks, arrange your program with the multi colored leaves of early Fall, or think of any collection of objects in the world as a program: from a striped shirt to a handful of M&Ms. In the limit, several interesting new programming concepts emerge from this paradigm: commands are no longer discrete and rigid but mixable and smearable; the program counter becomes visible, handheld, and nondeterministic; and when the color sensor becomes the program counter the application space and the programming space become intertwined.