KnittedKeyboard II

A keyboard made out of fabric, besides providing new interactions and tactile experiences for musical expressions, can be easily folded, rolled up, and packed in our luggage like a pair of socks or a scarf.  It can also be wearable, which extends the functionality of such fabric-based musical controllers, similar to the keyboard tie formerly demonstrated by Laurie Anderson in the "Home of the Brave."

Our prototype utilizes digital knitting technology and explores intarsia, interlock patterning, and a collection of functional (electrically-conductive and thermoplastic) and non-functional (polyester) fibers to develop a seamless and customized, 5-octave piano-patterned textile for expressive and virtuosic sonic interaction. The individual and combinations of keys could simultaneously sense touch, as well as continuous proximity, stretch, and pressure. The KnittedKeyboard II combines both discrete controls from the conventional keystrokes and expressive continuous controls from the non-contact theremin-inspired proximity sensors by waving and hovering on the air, as well as unique physical interactions enabled by the integrated fabric sensors (e.g. squeezing, pulling, stretching, and twisting). It enables performers to experience fabric-based multimodal embodied interaction and unique, intimate and organic tactile experience as they explore the seamless texture and materiality of the electronic textile. 

The sensing mechanism is based on capacitive and piezo-resistive sensing. Every key acts as an electrode and is sequentially charged and discharged. This creates an electromagnetic field that can be disrupted by hand’s approach, enabling us to detect not only contact touch, but also non-contact proxemic gesture such as hovering or waving on the air, contact touch, as well as to calculate strike velocity. The piezo-resistive layers underneath can measure pressure and stretch exerted on the knitted keyboard. All of the sensor data is converted to musical instrument digital interface (MIDI) messages by a central microprocessor, which will correspond to certain timbral, dynamic, and temporal variations (filter resonance, frequency, glide, reverb, amp, distortion, et cetera), as well as pitch-bend. Audio sequencing and generation software such as Ableton Live and Max/MSP map these MIDI messages to their corresponding channels, controls, notes, and effects. 

"Fabric of Time and Space" is a contemporary musical piece exclusively written for KnittedKeyboard II to demonstrate and illustrate the multi-dimensional expressiveness of the instrument. The piece is a metaphor for the expanding and contracting nature of the universe and this is represented musically by the glissandi of the melody as well as the interplay between major and minor chords. The metaphorical perturbations of space-time were expressed by the interaction between the performer and the fabric. The musical translation of these expressions were used to shape the envelope of the sound.

This work brings us one step closer to the vision of seamless fabric-based interactive surfaces, that is not only applicable in novel musical controllers, but also in wearables, smart objects, and responsive environments. The underlining technology would enable further exploration of soft and malleable gestural interfaces that leverage the unique mechanical structures of the materials, as well as the intrinsic electrical properties of the knitted sensors.