Bibliodoptera is an installation commissioned for the MIT 150th Anniversary Celebration FAST Festival of Art, Science, and Technology. A cloud of vellum butterflies, newly emerged from the chrysalis of MIT’s diverse library pages, floats above in the corridor between the Lewis and Hayden Libraries on MIT’s campus. Trajectories through the cloud illuminate to guide passersby along the length of the corridor. This installation is an unobtrusive but strikingly beautiful symbol of the guiding knowledge of the arts and humanities that have been developed and pursued at MIT over the last 150 years. The butterflies, printed with text from books, sheet music and pages of MIT theses, are interactively illuminated by small lights from within.

Chroma District was installed as a part of MIT's FAST festival. It consisted of lanterns hung from trees near the David H. Koch Institute for Integrative Cancer Research. The lanterns were illuminated with different colors, and each one produced its own unique sound, which had been recorded locally. In its idle state, each lantern softly played a sound and emitted a gentle light. When a visitor approached a lantern, its sound and color passed from one lantern to another, increasing in intensity along the way and illuminating the path forward.

With CogNotes, new music composition tools become platforms for cognitive assessment. CogNotes users engage their memory as they emerge as composers and participants in their own health process. Together with partners the Lincoln Park Performing Arts School and the Yamaha Corporation, a group of seniors are undertaking an extensive multi-month music composition workshop built around Tod Machover's Hyperscore program. The program is outfitted with cognitive measures sensitive to the earliest transition to Alzheimer's Disease, validated as part of post-doc Adam Boulanger's research with Harvard Medical School and the Alzheimer's Association. Disease assessment can be part of your everyday, creative, and rewarding life. You can be the manager of your own health information as part of the activities you love.

"Death and the Powers" is a groundbreaking opera that brings a variety of technological, conceptual, and aesthetic innovations to the theatrical world. Created by Tod Machover (composer), Diane Paulus (director), and Alex McDowell (production designer), the opera uses the techniques of tomorrow to address age-old human concerns of life and legacy. The unique performance environment, including autonomous robots, expressive scenery, new Hyperinstruments, and human actors, blurs the line between animate and inanimate. The opera premiered in Monte-Carlo in fall 2010, with additional performances in Boston and Chicago in 2011 and continuing engagements worldwide.

This project aims to transform everyday objects into percussive musical instruments, encouraging people to rediscover their surroundings through musical interactions with the objects around them. DrumTop is a drum machine made up of eight transducers. Placing objects on top of the transducers triggers a "hit," causing sounds to come out from the objects themselves. In addition, users can program drum patterns by pushing on a transducer, and the weight of an object can be measured to control the strength of a “hit.”

The Hyperinstrument project creates expanded musical instruments and uses technology to give extra power and finesse to virtuosic performers. They were designed to augment a wide range of traditional musical instruments and have been used by some of the world's foremost performers (Yo-Yo Ma, the Los Angeles Philharmonic, Peter Gabriel, and Penn & Teller). Research focuses on designing computer systems that measure and interpret human expression and feeling, exploring appropriate modalities and content of interactive art and entertainment environments, and building sophisticated interactive musical instruments for non-professional musicians, students, music lovers, and the general public. Recent projects involve both new hyperinstruments for children and amateurs, and high-end hyperinstruments capable of expanding and transforming a symphony orchestra or an entire opera stage.

Media Scores extends the concept of a musical score to other modalities to facilitate the process of authoring and performing multimedia compositions, providing a medium through which to realize a modern-day Gesamtkunstwerk. Through research into the representation and the encoding of expressive intent, systems for composing with media scores are being developed. Using such a tool, the composer will be able to shape an artistic work that may be performed through human and technological means in a variety of media and modalities. Media scores offer the potential for authoring content considering live performance data and the potential for audience participation and interaction. This paradigm bridges the extremes of the continuum from composition to performance, allowing for improvisatory compositional acts at performance time. The media score also provides a common point of reference in collaborative productions as well as the infrastructure for real-time control of technologies used during live performance.

Our work in Music, Mind, and Health has culminated in a recent PhD thesis, showing the technologies and perspectives required to build on the transformative nature of music to drive specific neurological, physical, and psychological change. A radically new "Personal Instrument" is currently being used by Dan Ellsey, a quadraplegic individual, who controls this interface to sculpt an expressive performance of music in real time. A three-month study of Ellsey's expressive behavior—its potential as well as its limits—resulted in an interface tailored just for him, enabling him to access expressive performance despite his physical disability. This new line of work highlights principles for future instruments and applications, where the impact is in the marriage of the interface and uniqueness of the person. In this way, we are pursuing new design philosophies, technologies, and collaborations within the scientific community, public performance, and clinical research.

muStick is a tribal instrument turned digital. It is a simple stick to which sensors are attached. They detect the thumping of the stick against the ground and tapping on the stick. The angle at which the stick is held is also measured. These signals are tranformed into MIDI signals, and turn μStick into a simple yet powerful music controller.

Personal Opera is a radically innovative creative environment that enables anyone to create musical masterpieces sharing one’s deepest thoughts, feelings, and memories. Based on our design of, and experience with, such projects as Hyperscore and the Brain Opera, we are developing a totally new environment to allow the incorporation of personal stories, images, and both original and well-loved music and sounds. Personal Opera builds on our guiding principle that active music creation yields far more powerful benefits than passive listening. Using music as the through-line for assembling and conveying our own individual legacies, Personal Opera represents a new form of expressive archiving; easy to use and powerful to experience. In partnership with the Royal Opera House in London, we have begun conducting Personal Opera workshops specifically targeting seniors to help them tell their own meaningful stories through music, text, visuals, and acting.

Mobile social computing is becoming a practical and alternative way to reach audience members in music performance scenes, but musicians and vendors of music performances need to know how to design social music computing services that encourage audience members to actively participate in a given musical performance. We are developing new and compelling technologies for Social Live Enhanced Audience Performance (SLEAP) that connect audience members and musicians in new ways based on mobile social computing systems. We are collaborating with a consortium of major symphony orchestras as well as the Punchdrunk theater group.

Spheres and Splinters is a new work composed by Tod Machover for hypercello, electronics, and responsive visuals commissioned fo Faster than Sound at Aldeburgh Music. The work premiered with celllist Peter Gregson in the UK in 2010 and had its US premiere as part of FAST Festival of Art, Science, and Technology in celebration of MIT's 150th anniversary. Utilizing audio analysis and a multitude of wireless sensors on the cello and the bow that capture how the instrument is being played, the performer has control over transformations and extensions of the sound produced. This control extends into the ambisonic spatialization of sound in the performance space. The performance data is also used to produce realtime visual accompaniment on an array of LED strips surrounding the cellist.

Toy Symphony combines children, virtuosic soloists, composers, and symphony orchestras around the world to alter radically how children are introduced to music, as well as to redefine the relationship between professional musicians and young people. A complete set of Music Toys will be distributed to children in each host city (including Berlin, Dublin, Glasgow, Manchester/London, and Tokyo), where children will be mentored to create their own sounds and compositions for toys and traditional instruments. A pedagogy for using these Music Toys to teach and to instill a love for musical creativity will also be developed. Final concerts will be presented in each host city including children's compositions and specially commissioned works by young composers, to be performed by children, soloists, and orchestra, playing Music Toys, Hyperinstruments, and traditional instruments.