Research Projects
A Toronto Symphony: Massive Musical Collaboration
Tod Machover, Akito Van Troyer, Ben Bloomberg, and Peter TorpeyThus far, the results of existing crowdsourced and interactive music are limited, with the public only a small part of a final musical result, and often disconnected from the artist leading the project. We believe that a new “musical ecology” is needed for true creative collaboration between experts and amateurs that benefits both. For this purpose, we have created a new work for symphony orchestra in collaboration with the entire city of Toronto. Called “A Toronto Symphony,” the work–commissioned by the Toronto Symphony Orchestra–premiered in March 2013. We designed the necessary infrastructure, a variety of web-based music composition applications, a social media framework, and real-world community-building activities to bring together an unprecedented number of people from diverse ages, experiences, and musical backgrounds to create this new work. This process establishes a new model for creating complex creative collaborations between experts and everyone else.
Advanced Audio Systems for Live Performance
Tod Machover and Ben BloombergThis project explores the contribution of advanced audio systems to live performance, their design and construction, and their integration into the theatrical design process. We look specifically at innovative input and control systems for shaping the analysis and processing of live performance; and at large-scale output systems which provide a meaningful virtual abstraction to DSP in order to create flexible audio systems that can both adapt to many environments and achieve a consistent and precise sound field for large audiences.
Death and the Powers: Redefining Opera
"Death and the Powers" is a groundbreaking opera that brings a variety of technological, conceptual, and aesthetic innovations to the theatrical world. Created by Tod Machover (composer), Diane Paulus (director), and Alex McDowell (production designer), the opera uses the techniques of tomorrow to address age-old human concerns of life and legacy. The unique performance environment, including autonomous robots, expressive scenery, new Hyperinstruments, and human actors, blurs the line between animate and inanimate. The opera premiered in Monte-Carlo in fall 2010, with additional performances in Boston and Chicago in 2011 and continuing engagements worldwide, including upcoming performances in Dallas in February 2014.Tod Machover, Ben Bloomberg, Peter Torpey, Elena Jessop, Bob Hsiung, Michael Miller, Akito van Troyer, and Eyal ShaharDesigning Immersive Multi-Sensory Eating Experiences
Tod Machover and Janice WangFood offers a rich multi-modal experience that can deeply affect emotion and memory. We're interested in exploring the artistic and expressive potential of food beyond mere nourishment, as a means of creating memorable experiences that involve multiple senses. For instance, music can change our eating experience by altering our emotions during the meal, or by evoking a specific time and place. Similarly, sight, smell, temperature can all be manipulated to combine with food for expressive effect. In addition, by drawing upon people's physiology and upbringing, we seek to create individual, meaningful sensory experiences. Specifically, my masters thesis looks at the connection between music and flavor perception.
Disembodied Performance
Early in the opera "Death and the Powers," the main character Simon Powers is subsumed into a technological environment of his own creation. The set comes alive through robotic, visual, and sonic elements that allow the actor to extend his range and influence across the stage in unique and dynamic ways. This environment must assume the behavior and expression of the absent Simon; to distill the essence of this character, we recover performance parameters in real time from physiological sensors, voice, and vision systems. Gesture and performance parameters are then mapped to a visual language that allows the off-stage actor to express emotion and interact with others on stage. To accomplish this, we developed a suite of innovative analysis, mapping, and rendering software systems. Our approach takes a new direction in augmented performance, employing a non-representational abstraction of a human presence that fully translates a character into an environment.Tod Machover, Peter Torpey and Elena JessopDrumTop
Tod Machover and Akito Oshiro van TroyerThis project aims to transform everyday objects into percussive musical instruments, encouraging people to rediscover their surroundings through musical interactions with the objects around them. DrumTop is a drum machine made up of eight transducers. Placing objects on top of the transducers triggers a "hit," causing sounds to come out from the objects themselves. In addition, users can program drum patterns by pushing on a transducer, and the weight of an object can be measured to control the strength of a “hit.”
Future of the Festival
Tod Machover, Ben Bloomberg, Elena Jessop, Rebecca Kleinberger, Simone Ovsey, Peter Torpey, Akito van Troyer and Janice WangThe Opera of the Future group designed and ran the Future of the Festival class, which helped to define and shape The Other Festival. The group also designed and created a wide range of projects for the Festival, including Figments, a performance shaping multiple theatrical modalities using Media Scores; Crenulations and Excursions, an installation and performance space where expressive qualities of movement control a rich soundscape; a multisensory culinary experience for a large group; a multiplexed performance exploring the role of production techniques in live music; an interactive book exploring the musicality of the spoken word; a collaborative vocal improvisation experience for novices; objects with unexpected behaviors; and a mass anatine migration.
Gestural Media Framework
We are all equipped with two extremely expressive instruments for performance: the body and the voice. By using computer systems to sense and analyze human movement and voices, artists can take advantage of technology to augment the body's communicative powers. However, the sophistication, emotional content, and variety of expression possible through the original physical channels is often not captured by or addressed in the technologies used for analyzing them, and thus cannot be transferred from body to digital media. To address these issues, we are developing systems that use machine learning to map continuous input data, whether of gesture or voice or biological/physical states, to a space of expressive, qualitative parameters. We are also developing a new framework for expressive performance augmentation, allowing users to easily create clear, intuitive, and comprehensible mappings by using high-level qualitative movement descriptions, rather than low-level descriptions of sensor data streams.Tod Machover and Elena JessopHyperinstruments
The Hyperinstrument project creates expanded musical instruments and uses technology to give extra power and finesse to virtuosic performers. They were designed to augment a wide range of traditional musical instruments and have been used by some of the world's foremost performers (Yo-Yo Ma, the Los Angeles Philharmonic, Peter Gabriel, and Penn & Teller). Research focuses on designing computer systems that measure and interpret human expression and feeling, exploring appropriate modalities and content of interactive art and entertainment environments, and building sophisticated interactive musical instruments for non-professional musicians, students, music lovers, and the general public. Recent projects involve both new hyperinstruments for children and amateurs, and high-end hyperinstruments capable of expanding and transforming a symphony orchestra or an entire opera stage.Tod MachoverHyperscore
Hyperscore is an application to introduce children and non-musicians to musical composition and creativity in an intuitive and dynamic way. The "narrative" of a composition is expressed as a line-gesture, and the texture and shape of this line are analyzed to derive a pattern of tension-release, simplicity-complexity, and variable harmonization. The child creates or selects individual musical fragments in the form of chords or melodic motives, and layers them onto the narrative-line with expressive brushstokes. The Hyperscore system automatically realizes a full composition from a graphical representation, allowing individuals with no musical training to create professional pieces. Currently, Hyperscore uses a mouse-based interface; the final version will support freehand drawing, and integration with the Music Shapers and Beatbugs to provide a rich array of tactile tools for manipulation of the graphical score.Mary Farbood and Tod MachoverMedia Scores
Tod Machover and Peter TorpeyMedia Scores extends the concept of a musical score to other modalities, facilitating the process of authoring and performing multimedia compositions and providing a medium through which to realize a modern-day Gesamtkunstwerk. Through research into representation and encoding of expressive intent, systems for composing with media scores are being developed. Using such a tool, the composer will be able to shape an artistic work that may be performed through human and technological means in a variety of media and modalities. Media scores offer the potential for authoring content considering live performance data as well as audience participation and interaction. This paradigm bridges the extremes of the continuum from composition to performance, allowing for improvisatory compositional acts at performance time. The media score also provides a common point of reference in collaborative productions as well as the infrastructure for real-time control of technologies used during live performance.
Personal Opera
Personal Opera is a radically innovative creative environment that enables anyone to create musical masterpieces sharing one’s deepest thoughts, feelings, and memories. Based on our design of, and experience with, such projects as Hyperscore and the Brain Opera, we are developing a totally new environment to allow the incorporation of personal stories, images, and both original and well-loved music and sounds. Personal Opera builds on our guiding principle that active music creation yields far more powerful benefits than passive listening. Using music as the through-line for assembling and conveying our own individual legacies, Personal Opera represents a new form of expressive archiving; easy to use and powerful to experience. In partnership with the Royal Opera House in London, we have begun conducting Personal Opera workshops specifically targeting seniors to help them tell their own meaningful stories through music, text, visuals, and acting.Tod Machover and Peter TorpeyRemote Theatrical Immersion: Extending "Sleep No More"
Tod Machover, Punchdrunk, Akito Van Troyer, Ben Bloomberg, Gershon Dublon, Jason Haas, Elena Jessop, Brian Mayton, Eyal Shahar, Jie Qi, Nicholas Joliat, and Peter TorpeyWe have collaborated with London-based theater group Punchdrunk to create an online platform connected to their NYC show, Sleep No More. In the live show, masked audience members explore and interact with a rich environment, discovering their own narrative pathways. We have developed an online companion world to this real-life experience, through which online participants partner with live audience members to explore the interactive, immersive show together. Pushing the current capabilities of web standards and wireless communications technologies, the system delivers personalized multimedia content, allowing each online participant to have a unique experience co-created in real time by his own actions and those of his onsite partner. This project explores original ways of fostering meaningful relationships between online and onsite audience members, enhancing the experiences of both through the affordances that exist only at the intersection of the real and the virtual worlds.
The Other Feast
Tod Machover, Janice Wang, Benjamin Bloomberg, Peter A. Torpey and Philippa MothersillThe feast is a immersive, multi-sensory experience in four acts. Each act explores a different theme of eating, from mysterious to playful to communal. The feast is not a normal dinner where people passive eat what's in front of them. In the small hours of the night, we will explore the active, creative role of the diner and build community around food and companionship.
Vocal Vibrations: Expressive Performance for Body-Mind Wellbeing
Tod Machover, Elena Jessop, Rebecca Kleinberger, Le Laboratoire, and the Dalai Lama Center at MITVocal Vibrations is exploring the relationships between human physiology and the resonant vibrations of the voice. The voice and body are instruments everyone possesses–they are incredibly individual, infinitely expressive, and intimately linked to one's own physical form. In collaboration with Le Laboratoire in Paris and the Dalai Lama Center at MIT, we are exploring the hypothesis that the singing voice can influence mental and physical health through physicochemical phenomena and in ways consistent with contemplative practices. We are developing a series of multimedia experiences, including individual "meditations," a group "singing circle," and an iPad application, all effecting mood modulation and spiritual enhancement in an enveloping context of stunningly immersive, responsive music. For Fall 2013, we are developing a vocal art installation in Paris where a private "grotto” environment allows individual visitors to meditate using vibrations generated by their own voice, augmented by visual, acoustic, and physical stimuli.
