Breathing Window is a tool for non-verbal dialogue that reflects on your own breathing while also offering a window on another person's respiration. This prototype is an example of shared human experiences (SHEs) crafted to improve the quality of human understanding and interactions. Our work on SHEs focuses on first encounters with strangers. We meet strangers every day, and without prior background knowledge of the individual we often form opinions based on prejudices and differences. In this work, we bring respiration to the foreground as one common experience of all living creatures.

The live global interactive simulcast of the final February 2014 performance of "Death and the Powers" in Dallas made innovative use of satellite broadcast and Internet technologies to expand the boundaries of second-screen experience and interactivity during a live remote performance. In the opera, Simon Powers uploads his mind, memories, and emotions into The System, represented onstage through reactive robotic, visual, and sonic elements. Remote audiences, via simulcast, were treated as part of The System alongside Powers and the operabots. Audiences had an omniscient view of the action of the opera, as presented through the augmented, multi-camera video and surround sound. Multimedia content delivered to mobile devices, through the Powers Live app, privileged remote audiences with perspectives from within The System. Mobile devices also allowed audiences to influence The System by affecting the illumination of the Winspear Opera House's Moody Foundation Chandelier.

Nothing is more important in today's troubled world than the process of eliminating prejudice and misunderstanding, and replacing them with communication and empathy. We explore the possibility of creating public experiences to dramatically increase individual and community awareness of the power of empathy on an unprecedented scale. We draw on numerous precedents from the Opera of the Future group that have proposed concepts and technologies to inspire and intensify human connectedness (such as Sleep No More, Death and the Powers, Vocal Vibrations, City Symphonies, and Hyperinstruments) and from worldwide instances of transformative shared human experience (such as the Overview Effect, Human Libraries, Immersive Theatre, and non-sectarian spiritual traditions). The objective is to create a model of a multisensory, participatory, spatially radical installation that will break down barriers between people of immensely different backgrounds, providing instantaneous understanding of—as well as long-term commitment to—empathic communication.

Fensadense is a new work for 10-piece ensemble composed by Tod Machover, commissioned for the Lucerne Festival in summer 2015. The project represents the next generation of hyperinstruments, involving the measurement of relative qualities of many performers where previous systems only looked at a single performer. Off-the-shelf components were used to collect data about movement and muscle tension of each musician. The data was analyzed using the Hyperproduction platform to create meaningful production control for lighting and sound systems based on the connection of the performers, with a focus on qualities such as momentum, connection, and tension of the ensemble as a whole. The project premiered at the Lucerne Festival, and a European tour is scheduled for spring 2016.

Hyperproduction is a conceptual framework and a software toolkit that allows producers to specify a descriptive computational model and consequently an abstract state for a live experience through traditional operating paradigms, such as mixing audio or operation of lighting, sound, and video systems. The hyperproduction system is able to interpret this universal state and automatically utilize additional production systems, allowing for a small number of producers to cohesively guide the attention and perspective of an audience using many or very complex production systems simultaneously. The toolkit is under active development and has been used for new pieces such as Fensadense, and to recreate older systems such as those for the original Hyperstring Triolgy as part of the Lucerne Festival in 2015. Work continues to enable new structures and abstraction within the framework.

Expert or fraud, the powerful person in front of an orchestra or choir attracts both hate and admiration. But what is the actual influence a conductor has on the musician and the sounding result? To throw light on the fundamental principles of this special gestural language, we try to prove a direct correlation between the conductor's gestures, muscle tension, and the physically measurable reactions of musicians in onset-precision, muscle tension, and sound quality. We also measure whether the mere form of these gestures causes different levels of stress or arousal. With this research we aim not only to contribute to the development of a theoretical framework on conducting, but also to enable a precise mapping of gestural parameters in order to develop and demonstrate a new system to the optional enhancement of musical learning, performance, and expression.

In a study of human perception of music in relation to different representations of video graphics, this project explores the automatic synchronization in real time between audio and image. This aims to make the relationship seem smaller and more consistent. The connection is made using techniques that rely on audio signal processing to automatically extract data from the music, which subsequently are mapped to the visual objects. The visual elements are influenced by data obtained from various Musical Information Retrieval (MIR) techniques. By visualizing music, one can stimulate the nervous system to recognize different musical patterns and extract new features.

Consider each of our individual voices as a flashlight to illuminate how we project ourselves in society and how much sonic space we give ourselves or others. Thus, turn-taking computation through speaker recognition systems has been used as a tool to understand social situations or work meetings. We present SIDR, a deep learning-based, real-time speaker recognition system designed to be used in real-world settings. The system is resilient to noise, changes in room acoustics, different languages, and overlapping dialogues. While existing systems require the use of several microphones for each speaker or the need to couple video and sound recordings for accurate recognition of a speaker, SIDR only requires a medium-quality microphone or computer-embedded microphone.

Our voice is an important part of our individuality. From the voices of others, we understand a wealth of non-linguistic information, such as identity, social-cultural clues, and emotional state. But the relationship we have with our own voice is less obvious. We don't hear it the way others do, and our brain treats it differently from any other sound. Yet its sonority is deeply connected with how we are perceived by society and how we see ourselves, body and mind. This project is composed of software, devices, installations, and thoughts used to challenge us to gain new insights on our voices. To increase self-awareness, we propose different ways to extend, project, and visualize the voice. We show how our voices sometimes escape our control, and we explore the consequences in terms of self-reflection, cognitive processes, therapy, affective features visualization, and communication improvement.