Xiao Xiao, Anna Pereira, Hiroshi Ishii

Abstract

The body channels rich layers of information when playing music, from intricate manipulations of the instrument to vivid personifications of expression. But when music is captured and replayed across distance and time, the performer’s body is too often trapped behind a small screen or absent entirely. This paper introduces MirrorFugue III, an interface to experience recorded piano performance that combines the moving keys of a player piano with life-sized projection of the pianist’s hands and upper body. Inspired by reflections on a lacquered grand piano, MirrorFugue III evokes the sense that the virtual pianist is playing the physically moving keys. Through MirrorFugue III, we explore the question of how to viscerally simulate a performer’s presence to create immersive experiences. We discuss design choices, report reactions from users and outline a space of usage scenarios.