Russell, S. "HearThere: Infrastructure for Ubiquitous Augmented-Reality Audio"

Abstract

This thesis presents HearThere, a system to present spatial audio that preserves alignment between the virtual audio sources and the user's environment. HearThere creates an auditory augmented reality with minimal equipment required for the user. Sound designers can create large-scale experiences to sonify a city with no infrastructure required, or by installing tracking anchors can take advantage of sub-meter location information to create more refined experiences.

Audio is typically presented to listeners via speakers or headphones. Speakers make it extremely difficult to control what sound reaches each ear, which is necessary for accurately spatializing sounds. Headphones make it trivial to send separate left and right channels, but discard the relationship between the head the the rest of the world, so when the listener turns their head the whole world rotates with them.

Head-tracking headphone systems have been proposed and implemented as a best of both worlds solution, but typically only operate within a small detection area (e.g. Oculus Rift) or with coarse-grained accuracy (e.g., GPS) that makes up-close interactions impossible. HearThere is a multi-technology solution to bridge this gap and provide large-area and outdoor tracking that is precise enough to imbue nearby objects with virtual sound that maintains the spatial persistence as the user moves throughout the space. Using bone-conduction headphones that don't occlude the ears along with this head tracking will enable true auditory augmented reality, where real and virtual sounds can be seamlessly mixed.