Project

Speech Companion

Groups

Speech Companion is an exploration in the domain of real-time extraction of musicality from speech. Speech is one of the richest and most ubiquitous modalities of communication used by human beings. Its richness lies in the combination of linguistic and nonlinguistic information. Musicality is one of the most crucial nonlinguistic components of speech and covers tempo and rhythms of the speaker as well as the pitch variation and unique texture of the vocal sounds. Abstracting musicality from a speech in real time presents several challenges from latency to subjective pitch identification or recognizing voiced/unvoiced sounds. In this paper, we describe a new system for real-time extraction of the music present in everyday speech based on time and pitch quantization. Our system offers several modes from a simple synchronized melody line to a more complex accompaniment much like a singer accompanying herself at the guitar. 

 With such a system, we offer a proof of concept and a working prototype to explore the real-life situations where the music of speech impacts speakers or listeners such as in the contexts of infant-directed speech, language acquisition, human-animal communication, speech pathology, aphasia reeducation, or even music learning and musical composition.