Project

SpeechBlocks

LSM

Groups

SpeechBlocks is a smartphone app that allows children to explore spelling principles in an open-ended way. Children rearrange words on the screen by pulling them apart and putting them together with their fingers. Each word formed is pronounced aloud by a speech synthesizer. The interactions afforded by the app were informed by current research findings in early literacy learning. To alleviate the fear of making a mistake, and to support fun exploration, we excluded the notion of “right” and “wrong” from our design: Nonsense words are pronounced along with regular words, much to children’s amusement. While the app is intended to help children acquire basic reading and writing skills, it has also turned out to be a medium for self-expression and creativity.

SpeechBlocks is a smartphone app that allows children to explore spelling principles in an open-ended way. Children rearrange words on the screen by pulling them apart and putting them together with their fingers. Each word formed is pronounced aloud by a speech synthesizer. The interactions afforded by the app were informed by current research findings in early literacy learning. To alleviate the fear of making a mistake, and to support fun exploration, we excluded the notion of “right” and “wrong” from our design: Nonsense words are pronounced along with regular words, much to children’s amusement. While the app is intended to help children acquire basic reading and writing skills, it has also turned out to be a medium for self-expression and creativity.

Piloting SpeechBlocks in the Classroom

We conducted several studies with children using SpeechBlocks. We found that:

a) The app facilitated children’s engagement, in part through the fun of unexpected; 

b) It evoked a sense of authorship and self-efficacy;

c) Sometimes children would tell stories about words they made, even if those words surprised them at first

d) The app evoked a significant number of social interactions between children as they explored words.

Please see our paper: Sysoev et. al. (2017). SpeechBlocks: A Constructionist Early Literacy App. In Proceedings of the 2017 Conference on Interaction Design and Children (pp. 248-257). ACM.

Variations of SpeechBlocks

Hindi SpeechBlocks

Hindi is the largest spoken language in India. Written in the Devanagari script which is used for over 120 languages, it is also the fourth-most spoken language in the world. Keeping this in mind as well as the low literacy rates in developing India, we are in the process of creating a Hindi version of Speechblocks. We hope this can be used to complement formal education or be a learning tool in the lack of schooling—for children and adults alike. 

PhonoBlocks

PhonoBlocks is an exploration in understanding the optimal "blocks" for literacy learning. For details, proceed here.

Guided SpeechBlocks

In their play with SpeechBlocks, children often want to spell specific words. If adults are available, the kids will ask them for help. Such cases present valuable learning opportunities, as children’s skills grow largely within the “zone of proximate development” - the range of activities that they cannot do on their own yet, but can perform with some assistance. But what if there is no adult around willing to provide such assistance? Can the app do it on its own? We are trying to answer this question in our recent work. Our experimental system learns about the word that the child wanted to spell through one of the input channels (such as optical character recognition, as in the video below), and then helps to spell the word by highlighting the connection between sounds and letter combinations.

Sentences in SpeechBlocks

In our research, we noticed that even the youngest children often want to create phrases or full sentences after they have spent time playing with letters and individual words. Sometimes this seems to be a more complex form of playful experimentation, other times self-expression or purposeful communication. To investigate this expanded play pattern, we created Sentence SpeechBlocks.

Bilingual SpeechBlocks

Many children in the United States speak a language other than English in the home, yet the literacy learning mobile apps and games available do not reflect the country’s growing number of bilingual children. Furthermore, to our knowledge, none of these apps provide open-ended, constructionist literacy learning opportunities for young bilinguals.

In response, we created Bilingual SpeechBlocks, a version of the constructionist literacy learning app SpeechBlocks, for young children who speak both Spanish and English. This app allows children to tinker with pronunciation in both languages, with the hope that it may help them distinguish between languages at the phonetic level. We built and conducted an exploratory pilot-test of Bilingual SpeechBlocks, observing how children created real or nonsense Spanish or English words and heard how they are pronounced differently in the two languages.

Social SpeechBlocks: Word Pong

We are experimenting with ways to incorporate social elements into our SpeechBlocks prototype to add meaningful ways for children to interact within the app. In this video, you can see children sending their words to one another. A character, which is the child’s avatar on his/her friend’s phone, then pronounces the word aloud.

SpeechStickers

A major motivation for children to write is to leave their mark on the world. Speech Stickers provide them with this opportunity by moving outside of the screen. Children can use Speech Stickers to label their toys or compose letters to friends and family. A computer vision app on the smartphone will assist the child with this as well as with reading the messages of others.