MIT Media Lab colorbar
MIT Media Lab


GENERAL OVERVIEW Autism spectrum disorders (ASD) refer to a group of complex neurodevelopmental disorders that cause social, communication, and behavioral impairments in children and adults. ASD currently affects 1 in 110 children in the United States; as adults, most people on the autism spectrum find it difficult to live independently, maintain employment, or sustain relationships with those outside of their caregiver networks. In addition, ASD has a substantial economic impact on society, with the lifetime cost of care for those with ASD exceeding $3 million per person.

MIT Media Lab researchers Rosalind Picard, Matthew Goodwin, and Rana el Kaliouby, along with collaborators at Georgia Tech, Carnegie Mellon, the University of Southern California, Boston University, and the University of Illinois at Urbana-Champaign, recently received a joint $10 million, five-year National Science Foundation (NSF) Expeditions award to develop novel technologies for measuring and analyzing behavior during face-to-face social interactions. This groundbreaking collaboration is the first large-scale effort of computer and behavioral scientists to jointly address diagnosing and intervening early in the lives of those with ASD.

Over the past several years, Media Lab researchers have developed a number of technologies to track and measure emotional states, and, working with the ASD community, refined them into specialized tools. These technologies include novel wearable physiological sensors and corresponding software that can be used to measure variables such as heart rate and skin conductivity—indicators of internal stress and arousal. These technologies provide new, more precise ways to understand behavior, and will help individuals with ASD communicate cognitive and emotional states, as well as help others—scientists, therapists, teachers, caregivers—to understand those states. As part of the multidisciplinary NSF Expeditions effort, the Media Lab team will apply this expertise to the building of test beds and methodologies for capturing behavior data accurately and noninvasively, resulting in a more objective, data-driven approach to behavioral assessment.
RESEARCH OVERVIEW There is neither a known cause of ASD nor a biological marker for identifying the disorder, but a growing body of research suggests that behavioral features can be identified in children as young as 12-24 months. It has also been shown that identifying and providing intervention support for ASD at an early age—when the brain is undergoing rapid neural development—can significantly improve outcomes. As the number of people diagnosed with ASD increases, however, it is outpacing the number of qualified experts who can help.

By developing innovative methods to collect fine-grained behavioral data over extended periods of time, this project stands to facilitate large-scale, objective screening and more effective delivery and assessment of therapy to many in need, including families who are socio-economically disadvantaged or miles away from clinical experts.

This multi-year, multi-step program will include research in computer vision, audio and speech analysis, and physiological recording with wireless sensors. Work in these areas will then be combined into a multimodal system for:
  • Data collection in natural environments such as homes, schools, or clinics;
  • Data analysis; and
  • Real-time, intuitive visualizations of data that will make the information understandable to non-experts.
A NEW FIELD EMERGES This NSF-funded collaborative project aims to combine computer science and behavioral science to create an entirely new field of inquiry: computational behavioral science. Just as medical imaging such as MRIs or X-rays have allowed doctors to identify and treat previously unobservable problems (a brain tumor, aneurism, a broken bone), computational behavioral science aims to quantify behavioral dynamics in a concrete and objective way. Ultimately, this will enable more objective and accurate analyses of a wide range of human behavior and development, and many fields—public health, education, advertising, customer relations, security—could benefit from this work.

ABOUT THE EXPEDITIONS PROGRAM The Expeditions program was created by NSF’s Directorate for Computer and Information Science and Engineering (CISE) to provide the CISE research and education community with the opportunity to pursue ambitious, fundamental research agendas that promise to define the future of computing and information.

  • Rosalind W. Picard
    Professor, MIT Media Lab
    Head, Affective Computing Group, MIT Media Lab
    Co-Director, Autism & Communication Technology Initiative, MIT Media Lab
  • Matthew Goodwin
    Director of Clinical Research, MIT Media Lab
    Co-Director, Autism & Communication Technology Initiative, MIT Media Lab
  • Rana el Kaliouby
    Research Scientist, MIT Media Lab

ASSOCIATED RESEARCH ENTITIES Affective Computing Research Group
Autism & Communication Technology Initiative

READ NSF Announcement: "NSF Announces New Expeditions in Computing Awards" (August 19, 2010)

physiological sensor

click image to view larger
download high-res zip file
physiological sensor

click image to view larger
download high-res zip file
physiological sensor

click image to view larger
download high-res zip file
All images photo credit: MIT Media Lab | Download high-res image archive (3 files)
PHOTO CAPTION A wristband invented by MIT Media Lab researchers and being commercialized by an outside company captures various types of physiological data, such as movement, temperature, and electrodermal activity—the peripheral measure of the sympathetic nervous system, otherwise known as the “fight or flight” response. Lightweight and wireless, the wristband enables data collection in natural environments over long periods of time, providing researchers with the information needed to more accurately and objectively diagnose and support individuals with autism spectrum disorders.

SAMPLE RESEARCH PROJECTS Interactive Social Emotional Toolkit
Funded by the National Science Foundation
This project is creating a portable camera system capable of perceiving and visualizing social-emotional information in real-time human interaction. Our human-centered, participatory approach to the co-design and use of technology directly involves individuals on the autism spectrum and leverages their solutions to systematize social interactions, thereby empowering them to enhance their relationships with others while participating in the development of next-generation social-emotional intelligent technologies.

Computerized Interventions to Promote Verbal Expression
Funded by the Nancy Lurie Marks Family Foundation
This project is developing innovative computerized intervention technologies that assist individuals with an autism diagnosis in understanding and producing verbal expressions that carry much of the communicative and emotional information in language. This research aims to improve social communication capacities, as well as enable speech-language therapists, teachers, and parents to assess and teach verbal expression in a novel and fun way that is individually tailored for each person’s interest and sensory and perceptual capabilities.

Wearable Toolkit for Measuring and Communicating Physiological Arousal
Funded by the Nancy Lurie Marks Family Foundation
This project utilizes state-of-the-art knowledge in technology to construct a comfortable, low-cost wireless toolkit that makes it possible for people to continuously monitor and communicate autonomic arousal in daily life, including activity at home, school, and in community settings. To the extent that such technology is adapted for use by people who have difficulties communicating how they feel, there arise abundant opportunities for individuals to learn about and communicate their internal states. We are especially interested in exploring whether this technology can be used to reduce the incidence and severity of sensory and cognitive overload and stress-related behaviors by enabling individuals to communicate their internal arousal state to others who can help co-regulate.

Assessing and Communicating Movement Stereotypy and Arousal Telemetrically
Funded by Autism Speaks and the Nancy Lurie Marks Family Foundation
This project is exploring the relationship between movement stereotypy and physiological activity in persons with autism by combining state-of-the-art ambulatory heart-rate monitors to objectively assess arousal across settings; and wireless, wearable motion sensors and pattern-recognition software that can automatically and accurately detect stereotypical motor movements in real time. Obtaining objective and accurate measures of movement stereotypy and autonomic arousal is critical to understanding why individuals engage in repetitive movements. These measures may also provide non-verbal individuals with a means to communicate arousal to caregivers who can help them regulate stress and thereby enhance their ability to focus, attend, and learn.

PRESS CONTACT Alexandra Kahn
Senior Press Liaison
MIT Media Lab
akahn **at**
617 253-0365

MIT Media Lab       MIT