Publication

Zero-Shot Transfer Learning to Enhance Communication for Minimally Verbal Individuals with Autism using Naturalistic Data

Narain, J*., Johnson, K*., Picard, R., and Maes, P. “Zero-Shot Transfer Learning to Enhance Communication for Minimally Verbal Individuals with Autism using Naturalistic Data”. NeurIPS 2019 Joint Workshop on AI for Social Good. Accepted. *Equal Contribution.

Abstract

We use zero-shot transfer learning with models trained on a generic database and applied to a sparsely labeled, highly individualized audio dataset for the specialized population of people with minimally verbal autism (mvASD). Using an iterative participatory design approach, we developed a framework for collecting naturalistic data, including an open-source custom app that enables real-time data labeling. We then trained LSTM models on subclasses of generic audio embeddings from the AudioSet database and applied these models to audio recordings of a young autistic boy with no spoken words. The results show the importance of machine learning to enhance translational communication technologies and reduce inequalities in unique and underserved populations.

Related Content