Oct. 1, 2019
Narain, J.*, Johnson, K.T.*, Picard, R.W., and Maes, P. “Zero-Shot Transfer Learning to Enhance Communication for Minimally Verbal Individuals with Autism using Naturalistic Data”. NeurIPS 2019 Joint Workshop on AI for Social Good. (*Co-first authors/Equal contribution)
We use zero-shot transfer learning with models trained on a generic database and applied to a sparsely labeled, highly individualized audio dataset for the specialized population of people with minimally verbal autism (mvASD). Using an iterative participatory design approach, we developed a framework for collecting naturalistic data, including an open-source custom app that enables real-time data labeling. We then trained LSTM models on subclasses of generic audio embeddings from the AudioSet database and applied these models to audio recordings of a young autistic boy with no spoken words. The results show the importance of machine learning to enhance translational communication technologies and reduce inequalities in unique and underserved populations.