Advancing human wellbeing by developing new ways to communicate, understand, and respond to emotion

Can a Recommender System Support Treatment Personalisation in Digital Mental Health Therapy? A Quantitative Feasibility Assessment Using Data from a Behavioural Activation Therapy App, Lewis, R., Ferguson, F., Wilks, C., Jones, N., Picard, R.W.; CHI Conference on Human Factors in Computing Systems Extended Abstracts (CHI ’22 Extended Abstracts)

Matthew Groh, Ziv Epstein, Chaz Firestone, Rosalind Picard. "Deepfake detection by human crowds, machines, and machine-informed crowds." Proceedings of the National Academy of Sciences Jan 2022, 119 (1) e2110013119; DOI: 10.1073/pnas.2110013119

Cristina Bustos, Neska Elhaouij, Albert Sole-Ribalta, Javier Borge-Holthoefer, Agata Lapedriza, and Rosalind Picard. "Predicting Driver Self-Reported Stress by Analyzing the Road Scene." in Proceedings of International Conference on Affective Computing and Intelligent Interaction (ACII 2021), preprint arXiv:2109.13225 (2021).

Ferguson, C.* and Lewis, R.*, Wilks, C., Picard, R.W.; The Guardians: Designing a Game for Long-term Engagement with Mental Health Therapy; IEEE Conference of Games 2021 *Joint first authors

Robert Lewis, Yuanbo Liu, Matthew Groh and Rosalind Picard, "Habit Formation Dynamics: Finding Factors Associated with Building Strong Mindfulness Habits" HCI International 2021

R. Lewis, A. Ghandeharioun, S. Fedor, P. Pedrelli, R.W. Picard, D. Mischoulon, "Mixed Effects Random Forests for Personalised Predictions of Clinical Depression Severity", ICML 2021 Computational Approaches to Mental Health workshop

Jinmo Lee, Neska Elhaouij, and Rosalind Picard. 2021. AmbientBreath: Unobtrusive Just-in-time Breathing Intervention Using Multi-sensory Stimulation and its Evaluation in a Car Simulator. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 5, 2, Article 71 (June 2021), 30 pages. DOI:https://doi.org/10.1145/3463493

Groh, Matthew, Caleb Harris, Luis Soenksen, Felix Lau, Rachel Han, Aerin Kim, Arash Koochek, and Omar Badri. "Evaluating Deep Neural Networks Trained on Clinical Images in Dermatology with the Fitzpatrick 17k Dataset." arXiv preprint arXiv:2104.09957 (2021). Accepted at CVPR ISIC 2021 workshop. Publication pending.

Kyung Yun Choi, Jinmo Lee, Neska El Haouij, Rosalind W. Picard, and Hiroshi Ishii. 2021. aSpire: Clippable, Mobile Pneumatic-Haptic Device for Breathing Rate Regulation via Personalizable Tactile Feedback. In CHI Conference on Human Factors in Computing Systems Extended Abstracts (CHI ’21 Extended Abstracts), May 8–13, 2021, Yokohama, Japan. ACM, New York, NY, USA, 7 pages.

Lewis, Robert, et al. "Shaping Habit Formation Insights with Shapley Values: Towards an Explainable AI-system for Self-understanding and Health Behavior Change." CHI 2021 Workshop Paper. Realizing AI in Healthcare: Challenges Appearing in the Wild.

Pedrelli, P., Fedor, S., Ghandeharioun, A., Howe, E., Ionescu, D. F., Bhathena, D., ... & Picard, R. W. (2020). Monitoring changes in depression severity using wearable and mobile sensors. Frontiers in psychiatry, 11, 1413.

Narain, J.*, Johnson, K.T.*, Ferguson, C., O’Brien, A., Talkar, T., Zhang, Y., Wofford, P., Quatieri, T., Picard, R.W.,Maes, P., "Personalized Modeling of Real-World Vocalizations from Nonverbal Individuals," Proceedings of the International Conference on Multimodal Interaction (ICMI), Utrecht, Netherlands, October 2020. (*Co-first authors/Equal contribution)

Johnson, K.T. & Picard, R.W. (2020). Advancing Neuroscience through Wearable Devices. Neuron 108(1), 8-12.

Jeong, Sooyeon, et al. "A Robotic Positive Psychology Coach to Improve College Students’ Wellbeing." Proceedings of the 29th IEEE International Conference on Robot and Human Interactive Communication (Ro-Man 2020) **Best Paper Award**

Sebastian Zepf, Javier Hernandez, Alexander Schmitt, Wolfgang Minker, and Rosalind W. Picard. 2020. Driver Emotion Recognition for Intelligent Vehicles: A Survey. ACM Comput. Surv. 53, 3, Article 64 (June 2020), 30 pages. DOI:https://doi.org/10.1145/3388790

Narain, J.*, Johnson, K.T.*, O’Brien, A., Wofford, P., Maes, P., Picard, R.W. "Nonverbal Vocalizations as Speech: Characterizing Natural-Environment Audio from Nonverbal Individuals with Autism," Proceedings of the Workshop on Laughter and Other Nonverbal Vocalisations, Bielefeld, Germany, October 2020. (*Co-first authors/Equal contribution)

Taylor, Sara. Forecasting Mental Distress using Healthcare Claims Data. 2020. MIT, PhD dissertation.

Johnson, K.T.* & Narain, J.*, Maes, P., Picard, R.W., "Augmenting Natural Communication in Nonverbal Individuals with Autism," International Society for Autism Research (INSAR), Seattle, Washington, May 2020. (*Co-first authors/equal contribution)

Johnson, K.T.*, Narain, J.*, Ferguson, C., Picard, R.W., and Maes, P. The ECHOS Platform to Enhance Communication for Nonverbal Children with Autism: A Case Study. Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems (Case Studies Track of CHI 2020). (*Co-first authors/Equal contribution)

Sebastian Zepf, Neska ElHaouij, Wolfgang Minker, Javier Hernandez, and Rosalind W. Picard. 2020. EmpathicGPS: Exploring the Role of Voice Tonality in Navigation Systems during Simulated Driving. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems (CHI EA '20). Association for Computing Machinery, New York, NY, USA, 1–7. DOI:https://doi.org/10.1145/3334480.338293

Saleh, A.*, Jaques, N.*, Ghandeharioun, A., Shen, J. H., & Picard, R. (2020). Hierarchical reinforcement learning for open-domain dialog. AAAI 2020.

Nosakhare, E., and Picard. R. 2019. Toward Assessing and Recommending Combinations of Behaviors for Improving Health and Well-Being. ACM Trans. Comput. Healthcare 1, 1, Article 4 (December 2019), 29 pages.

Zhang Y, Weninger F, Schuller B, Picard R "Holistic Affect Recognition using PaNDA: Paralinguistic Non-metric Dimensional Analysis." IEEE Transactions on Affective Computing, December 2019.

Narain, J.*, Johnson, K.T.*, Picard, R.W., Maes, P. "Zero-Shot Transfer Learning to Enhance Communication for Minimally Verbal Individuals with Autism using Naturalistic Data," NeurIPS Workshop on AI for Social Good, December 2019. (*Co-first authors/Equal contribution)

Lopez-Martinez, Daniel, Neska Elhaouij, and Rosalind Picard. "Detection of Real-world Driving-induced Affective State Using Physiological Signals and Multi-view Multi-task Machine Learning." In 2019 8th International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW), pp. 356-361. IEEE, 2019.

Jaques N., Ghandeharioun, A., Shen, J., Ferguson, C., Jones, N., Lapedriza, A., Gu, S., Picard, R. (2019). Way Off-Policy Batch Deep Reinforcement Learning of Implicit Human Preferences in Dialog, NeurIPS workshop, Conversational AI.

Ghandeharioun, A.*, Shen, J. H.*, Jaques, N.*, Ferguson, C., Jones, N., Lapedriza, A., & Picard, R. (2019). Approximating interactive human evaluation with self-play for open-domain dialog systems. In Advances in Neural Information Processing Systems (pp. 13658-13669).

Ghandeharioun, A., Eoff, B., Jou, B., & Picard, R. W. (2019). Characterizing Sources of Uncertainty to Proxy Calibration and Disambiguate Annotator and Data Bias. arXiv preprint arXiv:1909.09285.

Umematsu, T., Sano, A., Taylor, S., and Picard, R. "Improving Students' Daily Life Stress Forecasting using LSTM Neural Networks.", 2019 IEICE Society Conference, Japan, September 2019.

A. Ghandeharioun, D. McDuff, M. Czerwinski and K. Rowan, "EMMA: An Emotion-Aware Wellbeing Chatbot," 2019 8th International Conference on Affective Computing and Intelligent Interaction (ACII), Cambridge, United Kingdom, 2019, pp. 1-7.

A. Ghandeharioun, D. McDuff, M. Czerwinski and K. Rowan, "Towards Understanding Emotional Intelligence for Behavior Change Chatbots," 2019 8th International Conference on Affective Computing and Intelligent Interaction (ACII), Cambridge, United Kingdom, 2019, pp. 8-14.

Leslie, G.*, Ghandeharioun, A.*, Zhou, D., & Picard, R. W. (2019, September). Engineering Music to Slow Breathing and Invite Relaxed Physiology. In 2019 8th International Conference on Affective Computing and Intelligent Interaction (ACII) (pp. 1-7). IEEE.

Jones, N., Jaques, N., Pataranutaporn, P., Ghandeharioun, A., & Picard, R. (2019, September). Analysis of Online Suicide Risk with Document Embeddings and Latent Dirichlet Allocation. In 2019 8th International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW) (pp. 1-5). IEEE.

Rudovic, O., Zhang, M, Schuller, B., Picard, R. "Multi-modal Active Learning From Human Data: A Deep Reinforcement Learning Approach", In 2019 International Conference on Multimodal Interaction (ICMI ’19), October 14–18, 2019, Suzhou, China. ACM, New York, NY, USA.

Nosakhare, E., Picard, R. W. Probabilistic Latent Variable Modeling for Assessing Behavioral Influences on Well-Being," ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD ’19), August 4–8, 2019, Anchorage, AK, USA. ACM, New York, NY, USA. https://doi.org/10.1145/3292500.3330738

Rudovic, O., Utsumi, Y., Guerrero, R., Peterson, K., Rueckert, D., Picard, R. W. "Meta-weighted Gaussian Process Experts for Personalized Forecasting of AD Cognitive Changes" Machine Learning for Healthcare Conference (ML4HC2019), August 2019.

Umematsu, T., Sano, A., and Picard, R. "Daytime Data and LSTM can Forecast Tomorrow's Stress, Health, and Happiness," the 41st International Engineering in Medicine and Biology Conference (EMBC), p.2186-2190, Berlin, July 2019.

Umematsu, T., Sano, A., Taylor, S., and Picard, R. "Improving Students' Daily Life Stress Forecasting using LSTM Neural Networks." IEEE International Conference on Biomedical and Health Informatics (BHI), Chicago, Illinois, May 2019. (BEST PAPER AWARD - 1st Prize)

Zepf, S., Dittrich, M., Hernandez, J., Schmitt, A. "Towards Empathetic Car Interfaces: Emotional Triggers while Driving," In the Late Breaking Work of CHI, Glasgow, UK, May 2019.

Lopez-Martinez D, Eschenfeldt P, Ostvar S, Ingram M, Hur C, Picard R. Deep Reinforcement Learning for Optimal Critical Care Pain Management with Morphine using Dueling Double-Deep Q Networks, in Engineering in Medicine and Biology (EMBC). Berlin, Germany ; 2019.

Rudovic, O., Park, H-W., Busche, J., Schuller, B. , Breazeal, C., Picard, R. W., " Personalized Estimation of Engagement from Videos Using Active Learning with Deep Reinforcement Learning," IEEE CVPR -AMFG W, June 2019.

Slavich G.M., Taylor S., Picard R.W. "Stress measurement using speech: Recent advancements, validation issues, and ethical and privacy considerations," Stress, 2019.

Zhang Y, Michi A, Wagner J, Andre E, Schuller B, Weninger F. "A Generic Human-Machine Annotation Framework Based on Dynamic Cooperative Learning." IEEE Transactions on Cybernetics, March 2019.

Taylor S., Ferguson C., Peng F., Schoeneich M., Picard R.W. "Use of In-Game Rewards to Motivate Daily Self-Report Compliance: Randomized Controlled Trial," Journal of Medical Internet Research, 21(1), e11683, 2019.

Hernandez J., McDuff D., Quigley K., Maes P., and Picard R., "Wearable Motion-based Heart-rate at Rest: A Workplace Evaluation," Journal of Biomedical and Health Informatics, 2018.

Rudovic, O., Utsumi, Y., Lee, J., Hernandez, J., Castello Ferrer, E., Schuller, B., Picard, R. "CultureNet: A Deep Learning Approach for Engagement Intensity Estimation from Face Images of Children with Autism." IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2018).

Subramanian, S., Barbieri, R., Johnson, K.T., Brown, E. "Characterizing Electrodermal Activity Using Point Processes in Young Children," 50th Annual Meeting of the Biomedical Engineering Society (BMES), Atlanta, Georgia, October 2018.

Jaques, N., Lazaridou, A., Hughes, E., Gulcehre, C., Ortega, P. A., Strouse, D. J., Leibo, J. Z., and de Freitas, N. "Intrinsic Social Motivation via Causal Influence in Multi-Agent RL," International Conference on Representation Learning (ICLR), New Orleans, Louisiana, May 2019 (submitted).

Dementyev A., Hernandez J., Choi I., Follmer S., Paradiso J., "Epidermal Robots: Wearable Sensors That Climb On The Skin" In Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 2 (3), September 2018.

Exposito, M., Hernandez, J., Picard, R., "Affective Keys: Towards Unobtrusive Stress Sensing of Smartphone Users" In Proceedings of Mobile Human Computer Interaction, Barcelona, Spain, September, 2018

Lopez-Martinez D, Peng K, Steele S, Lee A, Borsook D, Picard R. "Multi-task multiple kernel machines for personalized pain recognition from functional near-infrared spectroscopy brain signals." International Conference on Pattern Recognition (ICPR), Beijing, China, August 2018. "BEST STUDENT PAPER AWARD"

Nosakhare, E. "Probabilistic Latent Variable Modeling for Predicting Future Well-Being and Assessing Behavioral Influences on Mood, Stress and Health", MIT PhD Thesis, September 2018

Sano A, Chen W, Lopez-Martinez D, Taylor S, Picard R. Multimodal Ambulatory Sleep Detection Using LSTM Recurrent Neural Networks. Journal of Biomedical and Health Informatics. 2018.

Johnson, K.T., Taylor, S., Fedor, S., Jaques, N., Chen, W., Picard, R.W. "Vomit Comet Physiology: Autonomic Changes in Novice Flyers." Proceedings of the International Conference of IEEE Engineering in Medicine and Biology Society (EMBC), Honolulu, Hawaii, July 2018.

Amores, J., Hernandez, J., Dementyev, A., Wang, X., Maes, P. "BioEssence: A Wearable Olfactory Display that Monitors Cardio-respiratory Information to Support Mental Wellbeing," Proceedings of the International Conference of IEEE Engineering in Medicine and Biology Society (EMBC), Honolulu, Hawaii, July 2018

Sano,A., Taylor,S., Jaques,N., Chen,W., Lopez-Martinez,D., Nosakhare,E., Rudovic,O., Umematsu,T., and Picard,R., "Mood, Stress and Sleep Sensing with Wearable Sensors and Mobile Phone," The 40th International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), July 2018.

Lopez-Martinez D, Picard R. Continuous pain intensity estimation from autonomic signals with recurrent neural networks, in IEEE Engineering in Medicine and Biology Society (EMBC). Hawaii ; 2018.

Feffer, M., Rudovic, O., Picard, R. "A Mixture of Personalized Experts for Human Affect Estimation." The 14th International Conference on Machine Learning and Data Mining (MLDM). July 2018.

Rudovic, O., Lee, J., Dai, M., Schuller, B. , Picard, R. W., " Personalized machine learning for robot perception of affect and engagement in autism therapy," Science Robotics, June 2018.

Umematsu, T., Sano, A., Taylor, S., and Picard, R. "Improving Stress Forecasting using LSTM Neural Networks." The 40th International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Honolulu, Hawaii, July 2018.

Saunders Wilder, O., Sullivan, J., Johnson, K.T., Palumbo, R.V., Cumpanasiou, C., Picard, R.W., Goodwin, M.S. "Dyadic Physiological Interdependence and Social Reciprocity in ASD," International Society for Autism Research (INSAR), Rotterdam, Netherlands, May 2018.

Johnson, K.T., Ferguson, C.S., Picard, R.W., "Multi-SPRING: Facilitating Social Interaction through a Customizable, Multimodal Learning Platform," International Society for Autism Research (INSAR), Rotterdam, Netherlands, May 2018.

Sano, A., Taylor, S., McHill, A. W., Phillips, A. J. K., Barger, L. K., Klerman, E., and Picard, R., "Identifying objective physiological markers and modifiable behaviors for self-reported stress and mental health status using wearable sensors and Mobile Phones: Observational Study," Journal of Medical Internet Research, 2018;20(6):e210

Utsumi, Y., Rudovic, O., Peterson, K., Guerrero, R., Picard, R. "Personalized Gaussian Processes for Forecasting of Alzheimer's Disease Assessment Scale-Cognition Sub-Scale (ADAS-Cog13)." The 40th International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). May 2018.

Peng F., LaBelle V., Yue E., and Picard R.. "A Trip to the Moon: Personalized Animated Movies for Self-reflection." Proceedings of Conference on Human Factors in Computing Systems. ACM, 2018.

Kleiman, E. M., Turner, B. J., Fedor, S., Beale, E. E., Picard, R. W., Huffman, J. C., & Nock, M. K. "Digital phenotyping of suicidal thoughts." Depression and anxiety, April 2018