E. Vyzas, Rosalind W. Picard

Abstract

We develop a method for offline and online recognition of the emotional state of a person deliberately expressing one of eight emotions. In terms of offline recognition, this paper presents recent improvements to a method previously developed in the MIT Media Lab, which involved recognition using physiological data collected from an actress over many weeks. The improvements involve (1) more robust handling of day-to-day variations in the data, (2) use of longer episodes of data, (3) use of heart-rate information, extracted from a blood volume pressure sensor, and (4) the use of alternative features. The success rates thus increased from 50.62% to 81.25% for all 8 emotions. Additionally, the method has been adapted to run online, so that it can be used for real-time applications. The performance of the real-time version of the algorithm currently lags 8% behind that of the corresponding offline version, but we continue to investigate improvements. The success rates obtained with the physiological-based recognition are now comparable to those obtained in facial and vocal expression recognition, and offer complementary information or an alternative to such means. The recognition results demonstrated here indicate that there is significant information in physiological signals for classifying the affective state of a person who is deliberately expressing a small set of emotions.