E. Vyzas, Rosalind W. Picard

Abstract

We develop a method for recognizing the emotional state of a person who is deliberately expressing one of eight emotions. Four physiological signals were measured and six features of each of these signals were extracted. We investigated three methods for the recognition: (1) Sequential floating forward search (SFFS) feature selection with K-nearest neighbors classification, (2) Fisher projection on structured subsets of features with MAP classification, and (3) A hybrid SFFS-Fisher projection method. Each method was evaluated on the full set of eight emotions as well as on several subsets. The SFFS attained the highest rate for a trio of emotions, 2.7 times that of random guessing, while the Fisher projection with structured subsets attained the best performance on the full set of emotions, 3.9 times random. The emotion recognition problem is demonstrated to be a difficult one, with day-to-day variations within the same class often exceeding between-class variations on the same day. We present a way to take account of the day information, resulting in an improvement to the Fisher-based methods. The findings in this paper demonstrate that there is significant information in physiological signals for classifying the affective state of a person who is deliberately expressing a small set of emotions.