McDuff, D. "Crowdsourcing Affective Responses for Predicting Media Effectiveness"

Abstract

Emotion is key to the effectiveness of media, whether it be in influencing memory, likability or persuasion. Stories and narratives, even if fictional, have the ability to induce a genuine emotional response. However, the understanding of the role of emotions in media and advertising effectiveness has been limited due to the difficulty in measuring emotions in real-life contexts. Video advertising is a ubiquitous form of a short story, usually 30-60 seconds in length, designed to influence, persuade, entertain and engage, in which media with emotional content is frequently used. The lack of understanding of the effects of emotion in advertising results in large amounts of wasted time, money and other resources; in this thesis I present several studies measuring responses to advertising.

Facial expressions, heart rate, respiration rate and heart rate variability can inform us about the emotional valence, arousal and engagement of a person. In this thesis I demonstrate how automatically-detected naturalistic and spontaneous facial responses and physiological responses can be used to predict the effectiveness of stories.

I present a framework for automatically measuring facial and physiological responses in addition to self-report and behavioral measures to content (e.g. video advertisements) over the Internet in order to understand the role of emotions in story effectiveness. Specifically, I will present analysis of the first large scale data of facial, physiological, behavioral and self-report responses to video content collected ``in-the-wild" using the cloud. I have developed models for evaluating the effectiveness of media content (e.g. likability, persuasion and short-term sales impact) based on the automatically extracted features. This work shows success in predicting measures of story effectiveness that are useful in creation of content whether that be in copy-testing or content development.