Synthesizing Appropriate Facial Expressions

The heatmap below can be used by an intelligent virtual agent to synthesize appropriate facial expressions during a conversation. The figure was created using data from conversations between people, by computing the correlation between participants' facial expressions (vertical axis) and their partner's facial expression response (horizontal axis). The colour of each tile represents the difference in correlation between conversations with high bonding and conversations with low bonding. Blue tiles represent a correlation that is more strongly positive in high bonding conversations (an appropriate or empathetic response), while red represents a correlation more prevalent in low bonding (an inappropriate response). An agent interacting with a human user can first detect her expression (for example,  Head angle left-right means she is shaking her head), and then examine that row of the table to find the probabilities of synthesizing an appropriate response. We see that responding with Head angle left-right (the agent shaking its own head) should have a high probability, while smiling is a highly inappropriate response. 

Related Content