A quick overview of the ideas behind our new paper ‘Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasm’ and a code example showing how easy it is to apply our model pretrained on 1.2 billion tweets to new tasks.
Why emotion analysis?
For a long time I’ve felt that there’s been an important opportunity to improve the modeling of emotional content in text. While most computer science research in this field has focused on positive/negative sentiment analysis, the three dominant theories of emotion agree that humans operate with much more nuanced emotion representations . Some recent computer science research has tried to go beyond the 1-dimensional sentiment measure, but we’re still very far from capturing the full richness of human emotions expressed through language . With our work (accepted at EMNLP 2017), we try to learn richer representations of emotional content in text than what has been done previously.
Why is this relevant? Because emotional content is an important part of language. The classic use case is companies wanting to make sense of what their customers are saying about them. But there are many other use cases as well now that NLP is becoming an increasingly important part of consumer products. For instance, all of the chatbot services (Siri, Alexa and many others) might benefit from having a nuanced understanding of emotional content in text.
I personally experienced the issue of the limited sentiment analysis capabilities when I recently wanted to examine trends in offensive language on social media with colleagues at MIT. We quickly found that even when using all publicly available datasets annotated for offensive language, it was not going to be enough for the model to get a decent understanding of the nuances of offensive language. Using pretrained word vectors helped, but these word vectors are trained to predict the surrounding words and thus treat e.g. “happy” and “sad” as quite similar due to them often occurring with similar surrounding words . Perhaps even more importantly, word vector models are often trained in a bag-of-words manner, making it difficult for them to capture the impact of negations (e.g. ‘not’), amplifications (e.g. ‘very’) as well as more complex sequential patterns in text.
Our model, however, does not suffer from this shortcoming. For instance, the model can capture slang such as ‘this is the shit’ being a positive statement as well as very varied usage of the word ‘love’ (see predictions from our model below). We also make an online demo available at deepmoji.mit.edu.