At Plotto, we are using facial emotion analytics to enhance online video research, providing richer consumer insight and pushing back the boundaries of what you can learn about your consumers from their online web survey. Emotion recognition software is a relatively new technology, but despite the fact it’s in its infancy, increasingly more industries are realising the potential for its application, like advertising, education, gaming and well-being.

Plotto includes facial emotion analysis software that detects anger, contempt, disgust, fear, happiness, sadness and surprise, as well as neutral states. These facial expressions and emotions are universal, communicated cross-culturally without the need of a translator. Physical cues like body language and facial expressions can give away more information than the words someone is saying. Monitoring the emotional cues of your respondents through their expressions is the most efficient way to uncover the truth of what they are really trying to tell you.

With this burgeoning technology forming new trends in online market research, we wanted to take a look at where else facial emotion analytics is being used and how the tech is being developed across these sectors.

Ad Testing

Static surveys are becoming a thing of the past; written questions seeking written responses are no longer enough to glean efficiently detailed data about the success of advertising campaigns. Standard metrics like viewing time and page clicks are becoming increasingly less accurate in measuring genuine consumer appreciation. Today, companies like the BBC are using facial-recognition software from CrowdEmotion, to analyse their survey participants second-by-second, detecting tiny facial movements and measuring how their consumers are relating to content. Passive, engaged, surprised or unimpressed: It’s all recorded for the broadcasters to review. Targeted commercials based on your subconscious likes and dislikes may be the next step along from targeted adverts based on your search history.

Emotion Aware Gaming

Emotion aware gaming is still in its infancy. Nonetheless, Affectiva has developed an Interactive computer game that can detect your emotions and alter the state of play accordingly.  So far they only have one game, called ‘Never Mind’- a psychological thriller that adapts as you play. Still, the tech has arrived and here at Plotto, we’re excited about this new generation of gaming. The Boston-based start-up company has developed software that scans your facial expressions and then shares the data with the game, to adapt the course of action as you play. For example, the more stressed you become the harder the game will be. The game quantifies your emotions using a webcam and modifies itself to react in real time, whether you’re ready for the updates or not!


Facial emotion analysis is set to be used within digital outdoor advertising, like billboards and bus stop signs. Smart signs are set to be the future of digital outdoor advertising. Public reactive tech could vastly improve the user experience, tailoring adverts to match emotions as it recognises them. It would customise messages based on the facial expressions of viewers, gathered from emotion recognition software within the device itself. The tech will be able to recognise six emotional states by tracking a person’s eyes and mouth, including happiness, anger and surprise. Back in 2015, advertising giant M&C Saatchi tested this tech with a Microsoft Kinect camera built into a billboard, evolving its repertoire of adverts based on human interaction. While mood-aware advertising may not be in the public domain just yet, the advent of technologies like touch screens, movement sensitive video games and augmented reality means the public is ready for the next leap in interactive tech.

Video chat and conferencing

Imagine a webinar or a video conference during which you are fed helpful feedback about the participants’ mood. You wouldn’t have to second guess how the other participants of the conference were feeling, or whether the students of your webinar were losing attention and becoming bored. Emotional facial recognition software providers, like Kairos and Affectiva, are using emotional analysis to understand facial expressions in real-time, as you communicate via webcam, taking emotional reactivity to a deeper level than just basic instinct. Video chat is stepping into a whole new era and the emerging technology of emotional facial recognition is taking us smiling, frowning and blinking into the future.

Emotion aware technology to improve well-being

Emotion aware technology could enrich our lives as consumers, using everyday technology to detect, interpret and adjust to human emotions. Facial expressions, physiology, speech patterns and context can come together to paint a rich picture of a person’s emotional state. By using commonplace devices, like tablets and smartphones, our everyday lives can be enriched. This TedX presentation explains how this is set to drastically improve education, healthcare and lifestyle in the near future:

The same tech that we are building into online survey software is already being used in schools to detect struggling students, as well as those who are bored and need more challenging work. Being able to detect the face of a struggling student within a whole room of faces could vastly improve the quality of education worldwide, meaning teachers would be more efficient within large class sizes.

At Plotto, we pride ourselves on being at the forefront of new trends in market research and we’re excited about our emotion aware software and its applications for online video survey.

Plotto’s Emotion Analytics tool recognises the facial expressions of your respondents during their testimonial video and returns an overall summary of their feelings. The technology is able to track subtle changes in mouth and eye shape to give an interpreted emotion, even through sunglasses or other natural obstructions:

The software works in tandem with the Speech-to-Text conversion technology, transcribing your respondents’ answers into a comprehensive script of text. This allows the platform to deliver an automated report of top relevant keywords used, as well as a classification of the video’s overall tone, i.e. positive, negative or neutral. The combination of artificial intelligence and sophisticated analytical software can reveal precious emotion insights from huge quantities of online video research, transforming the quality of your data and uncovering the truth in a way not possible with words alone.

Try using our facial expression recognition tools, speech sentiment indicator and keyword themes detector to analyse your online survey responses and see for yourself how rich consumer insight can be.