Picture the scene – you've just rattled off a long list of everything that's going wrong in your life, and your friend replies, 'sounds like you're having a great day'. You'll most likely assume they're being ironic and carry on with the conversation as if they've just offered up words of sympathy. Put simply, you understand that the sentiment they're expressing doesn't match their words if taken at face value.

We Brits can be a sarcastic bunch, and growing up in this culture gives us the ability to pick up on contextual cues and understand when someone means what they say, and when they don't. A machine, while able to hear and understand the words we use to communicate, may have more difficulty comprehending such subtleties – and that's where sentiment analysis technology is at this point in time.

That's not to say that the technology is of no use – quite the opposite, in fact. Sentiment analysis can take a significant proportion of the grunt work out of interpreting qual research data. But, for now at least, the human touch is still required to make sure that it doesn't make the honest mistake of misinterpreting familiar words and phrases when they're used in unusual contexts.

Sentiment analysis is great for market research

Sentiment analysis, for the uninitiated, is technology that automatically analyses the words used in verbal communication to interpret the emotion being expressed.  It's used in market research in various ways, from poring through social media posts to discover whether the public reaction to a product or topic is positive or negative, to analysing the prevalent emotion expressed in responses to a question in a market research survey.

If you're a researcher, this technology is great – it cuts out the boring part of data analysis by navigating you towards interesting content, and it saves you the effort of categorising the responses yourself. However advanced it may be though, sentiment analysis won't replace humans and leave them jobless – at least not anytime soon – so there's no need to start updating your CV just yet.

More than words – why humans are needed

For all the benefit this technology brings, the finer points of the analysis still have to be done by humans – the AI used is fairly advanced, but its work still has to be checked through. The word 'great', for instance, can have two opposite meanings depending on how it’s said, plus a third when combined with big, as in 'a great big mess'. For a practical example, take these videos of someone saying 'the service at this restaurant is unbelievable' – as you can see, the same phrase can mean very different things depending on the tone of voice, facial expressions and body language used to express it, and this is something that sentiment analysis technology simply won't pick up on.

Language is complex – especially English, which is commonly spoken as a second language and therefore subject to misunderstanding. Also, because of the varied background of the language, there are often multiple words for the same thing, words that have different meanings in different contexts, and expressions that frankly make no sense from a logical point of view (ever thought about why we say that we’re in a car, but on a bus?)

Expecting technology to navigate all of these complexities flawlessly is like expecting a dog to throw a ball for itself – both have limitations that cannot be overcome without human intervention. And when you add in colloquialisms, idioms, slang and the many new words that appear in common usage on an almost daily basis, it starts to feel a little harsh to expect technology to cope with it all.

Technology won’t replace us completely

So why shouldn't we expect machines to be able to do what we can when it comes to interpreting the range of human emotion, whether expressed verbally or non-verbally? The fact is that we have techniques for using and understanding language that most of us are not even aware of, as nicely demonstrated by this BBC article written by author Mark Forsyth about a paragraph of his book, The Elements of Eloquence, that went viral last year.

Have you ever tried, without success, to explain a rule of the English language to a foreigner, only to end up getting exasperated and telling them 'that's just the way it is'? If we're able to pick up on nuances and subtleties without even knowing how we do it ourselves, how can we expect to programme a machine to do it? Sure, using Plotto’s video research tools to combine sentiment analysis with facial expression recognition can go a long way towards clearing up any uncertainties, but the fact is that we're still some way from an age when computers can do it all for us. And when you stop and think about it, is that such a bad thing? After all, computers don't need jobs to survive, and we don't really want them getting so smart that they take over the world now, do we?

Sentiment analysis – it’s all good news

Sentiment analysis technology is great, and it’s definitely worth using for market research. It'll do much of the boring part of your job and save you time and effort in the process, meaning you’ll be able to do your job better – all good things. And don’t let that worry you because it won't be replacing you anytime soon. This is not because the technology is flawed though – it's because we can't expect it to fully cater for the complexity of verbal language and all the ways people use it to express themselves in combination with other linguistic cues – at least, not just yet.

Thanks for the image @agkdesign