How do you overcome sentiment analysis limitations and biases?

Sentiment is usually a top 3 metric you'll get asked to pull or report on, and I think most folks think about this in binary terms: is the discussion topic positive? Or negative? (and neutral of course, or mixed, even) Despite being such an easy metric to summon it's very nuanced, as the AI says here. It's biggest challenge is with what's listed here, sarcasm, slang, emojis, and other speech-related issues. I'll let a linguist give the scientific answer there lol. In healthcare it's also tricky, for e.g., our brands see this all the time, "I have a raging headache, but I took an Advil and now I'm better". Two different and conflicting sentiments, that may net out to neutral. Entity-based sentiment would ideally tell us Advil = Positive.

A significant item to be aware of is if the tool you are using assigns sentiment at the overall mention/record level, or if it analyzes the context of your target concept at the phrase/sentence level. Take the example: "I had a great time at the movie last night. The popcorn was terrible, though. Loved seeing my friends and the film was fantastic!" Assigning sentiment to the overall record would potentially see the whole mention as positive - but if you're analyzing "popcorn", you would be misled if you relied on the overall record classification. Generally speaking, deeper context/phrase/entity based sentiment classification is far more useful (but it's not a "default" approach for many tools out there today).

If you analyse text in different languages, context is especially important and might not be immediately obvious, even to humans who are not native language speakers. A practical example: when analysing the sentiment of social media posts in Spanish language, it took the insight of a native Spanish speaking colleague to recognise sarcasm - this kind of localised context was vital for understanding sentiment.

Some social monitoring tools offer AI features that can be taught how to code with nuance, but I'm yet to find one that's a) reliable and/or b) won't bankrupt me. Currently, we stick with good old fashion sampling - download a random selection of posts - say 150 - and manually code them. Then, we'd heavily caveat that in any reports. While it may seem like a dull task, it's actually quite a useful way to get closer to the conversation about your brand, and spot trends. Of course, this approach won't be appropriate for brands that generate thousands of mentions - but they probably have the $$$ for an effective AI-driven too.

Similar @Daniel to your 'Spanish speaking colleague' ... I had an interesting conversation with an English speaking colleague from India who referred to a 'bomb product', meaning something he really loved ... other culture/linguistic views would take it as the complete sentiment polar opposite!

This is a great point, unless you know the person you're analyzing you have no idea what their tone of voice or personality. And even if you do, you can't always tell. And if you can't tell as a human, what makes you think a computer can? Subjectivity is inherent in this type of data, even if you're transcribing audio data, you'd have to manually note this. AI of course may improve over time to tell that, who knows. My guess if at all at least 10 years. It's hard enough to train analysts to make the call on whether something's positive or negative in a situation like this, that much harder to train computers. Weighting is a good tool to use too, that or look at changes in net sentiment. Take the AI for face value and analyze % change.

It can be dangerous to rely on machine learning alone for accurate sentiment analysis, especially in a field like healthcare, where there may be mixed messages within one statement, and tone of voice subtleties might be mixed with scientific statements. Make use of of human analysts with experience in the field to review and improve your sentiment analysis, particularly in specialised subject areas.

One thing I see missing a lot in this area is what you have pointed out @Daniel - multi-modal analysis for sentiment analysis! ... combining all available modalities from the original source (text/audio/video) brings a far richer understanding of the narrative.

Well said here. And this can be said for every model, if trained with biased inputs from biased people you're going to get biased outputs. Been having this argument a lot lately, that everyone is biased (fact), so how can we train models appropriately? Put a bunch of biases and opinions together and you're more likely to get a more representative sample. Will 3 or 4 of those 100 people let's say carry similar bias, sure, but there's also a greater chance that there's 10 more groups of 3-4 people who have different and unique beliefs. That's what great about social, to me it's inherently less biased. The users I analyze come from different social, economic, and educational backgrounds. This is a much larger debate than social listening.

It can be helpful to think beyond sentiment analysis as simply differentiating between positive vs negative. You can use NLP together with manual analysis to dive deeper into what attitudes and opinions are behind the sentiment itself, and to validate the sentiment analysis.