Can Artificial Intelligence Predict The Spread Of Online Hate Speech?

The rise in online hate speech and the way it is reflected in the offline world is a hot topic in politics right now.

The internet has given everyone a voice, which clearly has positive implications for the way citizens can publicly challenge authority and debate issues. On the other hand, when challenge and debate spill over into attacks on minorities or vulnerable people, there’s obviously a potential for harm.

It’s fairly commonly assumed that this form of hate speech, particularly when encountered alongside other factors such as social deprivation or mental illness, has the potential to radicalize individuals in dangerous ways, and inspire them to commit illegal and violent acts.

Just as terrorist organizations like ISIS can be seen using hate speech in videos and propaganda material intended to incite violence, racist and anti-Islamic material is thought to have inspired killers like Anders Breivik, who killed 69 youths in a 2011 shooting spree, and the 2019 Christchurch mosque shooting in which 51 died.

So far these links between online and real-world actions, though common sense tells us they are likely to exist, have been difficult to prove scientifically. However, a piece of the puzzle fell into place thanks to research carried out by the UN and the Universitat Pompeu Fabra, and co-ordinated by IBM.

IBM principle researcher Kush Varshney tells me “I think the main message was that this was the first study of its kind looking at the relationship between online and offline behaviors, and most importantly it demonstrates why we should be taking this technical approach to studying that relationship.”

Researchers began by compiling a list of keywords and phrases considered by governmental agencies and NGOs to be indicators of hate speech. These included expressions found in both Islamic-extremist and anti-Islamic posts made on Twitter and Reddit. As the researchers validated that these words and phrases were indeed common by searching across those platforms, they came across other co-occurring terms that were also added to the list. Along with news reports of Islamic terrorism or anti-Islamic violence, this list was the primary sources of data for the investigation.

This user-generated content – over 50 million tweets and 300,000 Reddit posts, made by around 15 million users – containing these words and phrases were then classified according to factors including their stance (Islamic-extremist or anti-Islamic), as well as the severity of the message. The scale of severity ranged from simple use of discriminatory language to outright incitement to violence, including genocide.

The study also considered the framing of the comments – whether the point of the post was to define a problem (“Muslims are likely to be terrorists”), diagnose a causes (“Immigration leads to increased terrorism”), make a moral judgement (“Christianity is an evil religion”) or proposes a solution such as carrying out terrorist attacks to achieve political aims.

After the dataset was compiled and classified, a timeline analysis was carried out, using machine learning to draw a picture of the correlation between the number of hate speech messages appearing online, and a number of real-world incidents including the 2016 Orlando nightclub shooting, the 2016 Istanbul airport attack, the 2016 Finsbury Park, London, vehicular attack and the 2016 Olethe, Kansas shooting. All of the incidents involved Muslims or Arabs as either victims or perpetrators, and took place within 19 months.

Previously, the majority of machine learning analysis around the concept of hate speech has focussed on building algorithms to determine whether or not particular posts or pieces of content are hateful.

Varshney tells me “A lot of people in the machine learning community are tackling the problem of classifying whether speech is offensive or hateful – we decided it wasn’t important for us to tackle that problem, and often it’s a question of where you draw the line, if something is verging on being hateful.

“What we were looking at is what’s the relationship between things that happen in the online world, and things that happen in the real world.”

The study found that, yes, following high-profile incidents of both Islamophobic or Islamic-extremist violence, incidents of online hate-speech do indeed increase. This didn’t really come as a surprise to anyone as it was commonly held to be true based on casual observation. But what was far more interesting was the fact that, in the case of Islamist-extremist violence it wasn’t just Muslims who faced an increase in hate speech against them, but attacks were frequently broadened to other minority groups.

Varshney told me, “The severity of the attacks also increases, so people are much more likely to incite violence … and the target of the online messages also broadens so other groups that have nothing to do with anything that’s happened in the real world also experience an increase in hate speech. It could be any other group, such as homosexuals … those were some interesting findings.”

So, is online hate speech and real-world violence a circular problem? It’s been shown that one (real-world violence) causes the other – but is the reverse also true, creating a vicious, self-feeding circle of hatred and violence?

Currently, that remains unclear. But proving the causal relationship between hate speech and violence is a natural next-step for research in the field, Varshney says.

Proving this reverse relationship is likely to be more problematic, however, for a number of reasons. Including the fact that the process of online radicalization itself is not yet well understood from a scientific perspective. The question of how much exposure to hateful material is needed to push a person to commit violence, over what period of time, and how the mental health of the individual plays its part, have yet to be answered.

Varshey told me “That would probably be an even more important study to do – we didn’t get into it in this particular project, as some of the causal relationships require techniques that we don’t yet have.

“That inspires us to do more technical work though – and this direction is clearly a next-step for the work, that should be done, for sure.”

The research, which can be viewed in full here, was carried out as part of IBM’s Science for Social Good, which aimed to apply machine learning to 17 issues identified by the UN as Sustainable Development Goals.


Original post:


31 comentários em “Can Artificial Intelligence Predict The Spread Of Online Hate Speech?

  1. Howdy! I know this is kinda off topic nevertheless I’d figured I’d ask.
    Would you be interested in exchanging links or maybe guest writing a blog post or vice-versa?
    My site goes over a lot of the same subjects as yours and I
    believe we could greatly benefit from each other. If you’re interested feel free to send me an email.

    I look forward to hearing from you! Wonderful blog
    by the way!

  2. I am the business owner of JustCBD company ( and am planning to develop my wholesale side of company. It would be great if anybody at targetdomain give me some advice ! I thought that the best way to do this would be to talk to vape shops and cbd retailers. I was really hoping if someone could recommend a trustworthy site where I can get Vape Shop International Sales Leads I am currently looking at, and Not exactly sure which one would be the very best selection and would appreciate any assistance on this. Or would it be simpler for me to scrape my own leads? Ideas?

  3. I’m the proprietor of JustCBD Store label ( and I’m presently seeking to grow my wholesale side of company. I really hope that anybody at targetdomain is able to provide some guidance 🙂 I thought that the most ideal way to do this would be to reach out to vape companies and cbd retailers. I was hoping if anyone could suggest a qualified web-site where I can buy CBD Shops B2B Mailing List I am currently considering, and Not exactly sure which one would be the best selection and would appreciate any support on this. Or would it be simpler for me to scrape my own leads? Ideas?

  4. Excellent web site you’ve got here.. It’s hard to find good quality writing like yours these days. I seriously appreciate individuals like you! Take care!!

  5. I’m very pleased to uncover this web site. I wanted to thank you for your time just for this fantastic read!! I definitely really liked every part of it and i also have you saved as a favorite to look at new information in your site.

  6. I blog quite often and I really thank you for your information. This great article has really peaked my interest. I’m going to bookmark your website and keep checking for new details about once per week. I subscribed to your RSS feed as well.

  7. Right here is the perfect web site for anyone who would like to understand this topic. You realize so much its almost hard to argue with you (not that I actually will need to…HaHa). You definitely put a fresh spin on a subject that has been discussed for years. Excellent stuff, just wonderful!

  8. After I initially commented I appear to have clicked the -Notify me when new comments are added- checkbox and from now on every time a comment is added I get 4 emails with the same comment. Is there a means you can remove me from that service? Appreciate it!

  9. Achieving your fitness goals does not need a certified personal trainer or an expensive gym memberships, it is not hard to exercise at home. It is easy to go down a training and fitness rabbit hole, however, when you are looking for the best home exercise equipment to outfit your personal home gym.

  10. After checking out a number of the blog posts on your site, I really appreciate your way of writing a blog. I saved it to my bookmark website list and will be checking back soon. Please check out my website too and let me know how you feel.

  11. Having read this I believed it was extremely enlightening. I appreciate you spending some time and energy to put this information together. I once again find myself spending way too much time both reading and leaving comments. But so what, it was still worth it!

  12. Hi, I do believe your site could be having browser compatibility problems. Whenever I look at your blog in Safari, it looks fine however when opening in Internet Explorer, it has some overlapping issues. I simply wanted to provide you with a quick heads up! Apart from that, excellent website!

  13. Hi there! This blog post couldn’t be written much better! Looking through this post reminds me of my previous roommate! He continually kept preaching about this. I most certainly will send this information to him. Fairly certain he’s going to have a very good read. Many thanks for sharing!

  14. This is a really good tip particularly to those new to the blogosphere. Brief but very precise info… Many thanks for sharing this one. A must read article!

  15. The very next time I read a blog, I hope that it does not fail me just as much as this particular one. After all, Yes, it was my choice to read, but I genuinely thought you would probably have something helpful to say. All I hear is a bunch of crying about something you could possibly fix if you were not too busy searching for attention.

  16. Oh my goodness! Impressive article dude! Thank you, However I am experiencing problems with your RSS. I don’t understand the reason why I can’t subscribe to it. Is there anybody getting similar RSS issues? Anybody who knows the answer will you kindly respond? Thanks!!

  17. Can I just say what a relief to find an individual who actually knows what they are talking about online. You actually understand how to bring a problem to light and make it important. More people really need to check this out and understand this side of the story. I was surprised you’re not more popular since you surely have the gift.

  18. Hi! I could have sworn I’ve been to your blog before but after looking at some of the articles I realized it’s new to me. Anyways, I’m certainly delighted I discovered it and I’ll be book-marking it and checking back often!

Leave a Reply

Your email address will not be published. Required fields are marked *