Contributors

🎉 🎉 We are making a more advance reading list for hate speech papers here. Please checkout and provide feedback

Reading List for Hate Speech Research 📖

A reading list of relevant research papers on Hate speech and related issues. This list is maintained by Binny Mathew and Punyajoy Saha, at CNeRG Lab, IIT Kharagpur

In the past few years we have witnessed an increase in the number of hate speech incidents world wide. While there is a rich literature in the social sciences, the research on the computational aspects have just started. This list is an effort to create a one stop comprehensive guide for all research related to Hate Speech. The list is still incomplete and the categorization might be inappropriate.

We will keep adding papers and improving the list. Any suggestions are super welcome 😄

Introduction
Dataset Papers
Survey Papers
Interesting Papers
Code Available
Shared Tasks
Papers
- Amharic
- Arabic
- Dutch
- English
- German
- Hindi
- Indonesian
- Italian
- Kenya
- Vietnamese
Videos
Blogs
Interesting Reads

Introduction

The Internet is one of the greatest innovations of mankind which has brought together people from every race, religion, and nationality. Social media sites such as Twitter and Facebook have connected billions of people and allowed them to share their ideas and opinions instantly. That being said, there are several ill consequences as well such as online harassment, trolling, cyber-bullying, and hate speech.

Datasets

Yi-Ling Chung, Elizaveta Kuzmenko, Serra Sinem Tekiroglu, and Marco Guerini. CONAN - COunter NArratives through Nichesourcing:a Multilingual Dataset of Responses to Fight Online Hate Speech. 2019. ACL
Jing Qian, Anna Bethke, Yinyin Liu, Elizabeth Belding, and William Yang Wang. A Benchmark Dataset for Learning to Intervene in Online Hate Speech". 2019. arXiv
Jing Qian, Mai ElSherief, Elizabeth Belding, and William Yang Wang. Learning to Decipher Hate Symbols. 2019. NAACL
Binny Mathew, Hardik Tharad, Subham Rajgaria, Prajwal Singhania, Suman Kalyan Maity, Pawan Goyal, and Animesh Mukherje. Thou shalt not hate: Countering Online Hate Speech. 2019. ICWSM
Antigoni-Maria Founta, Constantinos Djouvas, Despoina Chatzakou, Ilias Leontiadis, Jeremy Blackburn, Gianluca Stringhini, Athena Vakali, Michael Sirivianos, and Nicolas Kourtellis. Large Scale Crowdsourcing and Characterization of Twitter Abusive Behavior. 2018. ICWSM
Ziqi Zhang, David Robinson, and Jonathan Tepper. Detecting hate speech on Twitter using a convolution-GRU based deep neural network. 2018. European Semantic Web Conference
Ona de Gibert, Naiara Perez, Aitor García Pablos, and Montse Cuadros. Hate Speech Dataset from a White Supremacy Forum. 2018. ALW2
Manuela Sanguinetti, Fabio Poletto, Cristina Bosco, Viviana Patti, and Stranisci Marco. An italian Twitter corpus of hate speech against immigrants. 2018. LREC ELRA
Mai ElSherief, Vivek Kulkarni, Dana Nguyen, William Yang Wang, and Elizabeth Belding. Hate lingo: A target-based linguistic analysis of hate speech in social media. 2018. ICWSM
GAB Dataset Link --> What is gab: A bastion of free speech or an alt-right echo chamber. 2018 . WWW
Thomas Davidson, Dana Warmsley, Michael Macy, and Ingmar Weber. Automated hate speech detection and the problem of offensive language. 2017. ICWSM
Lei Gao, and Ruihong Huang. Detecting Online Hate Speech Using Context Aware Models. 2017. RANLP
Zeerak Waseem and Dirk Hovy. Hateful symbols or hateful people? predictive features for hate speech detection on twitter. 2016. NAACL SRW
Dataset Link for the paper A Measurement Study of Hate Speech in Social Media
Dataset Link for the paper Measuring the Reliability of Hate Speech Annotations:The Case of the European Refugee Crisis.

Survey Papers

Paula Fortuna and Sérgio Nunes. A survey on automatic detection of hate speech in text. 2018. ACM Computing Surveys (CSUR)
Anna Schmidt and Michael Wiegand. A survey on hate speech detection using natural language processing. 2017. SocialNLP

Interesting Papers/Books

Aymé Arango, Jorge Pérez and Barbara Poblete,Hate Speech Detection is Not as Easy as You May Think: A Closer Look at Model Validation. 2019 . SIGIR
Iginio Gagliardone, Danit Gal, Thiago Alves, and Gabriela Martinez. Countering online hate speech. 2015. Unesco Publishing
Nadine Strossen HATE: Why We Should Resist it with Free Speech, Not Censorship. A short book review

Code Available

Shared Tasks

Cristina Bosco, Dell'Orletta Felice, Fabio Poletto, Manuela Sanguinetti, and Tesconi Maurizio. Overview of the EVALITA 2018 Hate Speech Detection Task. 2018. EVALITA
Valerio Basile, Cristina Bosco,Elisabetta Fersini,Debora Nozza, Viviana Patti, Francisco Rangel ,Paolo Rosso and Manuela Sanguinetti, Page 54-63, Proceedings of 13th SemEval Workshop. Archived competition link-HatEval@SemEval 2019
Valerio Basile, Cristina Bosco, Elisabetta Fersini, Debora Nozza, Viviana Patti, Francisco Manuel Rangel Pardo, Paolo Rosso, and Manuela Sanguinetti. Semeval-2019 task 5: Multilingual detection of hate speech against immigrants and women in twitter. 2019. International Workshop on Semantic Evaluation

Language Wise

Amharic

2018

Zewdie Mossie, and Jenq-Haur Wang. SOCIAL NETWORK HATE SPEECH DETECTION FOR AMHARIC LANGUAGE. 2018. Computer Science & Information Technology

Arabic

2019

Arijit Ghosh Chowdhury, Aniket Didolkar, Ramit Sawhney and Rajiv Ratn Shah. ARHNet - Leveraging Community Interaction For Detection Of ReligiousHate Speech In Arabic. 2019. ACL:Student Research Workshop

2018

Nuha Albadi, Maram Kurdi, and Shivakant Mishra. Are they Our Brothers? Analysis and Detection of Religious Hate Speech in the Arabic Twittersphere. 2018. ASONAM
Sarah Eissa. Use of hate speech in Arabic language newspapers (2018).

Dutch

2016

Stéphan Tulkens, Lisa Hilte, Elise Lodewyckx, Ben Verhoeven, and Walter Daelemans. A Dictionary-based Approach to Racism Detection in Dutch Social Media. 2016. TA-COS

English

2020

Raul Gomez, Jaume Gibert, Lluis Gomez, and Dimosthenis Karatzas. Exploring Hate Speech Detection in Multimodal Publications. 2020. IEEE Winter Conference on Applications of Computer Vision
Manoel Horta Ribeiro, Jeremy Blackburn, Barry Bradlyn, Emiliano De Cristofaro, Gianluca Stringhini, Summer Long, Stephanie Greenberg, and Savvas Zannettou. From Pick-Up Artists to Incels: A Data-Driven Sketch of the Manosphere. 2020. arXiv

2019

Bertie Vidgen, Taha Yasseri, and Helen Margetts. Trajectories of Islamophobic hate amongst far right actors on Twitter. 2019 arXiv
Kosisochukwu Judith Madukwe, and Xiaoying Gao. The Thin Line Between Hate and Profanity. 2019. Australasian Joint Conference on Artificial Intelligence
Sohail Akhtar, Valerio Basile, and Viviana Patti. A New Measure of Polarization in the Annotation of Hate Speech. 2019. International Conference of the Italian Association for Artificial Intelligence (AI*IA)
Akash Gautam, Puneet Mathur, Rakesh Gosangi, Debanjan Mahata, Ramit Sawhney, and Rajiv Ratn Shah. # MeTooMA: Multi-Aspect Annotations of Tweets Related to the MeToo Movement. 2019. arXiv
Marzieh Mozafari, Reza Farahbakhsh, and Noel Crespi. A BERT-based transfer learning approach for hate speech detection in online social media. 2019. International Conference on Complex Networks and Their Applications
Pinkesh Badjatiya, Manish Gupta, and Vasudeva Varma. Stereotypical bias removal for hate speech detection task using knowledge-based generalizations. 2019. The World Wide Web Conference (WWW)
Maarten Sap, Dallas Card, Saadia Gabriel, Yejin Choi, Noah A. Smith. The Risk of Racial Bias in Hate Speech Detection. 2019. ACL
Thomas Davidson, Debasmita Bhattacharya, and Ingmar Weber. Racial Bias in Hate Speech and Abusive Language Detection Datasets. 2019. arXiv
Wafa Alorainy, Pete Burnap, Han Liu, and Matthew L. Williams. "The Enemy Among Us": Detecting Cyber Hate Speech with Threats-based Othering Language Embeddings. 2019. ACM Transactions on the Web (TWEB)
Jing Qian, Anna Bethke, Yinyin Liu, Elizabeth Belding, and William Yang Wang. A Benchmark Dataset for Learning to Intervene in Online Hate Speech". 2019. arXiv
Kristian Miok, Dong Nguyen-Doan, Blaž Škrlj, Daniela Zaharie, and Marko Robnik-Šikonja. Prediction Uncertainty Estimation for Hate Speech Classification. 2019. arXiv
Jing Qian, Mai ElSherief, Elizabeth Belding, and William Yang Wang. Learning to Decipher Hate Symbols. 2019. arXiv
Binny Mathew, Punyajoy Saha, Hardik Tharad, Subham Rajgaria, Prajwal Singhania, Suman Kalyan Maity, Pawan Goyal, and Animesh Mukherje. Thou shalt not hate: Countering Online Hate Speech. 2019. ICWSM
Binny Mathew, Ritam Dutt, Pawan Goyal, and Animesh Mukherjee. Spread of hate speech in online social media. 2019. WebSci
Pushkar Mishra, Marco Del Tredici, Helen Yannakoudakis, and Ekaterina Shutova. Author Profiling for Hate Speech Detection. 2019. arXiv
T. Y. S. S. Santosh, and K. V. S. Aravind. Hate Speech Detection in Hindi-English Code-Mixed Social Media Text. 2019. CoDS-COMAD
Junanda Patihullah, and Edi Winarko. Hate Speech Detection for Indonesia Tweets Using Word Embedding And Gated Recurrent Unit. 2019. IJCCS

2018

Wiktor Soral, Michał Bilewicz, and Mikołaj Winiewski. Exposure to hate speech increases prejudice through desensitization. 2018. Aggressive behavior
Mai ElSherief, Shirin Nilizadeh, Dana Nguyen, Giovanni Vigna, and Elizabeth Belding. Peer to peer hate: Hate speech instigators and their targets. 2018. ICWSM
Savvas Zannettou, Barry Bradlyn, Emiliano De Cristofaro, Haewoon Kwak, Michael Sirivianos, Gianluca Stringini, and Jeremy Blackburn. What is gab: A bastion of free speech or an alt-right echo chamber. 2018. WWW
Punyajoy Saha, Binny Mathew, Pawan Goyal, and Animesh Mukherjee. Hateminers: Detecting Hate speech against Women. 2018. arXiv
Resham Ahluwalia, Himani Soni, Edward Callow, Anderson Nascimento, and Martine De Cock. Detecting Hate Speech Against Women in English Tweets. 2018. EVALITA
Ziqi Zhang, and Lei Luo. Hate speech detection: A solved problem? The challenging case of long tail on Twitter. 2018. Semantic Web Preprint
Karsten Müller, and Carlo Schwarz. Fanning the flames of hate: Social media and hate crime. 2018. SSRN
Jing Qian, Mai ElSherief, Elizabeth Belding, and William Yang Wang. Leveraging Intra-User and Inter-User Representation Learning for Automated Hate Speech Detection. 2018. NAACL
David Robinson, Ziqi Zhang, and Jonathan Tepper. Hate speech detection on twitter: Feature engineering vs feature selection. 2018. European Semantic Web Conference
Jing Qian, Mai ElSherief, Elizabeth Belding, and William Yang Wang. Hierarchical CVAE for Fine-Grained Hate Speech Classification. 2018. EMNLP
Tommi Gröndahl, Luca Pajola, Mika Juuti, Mauro Conti, and N. Asokan. All You Need is "Love": Evading Hate-speech Detection. 2018. arXiv
Shervin Malmasi, and Marcos Zampieri. Challenges in discriminating profanity from hate speech. 2018. Journal of Experimental & Theoretical Artificial Intelligence
Joel Finkelstein, Savvas Zannettou, Barry Bradlyn, and Jeremy Blackburn. A quantitative approach to understanding online antisemitism. 2018. arXiv
Alexandra Olteanu, Carlos Castillo, Jeremy Boy and Kush R. Varshney. The Effect of Extremist Violence on Hateful Speech Online. 2018. ICWSM
Bertie Vidgen, and Taha Yasseri. Detecting weak and strong Islamophobic hate speech on social media. 2018. arXiv
Manoel Horta Ribeiro, Pedro H. Calais, Yuri A. Santos, Virgílio AF Almeida, and Wagner Meira Jr. Characterizing and detecting hateful users on twitter. 2018. ICWSM
Joni Salminen, Fabio Veronesi, Hind Almerekhi, Soon-Gvo Jung, and Bernard J. Jansen. Online Hate Interpretation Varies by Country, But More by Individual: A Statistical Analysis Using Crowdsourced Ratings. 2018. SNAMS
Phoey Lee Teh, Chi-Bin Cheng, and Weng Mun Chee. Identifying and categorising profane words in hate speech. 2018. ICCDA
Shruti Phadke, Jonathan Lloyd, James Hawdon, Mattia Samory, and Tanushree Mitra. Framing Hate with Hate Frames: Designing the Codebook. 2018. CSCW
Joni Salminen, Hind Almerekhi, Milica Milenković, Soon-gyo Jung, Jisun An, Haewoon Kwak, and Bernard J. Jansen. Anatomy of online hate: developing a taxonomy and machine learning models for identifying and classifying hate in online news media. 2018. ICWSM

2017

Pinkesh Badjatiya, Shashank Gupta, Manish Gupta, and Vasudeva Varma. Deep learning for hate speech detection in tweets. 2017. WWW
Haji Mohammad Saleem, Kelly P. Dillon, Susan Benesch, and Derek Ruths. A web of hate: Tackling hateful speech in online social spaces. 2017. TA-COS
Eshwar Chandrasekharan, Umashanthi Pavalanathan, Anirudh Srinivasan, Adam Glynn, Jacob Eisenstein, and Eric Gilbert. You can't stay here: The efficacy of reddit's 2015 ban examined through hate speech. 2017. CSCW
Björn Gambäck, and Utpal Kumar Sikdar. Using convolutional neural networks to classify hate-speech. 2017. Workshop on Abusive Language Online
Rijul Magu, Kshitij Joshi, and Jiebo Luo. Detecting the hate code on social media. 2017. ICWSM
Lucas Wright, Derek Ruths, Kelly P. Dillon, Haji Mohammad Saleem, and Susan Benesch Vectors for counterspeech on Twitter. 2017. Workshop on Abusive Language Online

2016

Sarah Hewitt, Thanassis Tiropanis, and Christian Bokhove. The problem of identifying misogynist language on Twitter (and other online social spaces. 2016. WebSci
Pete Burnap, and Matthew L. Williams. Us and them: identifying cyber hate on Twitter across multiple protected characteristics. 2016. EPJ Data Science
Imran Awan. Islamophobia on Social Media: A Qualitative Analysis of the Facebook's Walls of Hate. 2016. International Journal of Cyber Criminology
Silva, Leandro, Mainack Mondal, Denzil Correa, Fabrício Benevenuto, and Ingmar Weber. Analyzing the targets of hate in online social media. 2016. ICWSM
Zeerak Waseem. Are you a racist or am i seeing things? annotator influence on hate speech detection on twitter. 2016. NLP and Computational Social Science

2015

Njagi Dennis Gitari, Zhang Zuping, Hanyurwimfura Damien, and Jun Long. A lexicon-based approach for hate speech detection. 2015. International Journal of Multimedia and Ubiquitous Engineering
Shuhua Liu, and Thomas Forss. New classification models for detecting Hate and Violence web content. 2015. IC3K
Fabio Fasoli, Anne Maass, and Andrea Carnaghi. Labelling and discrimination: Do homophobic epithets undermine fair distribution of resources?. 2015. British Journal of Social Psychology
Jamie Bartlett and Alex Krasodomski-Jones. Counter-speech examining content that challenges extremism online. 2015 DEMOS
Nemanja Djuric, Jing Zhou, Robin Morris, Mihajlo Grbovic, Vladan Radosavljevic, and Narayan Bhamidipati. Hate speech detection with comment embeddings. 2015. WWW

2014

Jamie Bartlett, Richard Norrie, Sofia Patel, Rebekka Rumpel, and Simon Wibberley. Misogyny on twitter. 2014. Demos
Susan Benesch. Countering dangerous speech: new ideas for genocide prevention. 2014.

2013

Irene Kwok, and Yuzhou Wang. Locate the hate: Detecting tweets against blacks. 2013. AAAI
Michal Bilewicz, Mikołaj Winiewski, Mirosław Kofta, and Adrian Wójcik. Harmful Ideas, The Structure and Consequences of Anti‐Semitic Beliefs in Poland. 2013. Political Psychology

2012

Karmen Erjavec and Melita Poler Kovačič. “You Don't Understand, This is a New War!” Analysis of Hate Speech in News Web Sites' Comments. 2012. Mass Communication and Society

2002

Vasu Reddy. Perverts and sodomites: Homophobia as hate speech in Africa. 2002. Southern African Linguistics and Applied Language Studies

German

2019

Sylvia Jaki, and Tom De Smedt. Right-wing German hate speech on Twitter: Analysis and automatic detection. 2019. arXiv

2017

Björn Ross, Michael Rist, Guillermo Carbonell, Benjamin Cabrera, Nils Kurowsky, and Michael Wojatzki. Measuring the reliability of hate speech annotations: The case of the european refugee crisis. 2017. NLP4CMC

Indonesian

2017

Ika Alfina, Rio Mulia, Mohamad Ivan Fanany, and Yudo Ekanata. Hate speech detection in the indonesian language: A dataset and preliminary study. 2017. ICACSIS

2018

M. Ali Fauzi and Anny Yuniarti. Ensemble Method for Indonesian Twitter Hate Speech Detection. 2018. Indonesian Journal of Electrical Engineering and Computer Science

Italian

2018

Manuela Sanguinetti, Fabio Poletto, Cristina Bosco, Viviana Patti, and Stranisci Marco. An italian Twitter corpus of hate speech against immigrants. 2018. LREC ELRA

2017

Fabio Del Vigna, Andrea Cimino, Felice Dell'Orletta, Marinella Petrocchi, and Maurizio Tesconi Hate me, hate me not: Hate speech detection on Facebook. 2017. Italian Conference on Cybersecurity
Fabio Poletto, Marco Stranisci, Manuela Sanguinetti, Viviana Patti, and Cristina Bosco. Hate speech annotation: Analysis of an italian Twitter corpus. 2017. CLiC-it