Evaluating ChatGPT's Performance for Multilingual and Emoji-based Hate Speech Detection

by   Mithun Das, et al.

Hate speech is a severe issue that affects many online platforms. So far, several studies have been performed to develop robust hate speech detection systems. Large language models like ChatGPT have recently shown a great promise in performing several tasks, including hate speech detection. However, it is crucial to comprehend the limitations of these models to build robust hate speech detection systems. To bridge this gap, our study aims to evaluate the strengths and weaknesses of the ChatGPT model in detecting hate speech at a granular level across 11 languages. Our evaluation employs a series of functionality tests that reveals various intricate failures of the model which the aggregate metrics like macro F1 or accuracy are not able to unfold. In addition, we investigate the influence of complex emotions, such as the use of emojis in hate speech, on the performance of the ChatGPT model. Our analysis highlights the shortcomings of the generative models in detecting certain types of hate speech and highlighting the need for further research and improvements in the workings of these models.


page 1

page 2

page 3

page 4


Data Bootstrapping Approaches to Improve Low Resource Abusive Language Detection for Indic Languages

Abusive language is a growing concern in many social media platforms. Re...

Multilingual HateCheck: Functional Tests for Multilingual Hate Speech Detection Models

Hate speech detection models are typically evaluated on held-out test se...

Multilingual Multi-Figurative Language Detection

Figures of speech help people express abstract concepts and evoke strong...

HateCheck: Functional Tests for Hate Speech Detection Models

Detecting online hate is a difficult task that even state-of-the-art mod...

To BAN or not to BAN: Bayesian Attention Networks for Reliable Hate Speech Detection

Hate speech is an important problem in the management of user-generated ...

Hate is the New Infodemic: A Topic-aware Modeling of Hate Speech Diffusion on Twitter

Online hate speech, particularly over microblogging platforms like Twitt...

Investigating the stylistic relevance of adjective and verb simile markers

Similes play an important role in literary texts not only as rhetorical ...

Please sign up or login with your details

Forgot password? Click here to reset