Overview of CheckThat! 2020: Automatic Identification and Verification of Claims in Social Media

by   Alberto Barron-Cedeno, et al.

We present an overview of the third edition of the CheckThat! Lab at CLEF 2020. The lab featured five tasks in two different languages: English and Arabic. The first four tasks compose the full pipeline of claim verification in social media: Task 1 on check-worthiness estimation, Task 2 on retrieving previously fact-checked claims, Task 3 on evidence retrieval, and Task 4 on claim verification. The lab is completed with Task 5 on check-worthiness estimation in political debates and speeches. A total of 67 teams registered to participate in the lab (up from 47 at CLEF 2019), and 23 of them actually submitted runs (compared to 14 at CLEF 2019). Most teams used deep neural networks based on BERT, LSTMs, or CNNs, and achieved sizable improvements over the baselines on all tasks. Here we describe the tasks setup, the evaluation results, and a summary of the approaches used by the participants, and we discuss some lessons learned. Last but not least, we release to the research community all datasets from the lab as well as the evaluation scripts, which should enable further research in the important tasks of check-worthiness estimation and automatic claim verification.


Overview of the CLEF-2018 CheckThat! Lab on Automatic Identification and Verification of Political Claims. Task 1: Check-Worthiness

We present an overview of the CLEF-2018 CheckThat! Lab on Automatic Iden...

CheckThat! at CLEF 2020: Enabling the Automatic Identification and Verification of Claims in Social Media

We describe the third edition of the CheckThat! Lab, which is part of th...

Accenture at CheckThat! 2021: Interesting claim identification and ranking with contextually sensitive lexical training data augmentation

This paper discusses the approach used by the Accenture Team for CLEF202...

The workweek is the best time to start a family – A Study of GPT-2 Based Claim Generation

Argument generation is a challenging task whose research is timely consi...

Z-Index at CheckThat! Lab 2022: Check-Worthiness Identification on Tweet Text

The wide use of social media and digital technologies facilitates sharin...

Overview of CLEF 2019 Lab ProtestNews: Extracting Protests from News in a Cross-context Setting

We present an overview of the CLEF-2019 Lab ProtestNews on Extracting Pr...

Cognitive Analysis of Security Threats on Social Networking Services: Slovakia in need of stronger action

This short paper examines some of the ongoing research at the UMB Data a...

Please sign up or login with your details

Forgot password? Click here to reset