Are We Safe Yet? The Limitations of Distributional Features for Fake News Detection

by   Tal Schuster, et al.

Automatic detection of fake news — texts that are deceitful and misleading — is a long outstanding and largely unsolved problem. Worse yet, recent developments in language modeling allow for the automatic generation of such texts. One approach that has recently gained attention detects these fake news using stylometry-based provenance, i.e. tracing a text's writing style back to its producing source and determining whether the source is malicious. This was shown to be highly effective under the assumption that legitimate text is produced by humans, and fake text is produced by a language model. In this work, we identify a fundamental problem with provenance-based approaches against attackers that auto-generate fake news: fake and legitimate texts can originate from nearly identical sources. First, a legitimate text might be auto-generated in a similar process to that of fake text, and second, attackers can automatically corrupt articles originating from legitimate human sources. We demonstrate these issues by simulating attacks in such settings, and find that the provenance approach fails to defend against them. Our findings highlight the importance of assessing the veracity of the text rather than solely relying on its style or source. We also open up a discussion on the types of benchmarks that should be used to evaluate neural fake news detectors.


page 1

page 2

page 3

page 4


This Just In: Fake News Packs a Lot in Title, Uses Simpler, Repetitive Content in Text Body, More Similar to Satire than Real News

The problem of fake news has gained a lot of attention as it is claimed ...

A Stylometric Inquiry into Hyperpartisan and Fake News

This paper reports on a writing style analysis of hyperpartisan (i.e., e...

Exploring Semantic Perturbations on Grover

With news and information being as easy to access as they currently are,...

Human Brains Can't Detect Fake News: A Neuro-Cognitive Study of Textual Disinformation Susceptibility

The spread of digital disinformation (aka "fake news") is arguably one o...

Concepts and Experiments on Psychoanalysis Driven Computing

This research investigates the effective incorporation of the human fact...

Multiscale Positive-Unlabeled Detection of AI-Generated Texts

Recent releases of Large Language Models (LLMs), e.g. ChatGPT, are aston...

A Private Watermark for Large Language Models

Recently, text watermarking algorithms for large language models (LLMs) ...

Please sign up or login with your details

Forgot password? Click here to reset