Deceptive AI Systems That Give Explanations Are Just as Convincing as Honest AI Systems in Human-Machine Decision Making

by   Valdemar Danry, et al.

The ability to discern between true and false information is essential to making sound decisions. However, with the recent increase in AI-based disinformation campaigns, it has become critical to understand the influence of deceptive systems on human information processing. In experiment (N=128), we investigated how susceptible people are to deceptive AI systems by examining how their ability to discern true news from fake news varies when AI systems are perceived as either human fact-checkers or AI fact-checking systems, and when explanations provided by those fact-checkers are either deceptive or honest. We find that deceitful explanations significantly reduce accuracy, indicating that people are just as likely to believe deceptive AI explanations as honest AI explanations. Although before getting assistance from an AI-system, people have significantly higher weighted discernment accuracy on false headlines than true headlines, we found that with assistance from an AI system, discernment accuracy increased significantly when given honest explanations on both true headlines and false headlines, and decreased significantly when given deceitful explanations on true headlines and false headlines. Further, we did not observe any significant differences in discernment between explanations perceived as coming from a human fact checker compared to an AI-fact checker. Similarly, we found no significant differences in trust. These findings exemplify the dangers of deceptive AI systems and the need for finding novel ways to limit their influence human information processing.


Artificial intelligence is ineffective and potentially harmful for fact checking

Fact checking can be an effective strategy against misinformation, but i...

Using Small MUSes to Explain How to Solve Pen and Paper Puzzles

Pen and paper puzzles like Sudoku, Futoshiki and Skyscrapers are hugely ...

Perceptions of Fairness and Trustworthiness Based on Explanations in Human vs. Automated Decision-Making

Automated decision systems (ADS) have become ubiquitous in many high-sta...

Do People Engage Cognitively with AI? Impact of AI Assistance on Incidental Learning

When people receive advice while making difficult decisions, they often ...

Rating Reliability and Bias in News Articles: Does AI Assistance Help Everyone?

With the spread of false and misleading information in current news, man...

Please sign up or login with your details

Forgot password? Click here to reset