Role of Human-AI Interaction in Selective Prediction

by   Elizabeth Bondi, et al.

Recent work has shown the potential benefit of selective prediction systems that can learn to defer to a human when the predictions of the AI are unreliable, particularly to improve the reliability of AI systems in high-stakes applications like healthcare or conservation. However, most prior work assumes that human behavior remains unchanged when they solve a prediction task as part of a human-AI team as opposed to by themselves. We show that this is not the case by performing experiments to quantify human-AI interaction in the context of selective prediction. In particular, we study the impact of communicating different types of information to humans about the AI system's decision to defer. Using real-world conservation data and a selective prediction system that improves expected accuracy over that of the human or AI system working individually, we show that this messaging has a significant impact on the accuracy of human judgements. Our results study two components of the messaging strategy: 1) Whether humans are informed about the prediction of the AI system and 2) Whether they are informed about the decision of the selective prediction system to defer. By manipulating these messaging components, we show that it is possible to significantly boost human performance by informing the human of the decision to defer, but not revealing the prediction of the AI. We therefore show that it is vital to consider how the decision to defer is communicated to a human when designing selective prediction systems, and that the composite accuracy of a human-AI team must be carefully evaluated using a human-in-the-loop framework.


page 2

page 3

page 5


Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance

Increasingly, organizations are pairing humans with AI systems to improv...

On the Relation between Sensitivity and Accuracy in In-context Learning

In-context learning (ICL) suffers from oversensitivity to the prompt, wh...

Uncalibrated Models Can Improve Human-AI Collaboration

In many practical applications of AI, an AI model is used as a decision ...

Optimizing AI for Teamwork

In many high-stakes domains such as criminal justice, finance, and healt...

Learning Personalized Models of Human Behavior in Chess

Even when machine learning systems surpass human ability in a domain, th...

Human Uncertainty in Concept-Based AI Systems

Placing a human in the loop may abate the risks of deploying AI systems ...

AI-in-the-Loop – The impact of HMI in AI-based Application

Artificial intelligence (AI) and human-machine interaction (HMI) are two...

Please sign up or login with your details

Forgot password? Click here to reset