Counterfactual Editing for Search Result Explanation

by   Zhichao Xu, et al.

Recently substantial improvements in neural retrieval methods also bring to light the inherent blackbox nature of these methods, especially when viewed from an explainability perspective. Most of existing works on Search Result Explanation (SeRE) are designed to provide factual explanation, i.e. to find/generate supporting evidence about documents' relevance to search queries. However, research in cognitive sciences have shown that human explanations are contrastive i.e. people explain an observed event using some counterfactual events; such explanations reduce cognitive load, and provide actionable insights. Though already proven effective in machine learning and NLP communities, the formulation and impact of counterfactual explanations have not been well studied for search systems. In this work, we aim to investigate the effectiveness of this perspective via proposing and evaluating counterfactual explanations for the task of SeRE. Specifically, we first conduct a user study where we investigate if counterfactual explanations indeed improve search sessions' effectiveness. Taking this as a motivation, we discuss the desiderata that an ideal counterfactual explanation method for SeRE should adhere to. Next, we propose a method CFE^2 (CounterFactual Explanation with Editing) to provide pairwise explanations to search engine result page. Finally, we showcase that the proposed method when evaluated on four publicly available datasets outperforms baselines on both metrics and human evaluation.


page 1

page 2

page 3

page 4


Global Counterfactual Explanations: Investigations, Implementations and Improvements

Counterfactual explanations have been widely studied in explainability, ...

Explaining Documents' Relevance to Search Queries

We present GenEx, a generative model to explain search results to users ...

GRETEL: A unified framework for Graph Counterfactual Explanation Evaluation

Machine Learning (ML) systems are a building part of the modern tools wh...

Explaining Search Result Stances to Opinionated People

People use web search engines to find information before forming opinion...

Finding Counterfactual Explanations through Constraint Relaxations

Interactive constraint systems often suffer from infeasibility (no solut...

Counterfactually Evaluating Explanations in Recommender Systems

Modern recommender systems face an increasing need to explain their reco...

Counterfactuals of Counterfactuals: a back-translation-inspired approach to analyse counterfactual editors

In the wake of responsible AI, interpretability methods, which attempt t...

Please sign up or login with your details

Forgot password? Click here to reset