Model Explanations under Calibration

by   Rishabh Jain, et al.
Imperial College London

Explaining and interpreting the decisions of recommender systems are becoming extremely relevant both, for improving predictive performance, and providing valid explanations to users. While most of the recent interest has focused on providing local explanations, there has been a much lower emphasis on studying the effects of model dynamics and its impact on explanation. In this paper, we perform a focused study on the impact of model interpretability in the context of calibration. Specifically, we address the challenges of both over-confident and under-confident predictions with interpretability using attention distribution. Our results indicate that the means of using attention distributions for interpretability are highly unstable for un-calibrated models. Our empirical analysis on the stability of attention distribution raises questions on the utility of attention for explainability.


Towards Robust Interpretability with Self-Explaining Neural Networks

Most recent work on interpretability of complex machine learning models ...

Rethinking Attention-Model Explainability through Faithfulness Violation Test

Attention mechanisms are dominating the explainability of deep models. T...

Exploring The Role of Local and Global Explanations in Recommender Systems

Explanations are well-known to improve recommender systems' transparency...

Explaining Explanations in AI

Recent work on interpretability in machine learning and AI has focused o...

Where is the Model Looking At?–Concentrate and Explain the Network Attention

Image classification models have achieved satisfactory performance on ma...

Explaining Language Models' Predictions with High-Impact Concepts

The emergence of large-scale pretrained language models has posed unprec...

On Shapley Credit Allocation for Interpretability

We emphasize the importance of asking the right question when interpreti...

Please sign up or login with your details

Forgot password? Click here to reset