RISE: Randomized Input Sampling for Explanation of Black-box Models

by   Vitali Petsiuk, et al.

Deep neural networks are increasingly being used to automate data analysis and decision making, yet their decision process remains largely unclear and difficult to explain to end users. In this paper, we address the problem of Explainable AI for deep neural networks that take images as input and output a class probability. We propose an approach called RISE that generates an importance map indicating how salient each pixel is for the model's prediction. In contrast to white-box approaches that estimate pixel importance using gradients or other internal network state, RISE works on black-box models. It estimates importance empirically by probing the model with randomly masked versions of the input image and obtaining the corresponding outputs. We compare our approach to state-of-the-art importance extraction methods using both an automatic deletion/insertion metric and a pointing metric based on human-annotated object segments. Extensive experiments on several benchmark datasets show that our approach matches or exceeds the performance of other methods, including white-box approaches.


page 2

page 3

page 6

page 8

page 9

page 13

page 14

page 15


Iterative and Adaptive Sampling with Spatial Attention for Black-Box Model Explanations

Deep neural networks have achieved great success in many real-world appl...

MFPP: Morphological Fragmental Perturbation Pyramid for Black-Box Model Explanations

With the increasing popularity of deep neural networks (DNNs), it has re...

Don't Paint It Black: White-Box Explanations for Deep Learning in Computer Security

Deep learning is increasingly used as a basic building block of security...

Foiling Explanations in Deep Neural Networks

Deep neural networks (DNNs) have greatly impacted numerous fields over t...

Painting the black box white: experimental findings from applying XAI to an ECG reading setting

The shift from symbolic AI systems to black-box, sub-symbolic, and stati...

The Twin-System Approach as One Generic Solution for XAI: An Overview of ANN-CBR Twins for Explaining Deep Learning

The notion of twin systems is proposed to address the eXplainable AI (XA...

SS-CAM: Smoothed Score-CAM for Sharper Visual Feature Localization

Deep Convolution Neural Networks are often referred to as black-box mode...

Please sign up or login with your details

Forgot password? Click here to reset