On Model Explanations with Transferable Neural Pathways

by   Xinmiao Lin, et al.

Neural pathways as model explanations consist of a sparse set of neurons that provide the same level of prediction performance as the whole model. Existing methods primarily focus on accuracy and sparsity but the generated pathways may offer limited interpretability thus fall short in explaining the model behavior. In this paper, we suggest two interpretability criteria of neural pathways: (i) same-class neural pathways should primarily consist of class-relevant neurons; (ii) each instance's neural pathway sparsity should be optimally determined. To this end, we propose a Generative Class-relevant Neural Pathway (GEN-CNP) model that learns to predict the neural pathways from the target model's feature maps. We propose to learn class-relevant information from features of deep and shallow layers such that same-class neural pathways exhibit high similarity. We further impose a faithfulness criterion for GEN-CNP to generate pathways with instance-specific sparsity. We propose to transfer the class-relevant neural pathways to explain samples of the same class and show experimentally and qualitatively their faithfulness and interpretability.


page 3

page 8

page 9

page 16

page 17

page 18

page 19


Compositional Explanations of Neurons

We describe a procedure for explaining neurons in deep representations b...

Towards Robust Interpretability with Self-Explaining Neural Networks

Most recent work on interpretability of complex machine learning models ...

Generating Sparse Counterfactual Explanations For Multivariate Time Series

Since neural networks play an increasingly important role in critical se...

KS-GNNExplainer: Global Model Interpretation Through Instance Explanations On Histopathology images

Instance-level graph neural network explainers have proven beneficial fo...

Improve the Interpretability of Attention: A Fast, Accurate, and Interpretable High-Resolution Attention Model

The prevalence of employing attention mechanisms has brought along conce...

Please sign up or login with your details

Forgot password? Click here to reset