VisualBackProp: efficient visualization of CNNs

by   Mariusz Bojarski, et al.

This paper proposes a new method, that we call VisualBackProp, for visualizing which sets of pixels of the input image contribute most to the predictions made by the convolutional neural network (CNN). The method heavily hinges on exploring the intuition that the feature maps contain less and less irrelevant information to the prediction decision when moving deeper into the network. The technique we propose was developed as a debugging tool for CNN-based systems for steering self-driving cars and is therefore required to run in real-time, i.e. it was designed to require less computations than a forward propagation. This makes the presented visualization method a valuable debugging tool which can be easily used during both training and inference. We furthermore justify our approach with theoretical arguments and theoretically confirm that the proposed method identifies sets of input pixels, rather than individual pixels, that collaboratively contribute to the prediction. Our theoretical findings stand in agreement with the experimental results. The empirical evaluation shows the plausibility of the proposed approach on the road video data as well as in other applications and reveals that it compares favorably to the layer-wise relevance propagation approach, i.e. it obtains similar visualization results and simultaneously achieves order of magnitude speed-ups.


page 7

page 8

page 14

page 15

page 16

page 17

page 18

page 19


Beyond saliency: understanding convolutional neural networks from saliency prediction on layer-wise relevance propagation

Despite the tremendous achievements of deep convolutional neural network...

Explaining Convolutional Neural Networks using Softmax Gradient Layer-wise Relevance Propagation

Convolutional Neural Networks (CNN) have become state-of-the-art in the ...

Irrelevant Pixels are Everywhere: Find and Exclude Them for More Efficient Computer Vision

Computer vision is often performed using Convolutional Neural Networks (...

Visualization of Convolutional Neural Networks for Monocular Depth Estimation

Recently, convolutional neural networks (CNNs) have shown great success ...

Cluster-CAM: Cluster-Weighted Visual Interpretation of CNNs' Decision in Image Classification

Despite the tremendous success of convolutional neural networks (CNNs) i...

Real-time Convolutional Neural Networks for Emotion and Gender Classification

In this paper we propose an implement a general convolutional neural net...

VisualBackProp for learning using privileged information with CNNs

In many machine learning applications, from medical diagnostics to auton...

Please sign up or login with your details

Forgot password? Click here to reset