Black-box error diagnosis in deep neural networks: a survey of tools

by   Piero Fraternali, et al.

The application of Deep Neural Networks (DNNs) to a broad variety of tasks demands methods for coping with the complex and opaque nature of these architectures. The analysis of performance can be pursued in two ways. On one side, model interpretation techniques aim at "opening the box" to assess the relationship between the input, the inner layers, and the output. For example, saliency and attention models exploit knowledge of the architecture to capture the essential regions of the input that have the most impact on the inference process and output. On the other hand, models can be analysed as "black boxes", e.g., by associating the input samples with extra annotations that do not contribute to model training but can be exploited for characterizing the model response. Such performance-driven meta-annotations enable the detailed characterization of performance metrics and errors and help scientists identify the features of the input responsible for prediction failures and focus their model improvement efforts. This paper presents a structured survey of the tools that support the "black box" analysis of DNNs and discusses the gaps in the current proposals and the relevant future directions in this research field.


page 3

page 12


Analysis of Explainers of Black Box Deep Neural Networks for Computer Vision: A Survey

Deep Learning is a state-of-the-art technique to make inference on exten...

Towards Fully Interpretable Deep Neural Networks: Are We There Yet?

Despite the remarkable performance, Deep Neural Networks (DNNs) behave a...

Towards Lightweight Black-Box Attacks against Deep Neural Networks

Black-box attacks can generate adversarial examples without accessing th...

Interpretable Factorization for Neural Network ECG Models

The ability of deep learning (DL) to improve the practice of medicine an...

DeepOpt: Scalable Specification-based Falsification of Neural Networks using Black-Box Optimization

Decisions made by deep neural networks (DNNs) have a tremendous impact o...

Exclusion and Inclusion – A model agnostic approach to feature importance in DNNs

Deep Neural Networks in NLP have enabled systems to learn complex non-li...

Differential Analysis of Triggers and Benign Features for Black-Box DNN Backdoor Detection

This paper proposes a data-efficient detection method for deep neural ne...

Please sign up or login with your details

Forgot password? Click here to reset