Towards Structured Evaluation of Deep Neural Network Supervisors

by   Jens Henriksson, et al.

Deep Neural Networks (DNN) have improved the quality of several non-safety related products in the past years. However, before DNNs should be deployed to safety-critical applications, their robustness needs to be systematically analyzed. A common challenge for DNNs occurs when input is dissimilar to the training set, which might lead to high confidence predictions despite proper knowledge of the input. Several previous studies have proposed to complement DNNs with a supervisor that detects when inputs are outside the scope of the network. Most of these supervisors, however, are developed and tested for a selected scenario using a specific performance metric. In this work, we emphasize the need to assess and compare the performance of supervisors in a structured way. We present a framework constituted by four datasets organized in six test cases combined with seven evaluation metrics. The test cases provide varying complexity and include data from publicly available sources as well as a novel dataset consisting of images from simulated driving scenarios. The latter we plan to make publicly available. Our framework can be used to support DNN supervisor evaluation, which in turn could be used to motive development, validation, and deployment of DNNs in safety-critical applications.


page 3

page 5

page 6


Performance Analysis of Out-of-Distribution Detection on Trained Neural Networks

Several areas have been improved with Deep Learning during the past year...

Safely Entering the Deep: A Review of Verification and Validation for Machine Learning and a Challenge Elicitation in the Automotive Industry

Deep Neural Networks (DNN) will emerge as a cornerstone in automotive so...

Performance Analysis of Out-of-Distribution Detection on Various Trained Neural Networks

Several areas have been improved with Deep Learning during the past year...

DeepGalaxy: Testing Neural Network Verifiers via Two-Dimensional Input Space Exploration

Deep neural networks (DNNs) are widely developed and applied in many are...

PAC Confidence Predictions for Deep Neural Network Classifiers

A key challenge for deploying deep neural networks (DNNs) in safety crit...

NESTA, The NICTA Energy System Test Case Archive

In recent years the power systems research community has seen an explosi...

Assessing Systematic Weaknesses of DNNs using Counterfactuals

With the advancement of DNNs into safety-critical applications, testing ...

Please sign up or login with your details

Forgot password? Click here to reset