Coverage-Guided Fuzzing for Deep Neural Networks

by   Xiaofei Xie, et al.

In company with the data explosion over the past decade, deep neural network (DNN) based software has experienced unprecedented leap and is becoming the key driving force of many novel industrial applications, including many safety-critical scenarios such as autonomous driving. Despite great success achieved in various human intelligence tasks, similar to traditional software, DNNs could also exhibit incorrect behaviors caused by hidden defects causing severe accidents and losses. In this paper, we propose an automated fuzz testing framework for hunting potential defects of general-purpose DNNs. It performs metamorphic mutation to generate new semantically preserved tests, and leverages multiple plugable coverage criteria as feedback to guide the test generation from different perspectives. To be scalable towards practical-sized DNNs, our framework maintains tests in batch, and prioritizes the tests selection based on active feedback. The effectiveness of our framework is extensively investigated on 3 popular datasets (MNIST, CIFAR-10, ImageNet) and 7 DNNs with diverse complexities, under large set of 6 coverage criteria as feedback. The large-scale experiments demonstrate that our fuzzing framework can (1) significantly boost the coverage with guidance; (2) generate useful tests to detect erroneous behaviors and facilitate the DNN model quality evaluation; (3) accurately capture potential defects during DNN quantization for platform migration.


page 3

page 6


DeepHunter: Hunting Deep Neural Network Defects via Coverage-Guided Fuzzing

In company with the data explosion over the past decade, deep neural net...

DeepGini: Prioritizing Massive Tests to Reduce Labeling Cost

Deep neural network (DNN) based systems have been deployed to assist var...

Black-Box Testing of Deep Neural Networks through Test Case Diversity

Deep Neural Networks (DNNs) have been extensively used in many areas inc...

Enhancing Deep Neural Networks Testing by Traversing Data Manifold

We develop DEEPTRAVERSAL, a feedback-driven framework to test DNNs. DEEP...

Coverage Testing of Deep Learning Models using Dataset Characterization

Deep Neural Networks (DNNs), with its promising performance, are being i...

Is Neuron Coverage Needed to Make Person Detection More Robust?

The growing use of deep neural networks (DNNs) in safety- and security-c...

Using Neural Networks for Novelty-based Test Selection to Accelerate Functional Coverage Closure

Machine learning (ML) has been used to accelerate the closure of functio...

Please sign up or login with your details

Forgot password? Click here to reset