Towards Robust Neural Networks via Random Self-ensemble

by   Xuanqing Liu, et al.

Recent studies have revealed the vulnerability of deep neural networks - A small adversarial perturbation that is imperceptible to human can easily make a well-trained deep neural network mis-classify. This makes it unsafe to apply neural networks in security-critical applications. In this paper, we propose a new defensive algorithm called Random Self-Ensemble (RSE) by combining two important concepts: randomness and ensemble. To protect a targeted model, RSE adds random noise layers to the neural network to prevent from state-of-the-art gradient-based attacks, and ensembles the prediction over random noises to stabilize the performance. We show that our algorithm is equivalent to ensemble an infinite number of noisy models f_ϵ without any additional memory overhead, and the proposed training procedure based on noisy stochastic gradient descent can ensure the ensemble model has good predictive capability. Our algorithm significantly outperforms previous defense techniques on real datasets. For instance, on CIFAR-10 with VGG network (which has 92% accuracy without any attack), under the state-of-the-art C&W attack within a certain distortion tolerance, the accuracy of unprotected model drops to less than 10%, the best previous defense technique has 48% accuracy, while our method still has 86% prediction accuracy under the same level of attack. Finally, our method is simple and easy to integrate into any neural network.


page 1

page 2

page 3

page 4


Dynamic Defense Approach for Adversarial Robustness in Deep Neural Networks via Stochastic Ensemble Smoothed Model

Deep neural networks have been shown to suffer from critical vulnerabili...

Robust Training with Ensemble Consensus

Since deep neural networks are over-parametrized, they may memorize nois...

Adv-BNN: Improved Adversarial Defense through Robust Bayesian Neural Network

We present a new algorithm to train a robust neural network against adve...

Omni: Automated Ensemble with Unexpected Models against Adversarial Evasion Attack

BACKGROUND: Machine learning-based security detection models have become...

EnResNet: ResNet Ensemble via the Feynman-Kac Formalism

We propose a simple yet powerful ResNet ensemble algorithm which consist...

WaNet – Imperceptible Warping-based Backdoor Attack

With the thriving of deep learning and the widespread practice of using ...

The BeMi Stardust: a Structured Ensemble of Binarized Neural Networks

Binarized Neural Networks (BNNs) are receiving increasing attention due ...

Please sign up or login with your details

Forgot password? Click here to reset