Practical Detection of Trojan Neural Networks: Data-Limited and Data-Free Cases

by   Ren Wang, et al.

When the training data are maliciously tampered, the predictions of the acquired deep neural network (DNN) can be manipulated by an adversary known as the Trojan attack (or poisoning backdoor attack). The lack of robustness of DNNs against Trojan attacks could significantly harm real-life machine learning (ML) systems in downstream applications, therefore posing widespread concern to their trustworthiness. In this paper, we study the problem of the Trojan network (TrojanNet) detection in the data-scarce regime, where only the weights of a trained DNN are accessed by the detector. We first propose a data-limited TrojanNet detector (TND), when only a few data samples are available for TrojanNet detection. We show that an effective data-limited TND can be established by exploring connections between Trojan attack and prediction-evasion adversarial attacks including per-sample attack as well as all-sample universal attack. In addition, we propose a data-free TND, which can detect a TrojanNet without accessing any data samples. We show that such a TND can be built by leveraging the internal response of hidden neurons, which exhibits the Trojan behavior even at random noise inputs. The effectiveness of our proposals is evaluated by extensive experiments under different model architectures and datasets including CIFAR-10, GTSRB, and ImageNet.


page 6

page 14

page 20

page 21

page 25

page 26

page 27

page 28


Towards Adversarial-Resilient Deep Neural Networks for False Data Injection Attack Detection in Power Grids

False data injection attack (FDIA) is a critical security issue in power...

Quarantine: Sparsity Can Uncover the Trojan Attack Trigger for Free

Trojan attacks threaten deep neural networks (DNNs) by poisoning them to...

FreeEagle: Detecting Complex Neural Trojans in Data-Free Cases

Trojan attack on deep neural networks, also known as backdoor attack, is...

Beyond Mahalanobis-Based Scores for Textual OOD Detection

Deep learning methods have boosted the adoption of NLP systems in real-l...

An Explainable Adversarial Robustness Metric for Deep Learning Neural Networks

Deep Neural Networks(DNN) have excessively advanced the field of compute...

Distribution Shift Detection for Deep Neural Networks

To deploy and operate deep neural models in production, the quality of t...

Generative Poisoning Attack Method Against Neural Networks

Poisoning attack is identified as a severe security threat to machine le...

Please sign up or login with your details

Forgot password? Click here to reset