Efficient Search of Comprehensively Robust Neural Architectures via Multi-fidelity Evaluation

by   Jialiang Sun, et al.

Neural architecture search (NAS) has emerged as one successful technique to find robust deep neural network (DNN) architectures. However, most existing robustness evaluations in NAS only consider l_∞ norm-based adversarial noises. In order to improve the robustness of DNN models against multiple types of noises, it is necessary to consider a comprehensive evaluation in NAS for robust architectures. But with the increasing number of types of robustness evaluations, it also becomes more time-consuming to find comprehensively robust architectures. To alleviate this problem, we propose a novel efficient search of comprehensively robust neural architectures via multi-fidelity evaluation (ES-CRNA-ME). Specifically, we first search for comprehensively robust architectures under multiple types of evaluations using the weight-sharing-based NAS method, including different l_p norm attacks, semantic adversarial attacks, and composite adversarial attacks. In addition, we reduce the number of robustness evaluations by the correlation analysis, which can incorporate similar evaluations and decrease the evaluation cost. Finally, we propose a multi-fidelity online surrogate during optimization to further decrease the search cost. On the basis of the surrogate constructed by low-fidelity data, the online high-fidelity data is utilized to finetune the surrogate. Experiments on CIFAR10 and CIFAR100 datasets show the effectiveness of our proposed method.


page 2

page 4

page 5

page 6

page 10


Searching for Robust Neural Architectures via Comprehensive and Reliable Evaluation

Neural architecture search (NAS) could help search for robust network ar...

Bi-fidelity Evolutionary Multiobjective Search for Adversarially Robust Deep Neural Architectures

Deep neural networks have been found vulnerable to adversarial attacks, ...

DSRNA: Differentiable Search of Robust Neural Architectures

In deep learning applications, the architectures of deep neural networks...

Adversarially Robust Neural Architectures

Deep Neural Network (DNN) are vulnerable to adversarial attack. Existing...

Neural Architecture Design and Robustness: A Dataset

Deep learning models have proven to be successful in a wide range of mac...

Heat Source Layout Optimization Using Automatic Deep Learning Surrogate and Multimodal Neighborhood Search Algorithm

In satellite layout design, heat source layout optimization (HSLO) is an...

Please sign up or login with your details

Forgot password? Click here to reset