Generalizable Lightweight Proxy for Robust NAS against Diverse Perturbations

by   Hyeonjeong Ha, et al.

Recent neural architecture search (NAS) frameworks have been successful in finding optimal architectures for given conditions (e.g., performance or latency). However, they search for optimal architectures in terms of their performance on clean images only, while robustness against various types of perturbations or corruptions is crucial in practice. Although there exist several robust NAS frameworks that tackle this issue by integrating adversarial training into one-shot NAS, however, they are limited in that they only consider robustness against adversarial attacks and require significant computational resources to discover optimal architectures for a single task, which makes them impractical in real-world scenarios. To address these challenges, we propose a novel lightweight robust zero-cost proxy that considers the consistency across features, parameters, and gradients of both clean and perturbed images at the initialization state. Our approach facilitates an efficient and rapid search for neural architectures capable of learning generalizable features that exhibit robustness across diverse perturbations. The experimental results demonstrate that our proxy can rapidly and efficiently search for neural architectures that are consistently robust against various perturbations on multiple benchmark datasets and diverse search spaces, largely outperforming existing clean zero-shot NAS and robust NAS with reduced search cost.


page 1

page 2

page 3

page 4


ZiCo: Zero-shot NAS via Inverse Coefficient of Variation on Gradients

Neural Architecture Search (NAS) is widely used to automatically design ...

An Evaluation of Zero-Cost Proxies – from Neural Architecture Performance to Model Robustness

Zero-cost proxies are nowadays frequently studied and used to search for...

Multi-shot NAS for Discovering Adversarially Robust Convolutional Neural Architectures at Targeted Capacities

Convolutional neural networks (CNNs) are vulnerable to adversarial examp...

Neural Architecture Design and Robustness: A Dataset

Deep learning models have proven to be successful in a wide range of mac...

Learning Diverse-Structured Networks for Adversarial Robustness

In adversarial training (AT), the main focus has been the objective and ...

Robust Weight Signatures: Gaining Robustness as Easy as Patching Weights?

Given a robust model trained to be resilient to one or multiple types of...

EZNAS: Evolving Zero Cost Proxies For Neural Architecture Scoring

Neural Architecture Search (NAS) has significantly improved productivity...

Please sign up or login with your details

Forgot password? Click here to reset