Rethinking the Number of Shots in Robust Model-Agnostic Meta-Learning

by   Xiaoyue Duan, et al.

Robust Model-Agnostic Meta-Learning (MAML) is usually adopted to train a meta-model which may fast adapt to novel classes with only a few exemplars and meanwhile remain robust to adversarial attacks. The conventional solution for robust MAML is to introduce robustness-promoting regularization during meta-training stage. With such a regularization, previous robust MAML methods simply follow the typical MAML practice that the number of training shots should match with the number of test shots to achieve an optimal adaptation performance. However, although the robustness can be largely improved, previous methods sacrifice clean accuracy a lot. In this paper, we observe that introducing robustness-promoting regularization into MAML reduces the intrinsic dimension of clean sample features, which results in a lower capacity of clean representations. This may explain why the clean accuracy of previous robust MAML methods drops severely. Based on this observation, we propose a simple strategy, i.e., increasing the number of training shots, to mitigate the loss of intrinsic dimension caused by robustness-promoting regularization. Though simple, our method remarkably improves the clean accuracy of MAML without much loss of robustness, producing a robust yet accurate model. Extensive experiments demonstrate that our method outperforms prior arts in achieving a better trade-off between accuracy and robustness. Besides, we observe that our method is less sensitive to the number of fine-tuning steps during meta-training, which allows for a reduced number of fine-tuning steps to improve training efficiency.


page 1

page 2

page 3

page 4


On Fast Adversarial Robustness Adaptation in Model-Agnostic Meta-Learning

Model-agnostic meta-learning (MAML) has emerged as one of the most succe...

Adversarial Meta-Learning

Meta-learning enables a model to learn from very limited data to underta...

Long-term Cross Adversarial Training: A Robust Meta-learning Method for Few-shot Classification Tasks

Meta-learning model can quickly adapt to new tasks using few-shot labele...

MetaVA: Curriculum Meta-learning and Pre-fine-tuning of Deep Neural Networks for Detecting Ventricular Arrhythmias based on ECGs

Ventricular arrhythmias (VA) are the main causes of sudden cardiac death...

Revisiting the Trade-off between Accuracy and Robustness via Weight Distribution of Filters

Adversarial attacks have been proven to be potential threats to Deep Neu...

Repurposing Pretrained Models for Robust Out-of-domain Few-Shot Learning

Model-agnostic meta-learning (MAML) is a popular method for few-shot lea...

How to Inject Backdoors with Better Consistency: Logit Anchoring on Clean Data

Since training a large-scale backdoored model from scratch requires a la...

Please sign up or login with your details

Forgot password? Click here to reset