Robust Meta-learning with Sampling Noise and Label Noise via Eigen-Reptile

06/04/2022
by   Dong Chen, et al.
10

Recent years have seen a surge of interest in meta-learning techniques for tackling the few-shot learning (FSL) problem. However, the meta-learner is prone to overfitting since there are only a few available samples, which can be identified as sampling noise on a clean dataset. Moreover, when handling the data with noisy labels, the meta-learner could be extremely sensitive to label noise on a corrupted dataset. To address these two challenges, we present Eigen-Reptile (ER) that updates the meta-parameters with the main direction of historical task-specific parameters to alleviate sampling and label noise. Specifically, the main direction is computed in a fast way, where the scale of the calculated matrix is related to the number of gradient steps instead of the number of parameters. Furthermore, to obtain a more accurate main direction for Eigen-Reptile in the presence of many noisy labels, we further propose Introspective Self-paced Learning (ISPL). We have theoretically and experimentally demonstrated the soundness and effectiveness of the proposed Eigen-Reptile and ISPL. Particularly, our experiments on different tasks show that the proposed method is able to outperform or achieve highly competitive performance compared with other gradient-based methods with or without noisy labels. The code and data for the proposed method are provided for research purposes https://github.com/Anfeather/Eigen-Reptile.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/22/2023

Enhanced Meta Label Correction for Coping with Label Corruption

Traditional methods for learning with the presence of noisy labels have ...
research
07/11/2020

Meta Soft Label Generation for Noisy Labels

The existence of noisy labels in the dataset causes significant performa...
research
08/03/2020

Learning to Purify Noisy Labels via Meta Soft Label Corrector

Recent deep neural networks (DNNs) can easily overfit to biased training...
research
09/12/2023

BatMan-CLR: Making Few-shots Meta-Learners Resilient Against Label Noise

The negative impact of label noise is well studied in classical supervis...
research
01/18/2023

Improve Noise Tolerance of Robust Loss via Noise-Awareness

Robust loss minimization is an important strategy for handling robust le...
research
02/20/2019

Tug the Student to Learn Right: Progressive Gradient Correcting by Meta-learner on Corrupted Labels

While deep networks have strong fitting capability to complex input patt...
research
02/20/2019

Push the Student to Learn Right: Progressive Gradient Correcting by Meta-learner on Corrupted Labels

While deep networks have strong fitting capability to complex input patt...

Please sign up or login with your details

Forgot password? Click here to reset