Few-Shot Learning with Class Imbalance

by   Mateusz Ochal, et al.

Few-shot learning aims to train models on a limited number of labeled samples given in a support set in order to generalize to unseen samples from a query set. In the standard setup, the support set contains an equal amount of data points for each class. However, this assumption overlooks many practical considerations arising from the dynamic nature of the real world, such as class-imbalance. In this paper, we present a detailed study of few-shot class-imbalance along three axes: meta-dataset vs. task imbalance, effect of different imbalance distributions (linear, step, random), and effect of rebalancing techniques. We extensively compare over 10 state-of-the-art few-shot learning and meta-learning methods using unbalanced tasks and meta-datasets. Our analysis using Mini-ImageNet reveals that 1) compared to the balanced task, the performances on class-imbalance tasks counterparts always drop, by up to 18.0% for optimization-based methods, and up to 8.4 for metric-based methods, 2) contrary to popular belief, meta-learning algorithms, such as MAML, do not automatically learn to balance by being exposed to imbalanced tasks during (meta-)training time, 3) strategies used to mitigate imbalance in supervised learning, such as oversampling, can offer a stronger solution to the class imbalance problem, 4) the effect of imbalance at the meta-dataset level is less significant than the effect at the task level with similar imbalance magnitude. The code to reproduce the experiments is released under an open-source license.


page 17

page 28

page 29


How Sensitive are Meta-Learners to Dataset Imbalance?

Meta-Learning (ML) has proven to be a useful tool for training Few-Shot ...

Addressing the Real-world Class Imbalance Problem in Dermatology

Class imbalance is a common problem in medical diagnosis, causing a stan...

Is Support Set Diversity Necessary for Meta-Learning?

Meta-learning is a popular framework for learning with limited data in w...

The Effect of Diversity in Meta-Learning

Few-shot learning aims to learn representations that can tackle novel ta...

Few-Shot Learning for Road Object Detection

Few-shot learning is a problem of high interest in the evolution of deep...

Adaptive manifold for imbalanced transductive few-shot learning

Transductive few-shot learning algorithms have showed substantially supe...

Meta-Album: Multi-domain Meta-Dataset for Few-Shot Image Classification

We introduce Meta-Album, an image classification meta-dataset designed t...

Please sign up or login with your details

Forgot password? Click here to reset