Meta-Transfer Learning through Hard Tasks

by   Qianru Sun, et al.

Meta-learning has been proposed as a framework to address the challenging few-shot learning setting. The key idea is to leverage a large number of similar few-shot tasks in order to learn how to adapt a base-learner to a new task for which only a few labeled samples are available. As deep neural networks (DNNs) tend to overfit using a few samples only, typical meta-learning models use shallow neural networks, thus limiting its effectiveness. In order to achieve top performance, some recent works tried to use the DNNs pre-trained on large-scale datasets but mostly in straight-forward manners, e.g., (1) taking their weights as a warm start of meta-training, and (2) freezing their convolutional layers as the feature extractor of base-learners. In this paper, we propose a novel approach called meta-transfer learning (MTL) which learns to transfer the weights of a deep NN for few-shot learning tasks. Specifically, meta refers to training multiple tasks, and transfer is achieved by learning scaling and shifting functions of DNN weights for each task. In addition, we introduce the hard task (HT) meta-batch scheme as an effective learning curriculum that further boosts the learning efficiency of MTL. We conduct few-shot learning experiments and report top performance for five-class few-shot recognition tasks on three challenging benchmarks: miniImageNet, tieredImageNet and Fewshot-CIFAR100 (FC100). Extensive comparisons to related works validate that our MTL approach trained with the proposed HT meta-batch scheme achieves top performance. An ablation study also shows that both components contribute to fast convergence and high accuracy.


page 3

page 4

page 6

page 8

page 9

page 10

page 11

page 12


Meta-Transfer Learning for Few-Shot Learning

Meta-learning has been proposed as a framework to address the challengin...

LCC: Learning to Customize and Combine Neural Networks for Few-Shot Learning

Meta-learning has been shown to be an effective strategy for few-shot le...

Evaluating Deep Neural Network Ensembles by Majority Voting cum Meta-Learning scheme

Deep Neural Networks (DNNs) are prone to overfitting and hence have high...

A Theoretical Analysis of the Number of Shots in Few-Shot Learning

Few-shot classification is the task of predicting the category of an exa...

Task Attended Meta-Learning for Few-Shot Learning

Meta-learning (ML) has emerged as a promising direction in learning mode...

FS-HGR: Few-shot Learning for Hand Gesture Recognition via ElectroMyography

This work is motivated by the recent advances in Deep Neural Networks (D...

MetaModulation: Learning Variational Feature Hierarchies for Few-Shot Learning with Fewer Tasks

Meta-learning algorithms are able to learn a new task using previously l...

Please sign up or login with your details

Forgot password? Click here to reset