Rethink and Redesign Meta learning

by   Yunxiao Qin, et al.

Recently, Meta-learning has been shown as a promising way to improve the ability of learning from few data for Computer Vision. However, previous Meta-learning approaches exposed below problems: 1) they ignored the importance of attention mechanism for Meta-learner, leading the Meta-learner to be interfered by unimportant information; 2) they ignored the importance of past knowledge which can help the Meta-learner accurately understand the input data and further express them into high representations, and they train the Meta-learner to solve few shot learning task directly on the few original input data instead of on the high representations; 3) they suffer from a problem which we named as task-over-fitting (TOF) problem, which is probably caused by that they are requested to solve few shot learning task based on the original high dimensional input data, and redundant input information leads themselves to be easier to suffer from TOF. In this paper, we rethink the Meta-learning algorithm and propose that the attention mechanism and the past knowledge are crucial for the Meta-learner, and the Meta-learner should well use its past knowledge and express the input data into high representations to solve few shot learning tasks. Moreover, the Meta-learning approach should be free from the TOF problem. Based on these arguments, we redesign the Meta-learning algorithm to solve these three aforementioned problems, and proposed three methods. Extensive experiments demonstrate the effectiveness of our designation and methods with state-of-the-art performances on several few shot learning benchmarks. The source code of our proposed methods will be released soon.


page 2

page 6

page 9

page 13


Representation based and Attention augmented Meta learning

Deep learning based computer vision fails to work when labeled images ar...

Meta-Meta-Classification for One-Shot Learning

We present a new approach, called meta-meta-classification, to learning ...

Exploring the Similarity of Representations in Model-Agnostic Meta-Learning

In past years model-agnostic meta-learning (MAML) has been one of the mo...

Bootstrapped Meta-Learning

Meta-learning empowers artificial intelligence to increase its efficienc...

Layer-Wise Adaptive Updating for Few-Shot Image Classification

Few-shot image classification (FSIC), which requires a model to recogniz...

Few-shot Learning for Decoding Brain Signals

Few-shot learning consists in addressing data-thrifty (inductive few-sho...

Attentional meta-learners are polythetic classifiers

Polythetic classifications, based on shared patterns of features that ne...

Please sign up or login with your details

Forgot password? Click here to reset