Continual Few-shot Relation Learning via Embedding Space Regularization and Data Augmentation

by   Chengwei Qin, et al.

Existing continual relation learning (CRL) methods rely on plenty of labeled training data for learning a new task, which can be hard to acquire in real scenario as getting large and representative labeled data is often expensive and time-consuming. It is therefore necessary for the model to learn novel relational patterns with very few labeled data while avoiding catastrophic forgetting of previous task knowledge. In this paper, we formulate this challenging yet practical problem as continual few-shot relation learning (CFRL). Based on the finding that learning for new emerging few-shot tasks often results in feature distributions that are incompatible with previous tasks' learned distributions, we propose a novel method based on embedding space regularization and data augmentation. Our method generalizes to new few-shot tasks and avoids catastrophic forgetting of previous tasks by enforcing extra constraints on the relational embeddings and by adding extra relevant data in a self-supervised manner. With extensive experiments we demonstrate that our method can significantly outperform previous state-of-the-art methods in CFRL task settings.


page 1

page 2

page 3

page 4


LFPT5: A Unified Framework for Lifelong Few-shot Language Learning Based on Prompt Tuning of T5

Existing approaches to lifelong language learning rely on plenty of labe...

Disentangle-based Continual Graph Representation Learning

Graph embedding (GE) methods embed nodes (and/or edges) in graph into a ...

Lifelong Learning of Few-shot Learners across NLP Tasks

Recent advances in large pre-trained language models have greatly improv...

Continual Local Replacement for Few-shot Image Recognition

The goal of few-shot learning is to learn a model that can recognize nov...

Lifelong Learning for Neural powered Mixed Integer Programming

Mixed Integer programs (MIPs) are typically solved by the Branch-and-Bou...

A Relational Model for One-Shot Classification

We show that a deep learning model with built-in relational inductive bi...

Centroids Matching: an efficient Continual Learning approach operating in the embedding space

Catastrophic forgetting (CF) occurs when a neural network loses the info...

Please sign up or login with your details

Forgot password? Click here to reset