Teamwork Is Not Always Good: An Empirical Study of Classifier Drift in Class-incremental Information Extraction

by   Minqian Liu, et al.

Class-incremental learning (CIL) aims to develop a learning system that can continually learn new classes from a data stream without forgetting previously learned classes. When learning classes incrementally, the classifier must be constantly updated to incorporate new classes, and the drift in decision boundary may lead to severe forgetting. This fundamental challenge, however, has not yet been studied extensively, especially in the setting where no samples from old classes are stored for rehearsal. In this paper, we take a closer look at how the drift in the classifier leads to forgetting, and accordingly, design four simple yet (super-) effective solutions to alleviate the classifier drift: an Individual Classifiers with Frozen Feature Extractor (ICE) framework where we individually train a classifier for each learning session, and its three variants ICE-PL, ICE-O, and ICE-PL O which further take the logits of previously learned classes from old sessions or a constant logit of an Other class as a constraint to the learning of new classifiers. Extensive experiments and analysis on 6 class-incremental information extraction tasks demonstrate that our solutions, especially ICE-O, consistently show significant improvement over the previous state-of-the-art approaches with up to 44.7 absolute F-score gain, providing a strong baseline and insights for future research on class-incremental learning.


page 1

page 2

page 3

page 4


Few-Shot Incremental Learning with Continually Evolved Classifiers

Few-shot class-incremental learning (FSCIL) aims to design machine learn...

Class-incremental Learning with Pre-allocated Fixed Classifiers

In class-incremental learning, a learning agent faces a stream of data w...

Cross-Class Feature Augmentation for Class Incremental Learning

We propose a novel class incremental learning approach by incorporating ...

Balanced Supervised Contrastive Learning for Few-Shot Class-Incremental Learning

Few-shot class-incremental learning (FSCIL) presents the primary challen...

eTag: Class-Incremental Learning with Embedding Distillation and Task-Oriented Generation

Class-Incremental Learning (CIL) aims to solve the neural networks' cata...

Improving Vision Transformers for Incremental Learning

This paper studies using Vision Transformers (ViT) in class incremental ...

GMM-IL: Image Classification using Incrementally Learnt, Independent Probabilistic Models for Small Sample Sizes

Current deep learning classifiers, carry out supervised learning and sto...

Please sign up or login with your details

Forgot password? Click here to reset