New Insights on Relieving Task-Recency Bias for Online Class Incremental Learning

by   Guoqiang Liang, et al.

To imitate the ability of keeping learning of human, continual learning which can learn from a never-ending data stream has attracted more interests recently. In all settings, the online class incremental learning (CIL), where incoming samples from data stream can be used only once, is more challenging and can be encountered more frequently in real world. Actually, the CIL faces a stability-plasticity dilemma, where the stability means the ability to preserve old knowledge while the plasticity denotes the ability to incorporate new knowledge. Although replay-based methods have shown exceptional promise, most of them concentrate on the strategy for updating and retrieving memory to keep stability at the expense of plasticity. To strike a preferable trade-off between stability and plasticity, we propose a Adaptive Focus Shifting algorithm (AFS), which dynamically adjusts focus to ambiguous samples and non-target logits in model learning. Through a deep analysis of the task-recency bias caused by class imbalance, we propose a revised focal loss to mainly keep stability. By utilizing a new weight function, the revised focal loss can pay more attention to current ambiguous samples, which can provide more information of the classification boundary. To promote plasticity, we introduce a virtual knowledge distillation. By designing a virtual teacher, it assigns more attention to non-target classes, which can surmount overconfidence and encourage model to focus on inter-class information. Extensive experiments on three popular datasets for CIL have shown the effectiveness of AFS. The code will be available at <>.


page 1

page 3


Online Prototype Learning for Online Continual Learning

Online continual learning (CL) studies the problem of learning continuou...

Summarizing Stream Data for Memory-Restricted Online Continual Learning

Replay-based methods have proved their effectiveness on online continual...

Bypassing Logits Bias in Online Class-Incremental Learning with a Generative Framework

Continual learning requires the model to maintain the learned knowledge ...

A Comprehensive Empirical Evaluation on Online Continual Learning

Online continual learning aims to get closer to a live learning experien...

Class Incremental Online Streaming Learning

A wide variety of methods have been developed to enable lifelong learnin...

Online Hyperparameter Optimization for Class-Incremental Learning

Class-incremental learning (CIL) aims to train a classification model wh...

Online Class Incremental Learning on Stochastic Blurry Task Boundary via Mask and Visual Prompt Tuning

Continual learning aims to learn a model from a continuous stream of dat...

Please sign up or login with your details

Forgot password? Click here to reset