Incremental Few-Shot Semantic Segmentation via Embedding Adaptive-Update and Hyper-class Representation

by   Guangchen Shi, et al.

Incremental few-shot semantic segmentation (IFSS) targets at incrementally expanding model's capacity to segment new class of images supervised by only a few samples. However, features learned on old classes could significantly drift, causing catastrophic forgetting. Moreover, few samples for pixel-level segmentation on new classes lead to notorious overfitting issues in each learning session. In this paper, we explicitly represent class-based knowledge for semantic segmentation as a category embedding and a hyper-class embedding, where the former describes exclusive semantical properties, and the latter expresses hyper-class knowledge as class-shared semantic properties. Aiming to solve IFSS problems, we present EHNet, i.e., Embedding adaptive-update and Hyper-class representation Network from two aspects. First, we propose an embedding adaptive-update strategy to avoid feature drift, which maintains old knowledge by hyper-class representation, and adaptively update category embeddings with a class-attention scheme to involve new classes learned in individual sessions. Second, to resist overfitting issues caused by few training samples, a hyper-class embedding is learned by clustering all category embeddings for initialization and aligned with category embedding of the new class for enhancement, where learned knowledge assists to learn new knowledge, thus alleviating performance dependence on training data scale. Significantly, these two designs provide representation capability for classes with sufficient semantics and limited biases, enabling to perform segmentation tasks requiring high semantic dependence. Experiments on PASCAL-5i and COCO datasets show that EHNet achieves new state-of-the-art performance with remarkable advantages.


page 4

page 8


Two-level Graph Network for Few-Shot Class-Incremental Learning

Few-shot class-incremental learning (FSCIL) aims to design machine learn...

Adaptive Masked Weight Imprinting for Few-Shot Segmentation

Deep learning has mainly thrived by training on large-scale datasets. Ho...

A Few Guidelines for Incremental Few-Shot Segmentation

Reducing the amount of supervision required by neural networks is especi...

Few-Shot Class-Incremental Learning via Feature Space Composition

As a challenging problem in machine learning, few-shot class-incremental...

Hyper-Class Representation of Data

Data representation is often of the natural form with their attribute va...

Prototypical Kernel Learning and Open-set Foreground Perception for Generalized Few-shot Semantic Segmentation

Generalized Few-shot Semantic Segmentation (GFSS) extends Few-shot Seman...

Learning Adaptive Embedding Considering Incremental Class

Class-Incremental Learning (CIL) aims to train a reliable model with the...

Please sign up or login with your details

Forgot password? Click here to reset