Efficient Multi-Grained Knowledge Reuse for Class Incremental Segmentation

by   Zhihe Lu, et al.

Class Incremental Semantic Segmentation (CISS) has been a trend recently due to its great significance in real-world applications. Although the existing CISS methods demonstrate remarkable performance, they either leverage the high-level knowledge (feature) only while neglecting the rich and diverse knowledge in the low-level features, leading to poor old knowledge preservation and weak new knowledge exploration; or use multi-level features for knowledge distillation by retraining a heavy backbone, which is computationally intensive. In this paper, we for the first time propose to efficiently reuse the multi-grained knowledge for CISS by fusing multi-level features with the frozen backbone and show a simple aggregation of varying-level features, i.e., naive feature pyramid, can boost the performance significantly. We further introduce a novel densely-interactive feature pyramid (DEFY) module that enhances the fusion of high- and low-level features by enabling their dense interaction. Specifically, DEFY establishes a per-pixel relationship between pairs of feature maps, allowing for multi-pair outputs to be aggregated. This results in improved semantic segmentation by leveraging the complementary information from multi-level features. We show that DEFY can be effortlessly integrated into three representative methods for performance enhancement. Our method yields a new state-of-the-art performance when combined with the current SOTA by notably averaged mIoU gains on two widely used benchmarks, i.e., 2.5 on PASCAL VOC 2012 and 2.3


page 1

page 2

page 4

page 8

page 10

page 13


ExFuse: Enhancing Feature Fusion for Semantic Segmentation

Modern semantic segmentation frameworks usually combine low-level and hi...

Feature Fusion Use Unsupervised Prior Knowledge to Let Small Object Represent

Fusing low level and high level features is a widely used strategy to pr...

Polyp-PVT: Polyp Segmentation with Pyramid Vision Transformers

Most polyp segmentation methods use CNNs as their backbone, leading to t...

Improved Few-shot Segmentation by Redefinition of the Roles of Multi-level CNN Features

This study is concerned with few-shot segmentation, i.e., segmenting the...

MARNet: Multi-Abstraction Refinement Network for 3D Point Cloud Analysis

Representation learning from 3D point clouds is challenging due to their...

Joint Speech Activity and Overlap Detection with Multi-Exit Architecture

Overlapped speech detection (OSD) is critical for speech applications in...

DSIC: Dynamic Sample-Individualized Connector for Multi-Scale Object Detection

Although object detection has reached a milestone thanks to the great su...

Please sign up or login with your details

Forgot password? Click here to reset