NIFF: Alleviating Forgetting in Generalized Few-Shot Object Detection via Neural Instance Feature Forging

by   Karim Guirguis, et al.

Privacy and memory are two recurring themes in a broad conversation about the societal impact of AI. These concerns arise from the need for huge amounts of data to train deep neural networks. A promise of Generalized Few-shot Object Detection (G-FSOD), a learning paradigm in AI, is to alleviate the need for collecting abundant training samples of novel classes we wish to detect by leveraging prior knowledge from old classes (i.e., base classes). G-FSOD strives to learn these novel classes while alleviating catastrophic forgetting of the base classes. However, existing approaches assume that the base images are accessible, an assumption that does not hold when sharing and storing data is problematic. In this work, we propose the first data-free knowledge distillation (DFKD) approach for G-FSOD that leverages the statistics of the region of interest (RoI) features from the base model to forge instance-level features without accessing the base images. Our contribution is three-fold: (1) we design a standalone lightweight generator with (2) class-wise heads (3) to generate and replay diverse instance-level base features to the RoI head while finetuning on the novel data. This stands in contrast to standard DFKD approaches in image classification, which invert the entire network to generate base images. Moreover, we make careful design choices in the novel finetuning pipeline to regularize the model. We show that our approach can dramatically reduce the base memory requirements, all while setting a new standard for G-FSOD on the challenging MS-COCO and PASCAL-VOC benchmarks.


page 4

page 15


CFA: Constraint-based Finetuning Approach for Generalized Few-Shot Object Detection

Few-shot object detection (FSOD) seeks to detect novel categories with l...

Towards Generalized and Incremental Few-Shot Object Detection

Real-world object detection is highly desired to be equipped with the le...

Lifelong Object Detection

Recent advances in object detection have benefited significantly from ra...

Bridging Non Co-occurrence with Unlabeled In-the-wild Data for Incremental Object Detection

Deep networks have shown remarkable results in the task of object detect...

DiGeo: Discriminative Geometry-Aware Learning for Generalized Few-Shot Object Detection

Generalized few-shot object detection aims to achieve precise detection ...

Fast Hierarchical Learning for Few-Shot Object Detection

Transfer learning based approaches have recently achieved promising resu...

MINI: Mining Implicit Novel Instances for Few-Shot Object Detection

Learning from a few training samples is a desirable ability of an object...

Please sign up or login with your details

Forgot password? Click here to reset