A Semantics-Guided Class Imbalance Learning Model for Zero-Shot Classification

by   Zhong Ji, et al.

Zero-Shot Classification (ZSC) equips the learned model with the ability to recognize the visual instances from the novel classes via constructing the interactions between the visual and the semantic modalities. In contrast to the traditional image classification, ZSC is easily suffered from the class-imbalance issue since it is more concerned with the class-level knowledge transfer capability. In the real world, the class samples follow a long-tailed distribution, and the discriminative information in the sample-scarce seen classes is hard to be transferred to the related unseen classes in the traditional batch-based training manner, which degrades the overall generalization ability a lot. Towards alleviating the class imbalance issue in ZSC, we propose a sample-balanced training process to encourage all training classes to contribute equally to the learned model. Specifically, we randomly select the same number of images from each class across all training classes to form a training batch to ensure that the sample-scarce classes contribute equally as those classes with sufficient samples during each iteration. Considering that the instances from the same class differ in class representativeness, we further develop an efficient semantics-guided feature fusion model to obtain discriminative class visual prototype for the following visual-semantic interaction process via distributing different weights to the selected samples based on their class representativeness. Extensive experiments on three imbalanced ZSC benchmark datasets for both the Traditional ZSC (TZSC) and the Generalized ZSC (GZSC) tasks demonstrate our approach achieves promising results especially for the unseen categories those are closely related to the sample-scarce seen categories.


page 1

page 8


Multi-modal Ensemble Classification for Generalized Zero Shot Learning

Generalized zero shot learning (GZSL) is defined by a training process c...

Meta-Transfer Networks for Zero-Shot Learning

Zero-Shot Learning (ZSL) aims at recognizing unseen categories using som...

Learning Class Prototypes via Structure Alignment for Zero-Shot Recognition

Zero-shot learning (ZSL) aims to recognize objects of novel classes with...

Learning Semantic Ambiguities for Zero-Shot Learning

Zero-shot learning (ZSL) aims at recognizing classes for which no visual...

Counterfactual Zero-Shot and Open-Set Visual Recognition

We present a novel counterfactual framework for both Zero-Shot Learning ...

TextManiA: Enriching Visual Feature by Text-driven Manifold Augmentation

Recent label mix-based augmentation methods have shown their effectivene...

Please sign up or login with your details

Forgot password? Click here to reset