Information Symmetry Matters: A Modal-Alternating Propagation Network for Few-Shot Learning

by   Zhong Ji, et al.

Semantic information provides intra-class consistency and inter-class discriminability beyond visual concepts, which has been employed in Few-Shot Learning (FSL) to achieve further gains. However, semantic information is only available for labeled samples but absent for unlabeled samples, in which the embeddings are rectified unilaterally by guiding the few labeled samples with semantics. Therefore, it is inevitable to bring a cross-modal bias between semantic-guided samples and nonsemantic-guided samples, which results in an information asymmetry problem. To address this problem, we propose a Modal-Alternating Propagation Network (MAP-Net) to supplement the absent semantic information of unlabeled samples, which builds information symmetry among all samples in both visual and semantic modalities. Specifically, the MAP-Net transfers the neighbor information by the graph propagation to generate the pseudo-semantics for unlabeled samples guided by the completed visual relationships and rectify the feature embeddings. In addition, due to the large discrepancy between visual and semantic modalities, we design a Relation Guidance (RG) strategy to guide the visual relation vectors via semantics so that the propagated information is more beneficial. Extensive experimental results on three semantic-labeled datasets, i.e., Caltech-UCSD-Birds 200-2011, SUN Attribute Database, and Oxford 102 Flower, have demonstrated that our proposed method achieves promising performance and outperforms the state-of-the-art approaches, which indicates the necessity of information symmetry.


page 1

page 10


Global Semantic Consistency for Zero-Shot Learning

In image recognition, there are many cases where training samples cannot...

DUET: Cross-modal Semantic Grounding for Contrastive Zero-shot Learning

Zero-shot learning (ZSL) aims to predict unseen classes whose samples ha...

Rich Semantics Improve Few-shot Learning

Human learning benefits from multi-modal inputs that often appear as ric...

Progressive Cluster Purification for Transductive Few-shot Learning

Few-shot learning aims to learn to generalize a classifier to novel clas...

Advancing Incremental Few-shot Semantic Segmentation via Semantic-guided Relation Alignment and Adaptation

Incremental few-shot semantic segmentation (IFSS) aims to incrementally ...

Adversarial Cross-Modal Retrieval via Learning and Transferring Single-Modal Similarities

Cross-modal retrieval aims to retrieve relevant data across different mo...

SemMAE: Semantic-Guided Masking for Learning Masked Autoencoders

Recently, significant progress has been made in masked image modeling to...

Please sign up or login with your details

Forgot password? Click here to reset