DeepAI AI Chat
Log In Sign Up

Learning Common Rationale to Improve Self-Supervised Representation for Fine-Grained Visual Recognition Problems

by   Yangyang Shu, et al.
The University of Adelaide

Self-supervised learning (SSL) strategies have demonstrated remarkable performance in various recognition tasks. However, both our preliminary investigation and recent studies suggest that they may be less effective in learning representations for fine-grained visual recognition (FGVR) since many features helpful for optimizing SSL objectives are not suitable for characterizing the subtle differences in FGVR. To overcome this issue, we propose learning an additional screening mechanism to identify discriminative clues commonly seen across instances and classes, dubbed as common rationales in this paper. Intuitively, common rationales tend to correspond to the discriminative patterns from the key parts of foreground objects. We show that a common rationale detector can be learned by simply exploiting the GradCAM induced from the SSL objective without using any pre-trained object parts or saliency detectors, making it seamlessly to be integrated with the existing SSL process. Specifically, we fit the GradCAM with a branch with limited fitting capacity, which allows the branch to capture the common rationales and discard the less common discriminative patterns. At the test stage, the branch generates a set of spatial weights to selectively aggregate features representing an instance. Extensive experimental results on four visual tasks demonstrate that the proposed method can lead to a significant improvement in different evaluation settings.


page 1

page 3

page 11


Distilling Localization for Self-Supervised Representation Learning

For high-level visual recognition, self-supervised learning defines and ...

Fine-Grained Object Classification via Self-Supervised Pose Alignment

Semantic patterns of fine-grained objects are determined by subtle appea...

Self-Supervised Learning for Fine-Grained Visual Categorization

Recent research in self-supervised learning (SSL) has shown its capabili...

Improving Fine-Grained Visual Recognition in Low Data Regimes via Self-Boosting Attention Mechanism

The challenge of fine-grained visual recognition often lies in discoveri...

Explored An Effective Methodology for Fine-Grained Snake Recognition

Fine-Grained Visual Classification (FGVC) is a longstanding and fundamen...

How Self-Supervised Learning Can be Used for Fine-Grained Head Pose Estimation?

Recent progress of Self-Supervised Learning (SSL) demonstrates the capab...

Spatial-Scale Aligned Network for Fine-Grained Recognition

Existing approaches for fine-grained visual recognition focus on learnin...