Spatial-Aware Token for Weakly Supervised Object Localization

by   Pingyu Wu, et al.

Weakly supervised object localization (WSOL) is a challenging task aiming to localize objects with only image-level supervision. Recent works apply visual transformer to WSOL and achieve significant success by exploiting the long-range feature dependency in self-attention mechanism. However, existing transformer-based methods synthesize the classification feature maps as the localization map, which leads to optimization conflicts between classification and localization tasks. To address this problem, we propose to learn a task-specific spatial-aware token (SAT) to condition localization in a weakly supervised manner. Specifically, a spatial token is first introduced in the input space to aggregate representations for localization task. Then a spatial aware attention module is constructed, which allows spatial token to generate foreground probabilities of different patches by querying and to extract localization knowledge from the classification task. Besides, for the problem of sparse and unbalanced pixel-level supervision obtained from the image-level label, two spatial constraints, including batch area loss and normalization loss, are designed to compensate and enhance this supervision. Experiments show that the proposed SAT achieves state-of-the-art performance on both CUB-200 and ImageNet, with 98.45 extreme setting of using only 1 image per class from ImageNet for training, SAT already exceeds the SOTA method by 2.1 available at


page 4

page 8

page 12

page 14

page 16

page 17

page 18

page 19


Re-Attention Transformer for Weakly Supervised Object Localization

Weakly supervised object localization is a challenging task which aims t...

ViTOL: Vision Transformer for Weakly Supervised Object Localization

Weakly supervised object localization (WSOL) aims at predicting object l...

ASOC: Adaptive Self-aware Object Co-localization

The primary goal of this paper is to localize objects in a group of sema...

Weakly Supervised Object Localization via Transformer with Implicit Spatial Calibration

Weakly Supervised Object Localization (WSOL), which aims to localize obj...

Cerberus Transformer: Joint Semantic, Affordance and Attribute Parsing

Multi-task indoor scene understanding is widely considered as an intrigu...

Weakly Supervised Object Localization as Domain Adaption

Weakly supervised object localization (WSOL) focuses on localizing objec...

TDAN: Top-Down Attention Networks for Enhanced Feature Selectivity in CNNs

Attention modules for Convolutional Neural Networks (CNNs) are an effect...

Please sign up or login with your details

Forgot password? Click here to reset