Exploiting Shape Cues for Weakly Supervised Semantic Segmentation

by   Sungpil Kho, et al.

Weakly supervised semantic segmentation (WSSS) aims to produce pixel-wise class predictions with only image-level labels for training. To this end, previous methods adopt the common pipeline: they generate pseudo masks from class activation maps (CAMs) and use such masks to supervise segmentation networks. However, it is challenging to derive comprehensive pseudo masks that cover the whole extent of objects due to the local property of CAMs, i.e., they tend to focus solely on small discriminative object parts. In this paper, we associate the locality of CAMs with the texture-biased property of convolutional neural networks (CNNs). Accordingly, we propose to exploit shape information to supplement the texture-biased CNN features, thereby encouraging mask predictions to be not only comprehensive but also well-aligned with object boundaries. We further refine the predictions in an online fashion with a novel refinement method that takes into account both the class and the color affinities, in order to generate reliable pseudo masks to supervise the model. Importantly, our model is end-to-end trained within a single-stage framework and therefore efficient in terms of the training cost. Through extensive experiments on PASCAL VOC 2012, we validate the effectiveness of our method in producing precise and shape-aligned segmentation results. Specifically, our model surpasses the existing state-of-the-art single-stage approaches by large margins. What is more, it also achieves a new state-of-the-art performance over multi-stage approaches, when adopted in a simple two-stage pipeline without bells and whistles.


page 2

page 7

page 8

page 9

page 10

page 11

page 15


Reliability Does Matter: An End-to-End Weakly Supervised Semantic Segmentation Approach

Weakly supervised semantic segmentation is a challenging task as it only...

Segment Anything is A Good Pseudo-label Generator for Weakly Supervised Semantic Segmentation

Weakly supervised semantic segmentation with weak labels is a long-lived...

Max Pooling with Vision Transformers reconciles class and shape in weakly supervised semantic segmentation

Weakly Supervised Semantic Segmentation (WSSS) research has explored man...

Convolutional Feature Masking for Joint Object and Stuff Segmentation

The topic of semantic segmentation has witnessed considerable progress d...

Gated-SCNN: Gated Shape CNNs for Semantic Segmentation

Current state-of-the-art methods for image segmentation form a dense ima...

Towards Single Stage Weakly Supervised Semantic Segmentation

The costly process of obtaining semantic segmentation labels has driven ...

Weakly Supervised Semantic Segmentation by Pixel-to-Prototype Contrast

Though image-level weakly supervised semantic segmentation (WSSS) has ac...

Please sign up or login with your details

Forgot password? Click here to reset