Unsupervised Part Discovery from Contrastive Reconstruction

by   Subhabrata Choudhury, et al.

The goal of self-supervised visual representation learning is to learn strong, transferable image representations, with the majority of research focusing on object or scene level. On the other hand, representation learning at part level has received significantly less attention. In this paper, we propose an unsupervised approach to object part discovery and segmentation and make three contributions. First, we construct a proxy task through a set of objectives that encourages the model to learn a meaningful decomposition of the image into its parts. Secondly, prior work argues for reconstructing or clustering pre-computed features as a proxy to parts; we show empirically that this alone is unlikely to find meaningful parts; mainly because of their low resolution and the tendency of classification networks to spatially smear out information. We suggest that image reconstruction at the level of pixels can alleviate this problem, acting as a complementary cue. Lastly, we show that the standard evaluation based on keypoint regression does not correlate well with segmentation quality and thus introduce different metrics, NMI and ARI, that better characterize the decomposition of objects into parts. Our method yields semantic parts which are consistent across fine-grained but visually distinct categories, outperforming the state of the art on three benchmark datasets. Code is available at the project page: https://www.robots.ox.ac.uk/ vgg/research/unsup-parts/.


page 4

page 9

page 10


Self-Supervised Learning of Object Parts for Semantic Segmentation

Progress in self-supervised learning has brought strong general image re...

Unsupervised Object Localization: Observing the Background to Discover Objects

Recent advances in self-supervised visual representation learning have p...

Towards Fine-grained Visual Representations by Combining Contrastive Learning with Image Reconstruction and Attention-weighted Pooling

This paper presents Contrastive Reconstruction, ConRec - a self-supervis...

Self-Supervised Learning for Fine-Grained Visual Categorization

Recent research in self-supervised learning (SSL) has shown its capabili...

Interpretable and Accurate Fine-grained Recognition via Region Grouping

We present an interpretable deep model for fine-grained visual recogniti...

Unsupervised Volumetric Animation

We propose a novel approach for unsupervised 3D animation of non-rigid d...

ProtoPShare: Prototype Sharing for Interpretable Image Classification and Similarity Discovery

In this paper, we introduce ProtoPShare, a self-explained method that in...

Please sign up or login with your details

Forgot password? Click here to reset