UNICON: Combating Label Noise Through Uniform Selection and Contrastive Learning

by   Nazmul Karim, et al.

Supervised deep learning methods require a large repository of annotated data; hence, label noise is inevitable. Training with such noisy data negatively impacts the generalization performance of deep neural networks. To combat label noise, recent state-of-the-art methods employ some sort of sample selection mechanism to select a possibly clean subset of data. Next, an off-the-shelf semi-supervised learning method is used for training where rejected samples are treated as unlabeled data. Our comprehensive analysis shows that current selection methods disproportionately select samples from easy (fast learnable) classes while reject those from relatively harder ones. This creates class imbalance in the selected clean set and in turn deteriorates performance under high label noise. In this work, we propose UNICON, a simple yet effective sample selection method which is robust to high label noise. To address the disproportionate selection of easy and hard samples, we introduce a Jensen-Shannon divergence based uniform selection mechanism which does not require any probabilistic modelling and hyperparameter tuning. We complement our selection method with contrastive learning to further combat memorization of noisy labels. Extensive experimentation on multiple benchmark datasets demonstrate the effectiveness of UNICON; we obtain 11.4 current state-of-the-art on CIFAR100 dataset with 90


page 1

page 2

page 3

page 4


Manifold DivideMix: A Semi-Supervised Contrastive Learning Framework for Severe Label Noise

Deep neural networks have proven to be highly effective when large amoun...

ProMix: Combating Label Noise via Maximizing Clean Sample Utility

The ability to train deep neural networks under label noise is appealing...

Jo-SRC: A Contrastive Approach for Combating Noisy Labels

Due to the memorization effect in Deep Neural Networks (DNNs), training ...

Adaptive Sample Selection for Robust Learning under Label Noise

Deep Neural Networks (DNNs) have been shown to be susceptible to memoriz...

PASS: Peer-Agreement based Sample Selection for training with Noisy Labels

Noisy labels present a significant challenge in deep learning because mo...

S3: Supervised Self-supervised Learning under Label Noise

Despite the large progress in supervised learning with Neural Networks, ...

No Regret Sample Selection with Noisy Labels

Deep Neural Network (DNN) suffers from noisy labeled data because of the...

Please sign up or login with your details

Forgot password? Click here to reset