Self-supervision of Feature Transformation for Further Improving Supervised Learning

06/09/2021
by   Zilin Ding, et al.
0

Self-supervised learning, which benefits from automatically constructing labels through pre-designed pretext task, has recently been applied for strengthen supervised learning. Since previous self-supervised pretext tasks are based on input, they may incur huge additional training overhead. In this paper we find that features in CNNs can be also used for self-supervision. Thus we creatively design the feature-based pretext task which requires only a small amount of additional training overhead. In our task we discard different particular regions of features, and then train the model to distinguish these different features. In order to fully apply our feature-based pretext task in supervised learning, we also propose a novel learning framework containing multi-classifiers for further improvement. Original labels will be expanded to joint labels via self-supervision of feature transformations. With more semantic information provided by our self-supervised tasks, this approach can train CNNs more effectively. Extensive experiments on various supervised learning tasks demonstrate the accuracy improvement and wide applicability of our method.

READ FULL TEXT

page 2

page 13

page 14

research
06/09/2021

Self-supervised Feature Enhancement: Applying Internal Pretext Task to Supervised Learning

Traditional self-supervised learning requires CNNs using external pretex...
research
07/20/2022

Tailoring Self-Supervision for Supervised Learning

Recently, it is shown that deploying a proper self-supervision is a pros...
research
07/18/2023

CSSL-RHA: Contrastive Self-Supervised Learning for Robust Handwriting Authentication

Handwriting authentication is a valuable tool used in various fields, su...
research
08/27/2021

Subjective Learning for Open-Ended Data

Conventional machine learning methods typically assume that data is spli...
research
02/23/2023

Improving Adaptive Conformal Prediction Using Self-Supervised Learning

Conformal prediction is a powerful distribution-free tool for uncertaint...
research
09/02/2021

Self-supervised Recommendation with Cross-channel Matching Representation and Hierarchical Contrastive Learning

Recently, using different channels to model social semantic information,...
research
09/01/2021

EventPoint: Self-Supervised Local Descriptor Learning for Event Cameras

We proposes a method of extracting intrest points and descriptors using ...

Please sign up or login with your details

Forgot password? Click here to reset