Realistic Evaluation of Deep Semi-Supervised Learning Algorithms

by   Avital Oliver, et al.

Semi-supervised learning (SSL) provides a powerful framework for leveraging unlabeled data when labels are limited or expensive to obtain. SSL algorithms based on deep neural networks have recently proven successful on standard benchmark tasks. However, we argue that these benchmarks fail to address many issues that these algorithms would face in real-world applications. After creating a unified reimplementation of various widely-used SSL techniques, we test them in a suite of experiments designed to address these issues. We find that the performance of simple baselines which do not use unlabeled data is often underreported, that SSL methods differ in sensitivity to the amount of labeled and unlabeled data, and that performance can degrade substantially when the unlabeled dataset contains out-of-class examples. To help guide SSL research towards real-world applicability, we make our unified reimplemention and evaluation platform publicly available.


page 1

page 2

page 3

page 4


Semi-supervised Learning with Contrastive Predicative Coding

Semi-supervised learning (SSL) provides a powerful framework for leverag...

Robust Deep Semi-Supervised Learning: A Brief Introduction

Semi-supervised learning (SSL) is the branch of machine learning that ai...

USB: A Unified Semi-supervised Learning Benchmark

Semi-supervised learning (SSL) improves model generalization by leveragi...

Robust Semi-Supervised Learning with Out of Distribution Data

Semi-supervised learning (SSL) based on deep neural networks (DNNs) has ...

Semi-supervised Learning for Marked Temporal Point Processes

Temporal Point Processes (TPPs) are often used to represent the sequence...

Mixed Semi-Supervised Generalized-Linear-Regression with applications to Deep learning

We present a methodology for using unlabeled data to design semi supervi...

Beyond Static Datasets: A Deep Interaction Approach to LLM Evaluation

Large Language Models (LLMs) have made progress in various real-world ta...

Code Repositories


Open source release of the evaluation benchmark suite described in "Realistic Evaluation of Deep Semi-Supervised Learning Algorithms"

view repo

Please sign up or login with your details

Forgot password? Click here to reset