Synthetic vs Real: Deep Learning on Controlled Noise

by   Lu Jiang, et al.

Performing controlled experiments on noisy data is essential in thoroughly understanding deep learning across a spectrum of noise levels. Due to the lack of suitable datasets, previous research have only examined deep learning on controlled synthetic noise, and real-world noise has never been systematically studied in a controlled setting. To this end, this paper establishes a benchmark of real-world noisy labels at 10 controlled noise levels. As real-world noise possesses unique properties, to understand the difference, we conduct a large-scale study across a variety of noise levels and types, architectures, methods, and training settings. Our study shows that: (1) Deep Neural Networks (DNNs) generalize much better on real-world noise. (2) DNNs may not learn patterns first on real-world noisy data. (3) When networks are fine-tuned, ImageNet architectures generalize well on noisy data. (4) Real-world noise appears to be less harmful, yet it is more difficult for robust DNN methods to improve. (5) Robust learning methods that work well on synthetic noise may not work as well on real-world noise, and vice versa. We hope our benchmark, as well as our findings, will facilitate deep learning research on noisy data.


page 5

page 10

page 16


Understanding and Utilizing Deep Neural Networks Trained with Noisy Labels

Noisy labels are ubiquitous in real-world datasets, which poses a challe...

Towards Harnessing Feature Embedding for Robust Learning with Noisy Labels

The memorization effect of deep neural networks (DNNs) plays a pivotal r...

The Learning of Fuzzy Cognitive Maps With Noisy Data: A Rapid and Robust Learning Method With Maximum Entropy

Numerous learning methods for fuzzy cognitive maps (FCMs), such as the H...

Generation and Analysis of Feature-Dependent Pseudo Noise for Training Deep Neural Networks

Training Deep neural networks (DNNs) on noisy labeled datasets is a chal...

Emphasis Regularisation by Gradient Rescaling for Training Deep Neural Networks with Noisy Labels

It is fundamental and challenging to train robust and accurate Deep Neur...

Confidence-based Reliable Learning under Dual Noises

Deep neural networks (DNNs) have achieved remarkable success in a variet...

Comparative Study on the Effects of Noise in ML-Based Anxiety Detection

Wearable health devices are ushering in a new age of continuous and noni...

Please sign up or login with your details

Forgot password? Click here to reset