Breaking Inter-Layer Co-Adaptation by Classifier Anonymization

06/04/2019

∙

This study addresses an issue of co-adaptation between a feature extractor and a classifier in a neural network. A naive joint optimization of a feature extractor and a classifier often brings situations in which an excessively complex feature distribution adapted to a very specific classifier degrades the test performance. We introduce a method called Feature-extractor Optimization through Classifier Anonymization (FOCA), which is designed to avoid an explicit co-adaptation between a feature extractor and a particular classifier by using many randomly-generated, weak classifiers during optimization. We put forth a mathematical proposition that states the FOCA features form a point-like distribution within the same class in a class-separable fashion under special conditions. Real-data experiments under more general conditions provide supportive evidences.

READ FULL TEXT

Breaking Inter-Layer Co-Adaptation by Classifier Anonymization

Re-energizing Domain Discriminator with Sample Relabeling for Adversarial Domain Adaptation

Recognition Of Surface Defects On Steel Sheet Using Transfer Learning

Fantastic DNN Classifiers and How to Identify them without Data

To Balance or Not to Balance: An Embarrassingly Simple Approach for Learning with Long-Tailed Distributions

POP-CNN: Predicting Odor's Pleasantness with Convolutional Neural Network

Is Discriminator a Good Feature Extractor?

Transferable Cross-Tokamak Disruption Prediction with Deep Hybrid Neural Network Feature Extractor

Breaking Inter-Layer Co-Adaptation by Classifier Anonymization

Related Research

Re-energizing Domain Discriminator with Sample Relabeling for Adversarial Domain Adaptation

Recognition Of Surface Defects On Steel Sheet Using Transfer Learning

Fantastic DNN Classifiers and How to Identify them without Data

To Balance or Not to Balance: An Embarrassingly Simple Approach for Learning with Long-Tailed Distributions

POP-CNN: Predicting Odor's Pleasantness with Convolutional Neural Network

Is Discriminator a Good Feature Extractor?

Transferable Cross-Tokamak Disruption Prediction with Deep Hybrid Neural Network Feature Extractor