Learning Robust Variational Information Bottleneck with Reference

by   Weizhu Qian, et al.

We propose a new approach to train a variational information bottleneck (VIB) that improves its robustness to adversarial perturbations. Unlike the traditional methods where the hard labels are usually used for the classification task, we refine the categorical class information in the training phase with soft labels which are obtained from a pre-trained reference neural network and can reflect the likelihood of the original class labels. We also relax the Gaussian posterior assumption in the VIB implementation by using the mutual information neural estimation. Extensive experiments have been performed with the MNIST and CIFAR-10 datasets, and the results show that our proposed approach significantly outperforms the benchmarked models.


Disentangled Variational Information Bottleneck for Multiview Representation Learning

Multiview data contain information from multiple modalities and have pot...

Towards Consistency and Complementarity: A Multiview Graph Information Bottleneck Approach

The empirical studies of Graph Neural Networks (GNNs) broadly take the o...

Advancing Adversarial Robustness Through Adversarial Logit Update

Deep Neural Networks are susceptible to adversarial perturbations. Adver...

Visualizing Information Bottleneck through Variational Inference

The Information Bottleneck theory provides a theoretical and computation...

Improving Subgraph Recognition with Variational Graph Information Bottleneck

Subgraph recognition aims at discovering a compressed substructure of a ...

Eliciting and Learning with Soft Labels from Every Annotator

The labels used to train machine learning (ML) models are of paramount i...

Sparsity-Inducing Categorical Prior Improves Robustness of the Information Bottleneck

The information bottleneck framework provides a systematic approach to l...

Please sign up or login with your details

Forgot password? Click here to reset