Prior choice affects ability of Bayesian neural networks to identify unknowns

by   Daniele Silvestro, et al.

Deep Bayesian neural networks (BNNs) are a powerful tool, though computationally demanding, to perform parameter estimation while jointly estimating uncertainty around predictions. BNNs are typically implemented using arbitrary normal-distributed prior distributions on the model parameters. Here, we explore the effects of different prior distributions on classification tasks in BNNs and evaluate the evidence supporting the predictions based on posterior probabilities approximated by Markov Chain Monte Carlo sampling and by computing Bayes factors. We show that the choice of priors has a substantial impact on the ability of the model to confidently assign data to the correct class (true positive rates). Prior choice also affects significantly the ability of a BNN to identify out-of-distribution instances as unknown (false positive rates). When comparing our results against neural networks (NN) with Monte Carlo dropout we found that BNNs generally outperform NNs. Finally, in our tests we did not find a single best choice as prior distribution. Instead, each dataset yielded the best results under a different prior, indicating that testing alternative options can improve the performance of BNNs.


page 12

page 13


Bayesian Restoration of Digital Images Employing Markov Chain Monte Carlo a Review

A review of Bayesian restoration of digital images based on Monte Carlo ...

On the underestimation of model uncertainty by Bayesian K-nearest neighbors

When using the K-nearest neighbors method, one often ignores uncertainty...

All You Need is a Good Functional Prior for Bayesian Deep Learning

The Bayesian treatment of neural networks dictates that a prior distribu...

Prior Density Learning in Variational Bayesian Phylogenetic Parameters Inference

The advances in variational inference are providing promising paths in B...

Bayesian nonparametric spectral density estimation using B-spline priors

We present a new Bayesian nonparametric approach to estimating the spect...

BNNpriors: A library for Bayesian neural network inference with different prior distributions

Bayesian neural networks have shown great promise in many applications w...

A Symmetric Prior for Multinomial Probit Models

Under standard prior distributions, fitted probabilities from Bayesian m...

Code Repositories


Bayesian neural networks using Numpy and Scipy

view repo

Please sign up or login with your details

Forgot password? Click here to reset