Variational Inference with NoFAS: Normalizing Flow with Adaptive Surrogate for Computationally Expensive Models

by   Yu Wang, et al.

Fast inference of numerical model parameters from data is an important prerequisite to generate predictive models for a wide range of applications. Use of sampling-based approaches such as Markov chain Monte Carlo may become intractable when each likelihood evaluation is computationally expensive. New approaches combining variational inference with normalizing flow are characterized by a computational cost that grows only linearly with the dimensionality of the latent variable space, and rely on gradient-based optimization instead of sampling, providing a more efficient approach for Bayesian inference about the model parameters. Moreover, the cost of frequently evaluating an expensive likelihood can be mitigated by replacing the true model with an offline trained surrogate model, such as neural networks. However, this approach might generate significant bias when the surrogate is insufficiently accurate around the posterior modes. To reduce the computational cost without sacrificing inferential accuracy, we propose Normalizing Flow with Adaptive Surrogate (NoFAS), an optimization strategy that alternatively updates the normalizing flow parameters and the weights of a neural network surrogate model. We also propose an efficient sample weighting scheme for surrogate model training that ensures some global accuracy of the surrogate while capturing the likely regions of the parameters that yield the observed data. We demonstrate the inferential and computational superiority of NoFAS against various benchmarks, including cases where the underlying model lacks identifiability. The source code and numerical experiments used for this study are available at


page 8

page 10

page 11

page 12

page 19


Surrogate-assisted parallel tempering for Bayesian neural learning

Parallel tempering addresses some of the drawbacks of canonical Markov C...

Constraining subglacial processes from surface velocity observations using surrogate-based Bayesian inference

Basal motion is the primary mechanism for ice flux outside Antarctica, y...

Relative gradient optimization of the Jacobian term in unsupervised deep learning

Learning expressive probabilistic models correctly describing the data i...

Accelerating sequential Monte Carlo with surrogate likelihoods

Delayed-acceptance is a technique for reducing computational effort for ...

VaiPhy: a Variational Inference Based Algorithm for Phylogeny

Phylogenetics is a classical methodology in computational biology that t...

VDL-Surrogate: A View-Dependent Latent-based Model for Parameter Space Exploration of Ensemble Simulations

We propose VDL-Surrogate, a view-dependent neural-network-latent-based s...

Surrogate Model for Geological CO2 Storage and Its Use in MCMC-based History Matching

Deep-learning-based surrogate models show great promise for use in geolo...

Please sign up or login with your details

Forgot password? Click here to reset