A likelihood approach to nonparametric estimation of a singular distribution using deep generative models

by   Minwoo Chae, et al.

We investigate statistical properties of a likelihood approach to nonparametric estimation of a singular distribution using deep generative models. More specifically, a deep generative model is used to model high-dimensional data that are assumed to concentrate around some low-dimensional structure. Estimating the distribution supported on this low-dimensional structure such as a low-dimensional manifold is challenging due to its singularity with respect to the Lebesgue measure in the ambient space. In the considered model, a usual likelihood approach can fail to estimate the target distribution consistently due to the singularity. We prove that a novel and effective solution exists by perturbing the data with an instance noise which leads to consistent estimation of the underlying distribution with desirable convergence rates. We also characterize the class of distributions that can be efficiently estimated via deep generative models. This class is sufficiently general to contain various structured distributions such as product distributions, classically smooth distributions and distributions supported on a low-dimensional manifold. Our analysis provides some insights on how deep generative models can avoid the curse of dimensionality for nonparametric distribution estimation. We conduct thorough simulation study and real data analysis to empirically demonstrate that the proposed data perturbation technique improves the estimation performance significantly.


page 20

page 21

page 23


Rates of convergence for nonparametric estimation of singular distributions using generative adversarial networks

We consider generative adversarial networks (GAN) for estimating paramet...

Diagnosing and Fixing Manifold Overfitting in Deep Generative Models

Likelihood-based, or explicit, deep generative models use neural network...

Ratio Matching MMD Nets: Low dimensional projections for effective deep generative models

Deep generative models can learn to generate realistic-looking images on...

Denoising Deep Generative Models

Likelihood-based deep generative models have recently been shown to exhi...

Understanding Deep Generative Models with Generalized Empirical Likelihoods

Understanding how well a deep generative model captures a distribution o...

On the Generative Utility of Cyclic Conditionals

We study whether and how can we model a joint distribution p(x,z) using ...

Manifold Topology Divergence: a Framework for Comparing Data Manifolds

We develop a framework for comparing data manifolds, aimed, in particula...

Please sign up or login with your details

Forgot password? Click here to reset