Identifiability and optimal rates of convergence for parameters of multiple types in finite mixtures

by   Nhat Ho, et al.

This paper studies identifiability and convergence behaviors for parameters of multiple types in finite mixtures, and the effects of model fitting with extra mixing components. First, we present a general theory for strong identifiability, which extends from the previous work of Nguyen [2013] and Chen [1995] to address a broad range of mixture models and to handle matrix-variate parameters. These models are shown to share the same Wasserstein distance based optimal rates of convergence for the space of mixing distributions --- n^-1/2 under W_1 for the exact-fitted and n^-1/4 under W_2 for the over-fitted setting, where n is the sample size. This theory, however, is not applicable to several important model classes, including location-scale multivariate Gaussian mixtures, shape-scale Gamma mixtures and location-scale-shape skew-normal mixtures. The second part of this work is devoted to demonstrating that for these "weakly identifiable" classes, algebraic structures of the density family play a fundamental role in determining convergence rates of the model parameters, which display a very rich spectrum of behaviors. For instance, the optimal rate of parameter estimation in an over-fitted location-covariance Gaussian mixture is precisely determined by the order of a solvable system of polynomial equations --- these rates deteriorate rapidly as more extra components are added to the model. The established rates for a variety of settings are illustrated by a simulation study.


page 1

page 2

page 3

page 4


Uniform Convergence Rates for Maximum Likelihood Estimation under Two-Component Gaussian Mixture Models

We derive uniform convergence rates for the maximum likelihood estimator...

Optimal Bayesian estimation of Gaussian mixtures with growing number of components

We study posterior concentration properties of Bayesian procedures for e...

Singularity structures and impacts on parameter estimation in finite mixtures of distributions

Singularities of a statistical model are the elements of the model's par...

Convergence of de Finetti's mixing measure in latent structure models for observed exchangeable sequences

Mixtures of product distributions are a powerful device for learning abo...

Minimum Φ-distance estimators for finite mixing measures

Finite mixture models have long been used across a variety of fields in ...

Minimum Wasserstein Distance Estimator under Finite Location-scale Mixtures

When a population exhibits heterogeneity, we often model it via a finite...

Covariance-engaged Classification of Sets via Linear Programming

Set classification aims to classify a set of observations as a whole, as...

Please sign up or login with your details

Forgot password? Click here to reset