Hierarchical Mixtures-of-Experts for Exponential Family Regression Models with Generalized Linear Mean Functions: A Survey of Approximation and Consistency Results

by   Wenxin Jiang, et al.

We investigate a class of hierarchical mixtures-of-experts (HME) models where exponential family regression models with generalized linear mean functions of the form psi(ga+fx^Tfgb) are mixed. Here psi(...) is the inverse link function. Suppose the true response y follows an exponential family regression model with mean function belonging to a class of smooth functions of the form psi(h(fx)) where h(...)in W_2^infty (a Sobolev class over [0,1]^s). It is shown that the HME probability density functions can approximate the true density, at a rate of O(m^-2/s) in L_p norm, and at a rate of O(m^-4/s) in Kullback-Leibler divergence. These rates can be achieved within the family of HME structures with no more than s-layers, where s is the dimension of the predictor fx. It is also shown that likelihood-based inference based on HME is consistent in recovering the truth, in the sense that as the sample size n and the number of experts m both increase, the mean square error of the predicted mean response goes to zero. Conditions for such results to hold are stated and discussed.


page 1

page 2

page 3

page 7


Convergence Rates for Mixture-of-Experts

In mixtures-of-experts (ME) model, where a number of submodels (experts)...

Online Algorithms For Parameter Mean And Variance Estimation In Dynamic Regression Models

We study the problem of estimating the parameters of a regression model ...

The Central Tendency, Weighted Likelihood, and Exponential family

In this paper, we establish the links between the Hölder and Lehmer cent...

Some Results on Tighter Bayesian Lower Bounds on the Mean-Square Error

In random parameter estimation, Bayesian lower bounds (BLBs) for the mea...

Best Linear Predictor with Missing Response: Locally Robust Approach

This paper provides asymptotic theory for Inverse Probability Weighing (...

Infinitely imbalanced binomial regression and deformed exponential families

The logistic regression model is known to converge to a Poisson point pr...

Please sign up or login with your details

Forgot password? Click here to reset