Uncertainty estimation in equality-constrained MAP and maximum likelihood estimation with applications to system identification and state estimation

In unconstrained maximum a posteriori (MAP) and maximum likelihood estimation, the inverse of minus the merit-function Hessian matrix is an approximation of the estimate covariance matrix. In the Bayesian context of MAP estimation, it is the covariance of a normal approximation of the posterior around the mode; while in maximum likelihood estimation, it an approximation of the inverse Fisher information matrix, to which the covariance of efficient estimators converge. These measures are routinely used in system identification to evaluate the estimate uncertainties and diagnose problems such as overparametrization, improper excitation and unidentifiability. A wide variety of estimation problems in systems and control, however, can be formulated as equality-constrained optimizations with additional decision variables to exploit parallelism in computer hardware, simplify implementation and increase the convergence basin and efficiency of the nonlinear program solver. The introduction of the extra variables, however, dissociates the inverse Hessian from the covariance matrix. Instead, submatrices of the inverse Hessian of the constrained-problem's Lagrangian must be used. In this paper, we derive these relationships, showing how the estimates' covariance can be estimated directly from the augmented problem. Application examples are shown in system identification with the output-error method and joint state-path and parameter estimation.


page 1

page 2

page 3

page 4


Asymptotic analysis of maximum likelihood estimation of covariance parameters for Gaussian processes: an introduction with proofs

This article provides an introduction to the asymptotic analysis of cova...

Solving Non-parametric Inverse Problem in Continuous Markov Random Field using Loopy Belief Propagation

In this paper, we address the inverse problem, or the statistical machin...

Maximum Likelihood-based Gridless DoA Estimation Using Structured Covariance Matrix Recovery and SBL with Grid Refinement

We consider the parametric data model employed in applications such as l...

Separable correlation and maximum likelihood

We consider estimation of the covariance matrix of a multivariate normal...

Bethe Learning of Conditional Random Fields via MAP Decoding

Many machine learning tasks can be formulated in terms of predicting str...

Certifiably Optimal Sparse Inverse Covariance Estimation

We consider the maximum likelihood estimation of sparse inverse covarian...

Please sign up or login with your details

Forgot password? Click here to reset