Wasserstein information matrix
share
We study the information matrix for statistical models by L^2-Wasserstein metric. We call it Wasserstein information matrix (WIM), which is an analog of classical Fisher information matrix. Based on this matrix, we introduce Wasserstein score functions and study covariance operators in statistical models. Using them, we establish Wasserstein-Cramer-Rao bound for estimation. Also, by the ratio of Wasserstein and Fisher information matrices, we prove various functional inequalities within statistical models, including both Log-Sobolev and Poincaré inequalities. These inequalities relate to a new efficiency property named Poincaré efficiency, introduced vis Wasserstein natural gradient for maximal likelihood estimation. Furthermore, online efficiency for Wasserstein natural gradient methods is also established. Several analytical examples and approximations of WIM are presented, including location-scale families, independent families, and Gaussian mixture models.
READ FULL TEXT