MARS: A second-order reduction algorithm for high-dimensional sparse precision matrices estimation

by   Qian Li, et al.

Estimation of the precision matrix (or inverse covariance matrix) is of great importance in statistical data analysis. However, as the number of parameters scales quadratically with the dimension p, computation becomes very challenging when p is large. In this paper, we propose an adaptive sieving reduction algorithm to generate a solution path for the estimation of precision matrices under the ℓ_1 penalized D-trace loss, with each subproblem being solved by a second-order algorithm. In each iteration of our algorithm, we are able to greatly reduce the number of variables in the problem based on the Karush-Kuhn-Tucker (KKT) conditions and the sparse structure of the estimated precision matrix in the previous iteration. As a result, our algorithm is capable of handling datasets with very high dimensions that may go beyond the capacity of the existing methods. Moreover, for the sub-problem in each iteration, other than solving the primal problem directly, we develop a semismooth Newton augmented Lagrangian algorithm with global linear convergence on the dual problem to improve the efficiency. Theoretical properties of our proposed algorithm have been established. In particular, we show that the convergence rate of our algorithm is asymptotically superlinear. The high efficiency and promising performance of our algorithm are illustrated via extensive simulation studies and real data applications, with comparison to several state-of-the-art solvers.


page 1

page 2

page 3

page 4


An efficient ADMM algorithm for high dimensional precision matrix estimation via penalized quadratic loss

The estimation of high dimensional precision matrices has been a central...

Fast and Adaptive Sparse Precision Matrix Estimation in High Dimensions

This paper proposes a new method for estimating sparse precision matrice...

Adaptive Regularization for Weight Matrices

Algorithms for learning distributions over weight-vectors, such as AROW ...

CARE: Large Precision Matrix Estimation for Compositional Data

High-dimensional compositional data are prevalent in many applications. ...

A greedy algorithm for sparse precision matrix approximation

Precision matrix estimation is an important problem in statistical data ...

Sparse Distance Weighted Discrimination

Distance weighted discrimination (DWD) was originally proposed to handle...

G-AMA: Sparse Gaussian graphical model estimation via alternating minimization

Several methods have been recently proposed for estimating sparse Gaussi...

Please sign up or login with your details

Forgot password? Click here to reset