On the Use of Minimum Penalties in Statistical Learning

by   Ben Sherwood, et al.

Modern multivariate machine learning and statistical methodologies estimate parameters of interest while leveraging prior knowledge of the association between outcome variables. The methods that do allow for estimation of relationships do so typically through an error covariance matrix in multivariate regression which does not scale to other types of models. In this article we proposed the MinPEN framework to simultaneously estimate regression coefficients associated with the multivariate regression model and the relationships between outcome variables using mild assumptions. The MinPen framework utilizes a novel penalty based on the minimum function to exploit detected relationships between responses. An iterative algorithm that generalizes current state of the art methods is proposed as a solution to the non-convex optimization that is required to obtain estimates. Theoretical results such as high dimensional convergence rates, model selection consistency, and a framework for post selection inference are provided. We extend the proposed MinPen framework to other exponential family loss functions, with a specific focus on multiple binomial responses. Tuning parameter selection is also addressed. Finally, simulations and two data examples are presented to show the finite sample properties of this framework.


page 1

page 2

page 3

page 4


Joint estimation of sparse multivariate regression and conditional graphical models

Multivariate regression model is a natural generalization of the classic...

A Cluster Fusion Penalty for Grouping Response Variables in Multivariate Regression Models

We propose a method for estimating coefficients in multivariate regressi...

Rotation to Sparse Loadings using L^p Losses and Related Inference Problems

Exploratory factor analysis (EFA) has been widely used to learn the late...

Sparse multivariate regression with missing values and its application to the prediction of material properties

In the field of materials science and engineering, statistical analysis ...

Sparse Multivariate ARCH Models: Finite Sample Properties

We provide finite sample properties of sparse multivariate ARCH processe...

Sparse Multivariate Factor Regression

We consider the problem of multivariate regression in a setting where th...

Relationship-aware Multivariate Sampling Strategy for Scientific Simulation Data

With the increasing computational power of current supercomputers, the s...

Please sign up or login with your details

Forgot password? Click here to reset