Efficient sparse Hessian based algorithms for the clustered lasso problem

by   Meixia Lin, et al.

We focus on solving the clustered lasso problem, which is a least squares problem with the ℓ_1-type penalties imposed on both the coefficients and their pairwise differences to learn the group structure of the regression parameters. Here we first reformulate the clustered lasso regularizer as a weighted ordered-lasso regularizer, which is essential in reducing the computational cost from O(n^2) to O(n (n)). We then propose an inexact semismooth Newton augmented Lagrangian (SSNAL) algorithm to solve the clustered lasso problem or its dual via this equivalent formulation, depending on whether the sample size is larger than the dimension of the features. An essential component of the SSNAL algorithm is the computation of the generalized Jacobian of the proximal mapping of the clustered lasso regularizer. Based on the new formulation, we derive an efficient procedure for its computation. Comprehensive results on the global convergence and local linear convergence of the SSNAL algorithm are established. For the purpose of exposition and comparison, we also summarize/design several first-order methods that can be used to solve the problem under consideration, but with the key improvement from the new formulation of the clustered lasso regularizer. As a demonstration of the applicability of our algorithms, numerical experiments on the clustered lasso problem are performed. The experiments show that the SSNAL algorithm substantially outperforms the best alternative algorithm for the clustered lasso problem.


page 1

page 2

page 3

page 4


Efficient Path Algorithms for Clustered Lasso and OSCAR

In high dimensional regression, feature clustering by their effects on o...

Semismooth Newton Augmented Lagrangian Algorithm for Adaptive Lasso Penalized Least Squares in Semiparametric Regression

This paper is concerned with a partially linear semiparametric regressio...

An Efficient Augmented Lagrangian Based Method for Constrained Lasso

Variable selection is one of the most important tasks in statistics and ...

The clustered Sparrow algorithm

In this paper, we study an extension of Schoning's algorithm [Schoning, ...

Algorithms for Fitting the Constrained Lasso

We compare alternative computing strategies for solving the constrained ...

A Scalable and Extensible Framework for Superposition-Structured Models

In many learning tasks, structural models usually lead to better interpr...

On the Duality between Network Flows and Network Lasso

The data arising in many application domains have an intrinsic network s...

Please sign up or login with your details

Forgot password? Click here to reset