A Unified Framework for Sparse Non-Negative Least Squares using Multiplicative Updates and the Non-Negative Matrix Factorization Problem

by   Igor Fedorov, et al.

We study the sparse non-negative least squares (S-NNLS) problem. S-NNLS occurs naturally in a wide variety of applications where an unknown, non-negative quantity must be recovered from linear measurements. We present a unified framework for S-NNLS based on a rectified power exponential scale mixture prior on the sparse codes. We show that the proposed framework encompasses a large class of S-NNLS algorithms and provide a computationally efficient inference procedure based on multiplicative update rules. Such update rules are convenient for solving large sets of S-NNLS problems simultaneously, which is required in contexts like sparse non-negative matrix factorization (S-NMF). We provide theoretical justification for the proposed approach by showing that the local minima of the objective function being optimized are sparse and the S-NNLS algorithms presented are guaranteed to converge to a set of stationary points of the objective function. We then extend our framework to S-NMF, showing that our framework leads to many well known S-NMF algorithms under specific choices of prior and providing a guarantee that a popular subclass of the proposed algorithms converges to a set of stationary points of the objective function. Finally, we study the performance of the proposed approaches on synthetic and real-world data.


Convergence to Second-Order Stationarity for Non-negative Matrix Factorization: Provably and Concurrently

Non-negative matrix factorization (NMF) is a fundamental non-convex opti...

A Block Coordinate Descent-based Projected Gradient Algorithm for Orthogonal Non-negative Matrix Factorization

This article utilizes the projected gradient method (PG) for a non-negat...

A Nonlinear Orthogonal Non-Negative Matrix Factorization Approach to Subspace Clustering

A recent theoretical analysis shows the equivalence between non-negative...

Minimum description length as an objective function for non-negative matrix factorization

Non-negative matrix factorization (NMF) is a dimensionality reduction te...

Relative Pairwise Relationship Constrained Non-negative Matrix Factorisation

Non-negative Matrix Factorisation (NMF) has been extensively used in mac...

An Improved Bayesian Framework for Quadrature of Constrained Integrands

Quadrature is the problem of estimating intractable integrals, a problem...

A Unified Framework for Stochastic Matrix Factorization via Variance Reduction

We propose a unified framework to speed up the existing stochastic matri...

Please sign up or login with your details

Forgot password? Click here to reset