Improved SVD-based Initialization for Nonnegative Matrix Factorization using Low-Rank Correction

by   Atif Muhammad Syed, et al.

Due to the iterative nature of most nonnegative matrix factorization (NMF) algorithms, initialization is a key aspect as it significantly influences both the convergence and the final solution obtained. Many initialization schemes have been proposed for NMF, among which one of the most popular class of methods are based on the singular value decomposition (SVD). However, these SVD-based initializations do not satisfy a rather natural condition, namely that the error should decrease as the rank of factorization increases. In this paper, we propose a novel SVD-based NMF initialization to specifically address this shortcoming by taking into account the SVD factors that were discarded to obtain a nonnegative initialization. This method, referred to as nonnegative SVD with low-rank correction (NNSVD-LRC), allows us to significantly reduce the initial error at a negligible additional computational cost using the low-rank structure of the discarded SVD factors. NNSVD-LRC has two other advantages compared to previous SVD-based initializations: (1) it provably generates sparse initial factors, and (2) it is faster as it only requires to compute a truncated SVD of rank r/2 + 1 where r is the factorization rank of the sought NMF decomposition (as opposed to a rank-r truncated SVD for other methods). We show on several standard dense and sparse data sets that our new method competes favorably with state-of-the-art SVD-based initializations for NMF.


page 1

page 2

page 3

page 4


Exact and Heuristic Algorithms for Semi-Nonnegative Matrix Factorization

Given a matrix M (not necessarily nonnegative) and a factorization rank ...

PD-ML-Lite: Private Distributed Machine Learning from Lighweight Cryptography

Privacy is a major issue in learning from distributed data. Recently the...

Depth from Small Motion using Rank-1 Initialization

Depth from Small Motion (DfSM) (Ha et al., 2016) is particularly interes...

Few-shot Learning for Inference in Medical Imaging with Subspace Feature Representations

Unlike the field of visual scene recognition where tremendous advances h...

Estimation with Low-Rank Time-Frequency Synthesis Models

Many state-of-the-art signal decomposition techniques rely on a low-rank...

Affine Iterations and Wrapping Effect: Various Approaches

Affine iterations of the form x(n+1) = Ax(n) + b converge, using real ar...

Please sign up or login with your details

Forgot password? Click here to reset