Optimal Sample Complexity for Matrix Completion and Related Problems via ℓ_2-Regularization

by   Maria-Florina Balcan, et al.

We study the strong duality of non-convex matrix factorization: we show under certain dual conditions, non-convex matrix factorization and its dual have the same optimum. This has been well understood for convex optimization, but little was known for matrix factorization. We formalize the strong duality of matrix factorization through a novel analytical framework, and show that the duality gap is zero for a wide class of matrix factorization problems. Although matrix factorization problems are hard to solve in full generality, under certain conditions the optimal solution of the non-convex program is the same as its bi-dual, and we can achieve global optimality of the non-convex program by solving its bi-dual. We apply our framework to matrix completion and robust Principal Component Analysis (PCA). While a long line of work has studied these problems, for basic problems in this area such as matrix completion, the information-theoretically optimal sample complexity was not known, and the sample complexity bounds if one also requires computational efficiency are even larger. In this work, we show that exact recoverability and strong duality hold with nearly-optimal sample complexity guarantees for matrix completion and robust PCA. For matrix completion, under the standard incoherence assumption that the underlying rank-r matrix X^* ∈R^n× n with skinny SVD UΣV^T has {U^Te_i_2^2, V^Te_i_2^2}≤μ r/n for all i, to the best of our knowledge we give (1) the first non-efficient algorithm achieving the optimal O(μ n r n) sample complexity, and (2) an efficient algorithm in our framework achieving O(κ^2μ n r (n)_2κ (n)) sample complexity.


page 1

page 2

page 3

page 4


The Leave-one-out Approach for Matrix Completion: Primal and Dual Analysis

In this paper, we introduce a powerful technique, Leave-One-Out, to the ...

Phase transitions and sample complexity in Bayes-optimal matrix factorization

We analyse the matrix factorization problem. Given a noisy measurement o...

Riemannian Perspective on Matrix Factorization

We study the non-convex matrix factorization approach to matrix completi...

Polar Alignment and Atomic Decomposition

Structured optimization uses a prescribed set of atoms to assemble a sol...

Matrix Tile Analysis

Many tasks require finding groups of elements in a matrix of numbers, sy...

Fast and Sample Efficient Inductive Matrix Completion via Multi-Phase Procrustes Flow

We revisit the inductive matrix completion problem that aims to recover ...

Global Optimality in Tensor Factorization, Deep Learning, and Beyond

Techniques involving factorization are found in a wide range of applicat...

Please sign up or login with your details

Forgot password? Click here to reset