Sparse Matrix Factorization

11/13/2013
by   Behnam Neyshabur, et al.
0

We investigate the problem of factorizing a matrix into several sparse matrices and propose an algorithm for this under randomness and sparsity assumptions. This problem can be viewed as a simplification of the deep learning problem where finding a factorization corresponds to finding edges in different layers and values of hidden units. We prove that under certain assumptions for a sparse linear deep network with n nodes in each layer, our algorithm is able to recover the structure of the network and values of top layer hidden units for depths up to Õ(n^1/6). We further discuss the relation among sparse matrix factorization, deep learning, sparse recovery and dictionary learning.

READ FULL TEXT

Please sign up or login with your details

Forgot password? Click here to reset