Matrix Factorization using Window Sampling and Negative Sampling for Improved Word Representations

06/02/2016
by   Alexandre Salle, et al.
0

In this paper, we propose LexVec, a new method for generating distributed word representations that uses low-rank, weighted factorization of the Positive Point-wise Mutual Information matrix via stochastic gradient descent, employing a weighting scheme that assigns heavier penalties for errors on frequent co-occurrences while still accounting for negative co-occurrence. Evaluation on word similarity and analogy tasks shows that LexVec matches and often outperforms state-of-the-art methods on many of these tasks.

READ FULL TEXT

Please sign up or login with your details

Forgot password? Click here to reset