Estimating the coefficients of a mixture of two linear regressions by expectation maximization
share
We give convergence guarantees for estimating the coefficients of a symmetric mixture of two linear regressions by expectation maximization (EM). In particular, we show that convergence of the empirical iterates is guaranteed provided the algorithm is initialized in an unbounded cone. That is, if the initializer has a large cosine angle with the population coefficient vector and the signal to noise ratio (SNR) is large, a sample-splitting version of the EM algorithm converges to the true coefficient vector with high probability. Here "large" means that each quantity is required to be at least a universal constant. Finally, we show that the population EM operator is not globally contractive by characterizing a region where it fails. We give empirical evidence that suggests that the sample based EM performs poorly when intitializers are drawn from this set. Interestingly, our analysis borrows from tools used in the problem of estimating the centers of a symmetric mixture of two Gaussians by EM.
READ FULL TEXT