Sketching Matrix Least Squares via Leverage Scores Estimates

01/25/2022

∙

by   Brett W. Larsen, et al.

∙

0

∙

share

We consider the matrix least squares problem of the form 𝐀𝐗-𝐁_F^2 where the design matrix 𝐀∈ℝ^N × r is tall and skinny with N ≫ r. We propose to create a sketched version 𝐀̃𝐗-𝐁̃_F^2 where the sketched matrices 𝐀̃ and 𝐁̃ contain weighted subsets of the rows of 𝐀 and 𝐁, respectively. The subset of rows is determined via random sampling based on leverage score estimates for each row. We say that the sketched problem is ϵ-accurate if its solution 𝐗̃_opt = argmin 𝐀̃𝐗-𝐁̃_F^2 satisfies 𝐀𝐗̃_opt-𝐁_F^2 ≤ (1+ϵ) min𝐀𝐗-𝐁_F^2 with high probability. We prove that the number of samples required for an ϵ-accurate solution is O(r/(βϵ)) where β∈ (0,1] is a measure of the quality of the leverage score estimates.