A complete characterization of optimal dictionaries for least squares representation
share
Dictionaries are collections of vectors used for representations of elements in Euclidean spaces. While recent research on optimal dictionaries is focussed on providing sparse (i.e., ℓ_0-optimal,) representations, here we consider the problem of finding optimal dictionaries such that representations of samples of a random vector are optimal in an ℓ_2-sense. For us, optimality of representation is equivalent to minimization of the average ℓ_2-norm of the coefficients used to represent the random vector, with the lengths of the dictionary vectors being specified a priori. With the help of recent results on rank-1 decompositions of symmetric positive semidefinite matrices and the theory of majorization, we provide a complete characterization of ℓ_2-optimal dictionaries. Our results are accompanied by polynomial time algorithms that construct ℓ_2-optimal dictionaries from given data.
READ FULL TEXT