Learning a Lie Algebra from Unlabeled Data Pairs
share
Deep convolutional networks (convnets) show a remarkable ability to learn disentangled representations. In recent years, the generalization of deep learning to Lie groups beyond rigid motion in ℝ^n has allowed to build convnets over datasets with non-trivial symmetries, such as patterns over the surface of a sphere. However, one limitation of this approach is the need to explicitly define the Lie group underlying the desired invariance property before training the convnet. Whereas rotations on the sphere have a well-known symmetry group (SO(3)), the same cannot be said of many real-world factors of variability. For example, the disentanglement of pitch, intensity dynamics, and playing technique remains a challenging task in music information retrieval. This article proposes a machine learning method to discover a nonlinear transformation of the space ℝ^n which maps a collection of n-dimensional vectors (x_i)_i onto a collection of target vectors (y_i)_i. The key idea is to approximate every target y_i by a matrix–vector product of the form y_i = ϕ(t_i) x_i, where the matrix ϕ(t_i) belongs to a one-parameter subgroup of GL_n (ℝ). Crucially, the value of the parameter t_i ∈ℝ may change between data pairs (x_i, y_i) and does not need to be known in advance.
READ FULL TEXT