Dimensionality Reduction as Probabilistic Inference

by   Aditya Ravuri, et al.

Dimensionality reduction (DR) algorithms compress high-dimensional data into a lower dimensional representation while preserving important features of the data. DR is a critical step in many analysis pipelines as it enables visualisation, noise reduction and efficient downstream processing of the data. In this work, we introduce the ProbDR variational framework, which interprets a wide range of classical DR algorithms as probabilistic inference algorithms in this framework. ProbDR encompasses PCA, CMDS, LLE, LE, MVU, diffusion maps, kPCA, Isomap, (t-)SNE, and UMAP. In our framework, a low-dimensional latent variable is used to construct a covariance, precision, or a graph Laplacian matrix, which can be used as part of a generative model for the data. Inference is done by optimizing an evidence lower bound. We demonstrate the internal consistency of our framework and show that it enables the use of probabilistic programming languages (PPLs) for DR. Additionally, we illustrate that the framework facilitates reasoning about unseen data and argue that our generative models approximate Gaussian processes (GPs) on manifolds. By providing a unified view of DR, our framework facilitates communication, reasoning about uncertainties, model composition, and extensions, particularly when domain knowledge is present.


page 1

page 2

page 3

page 4


A Latent Variable Model for Two-Dimensional Canonical Correlation Analysis and its Variational Inference

Describing the dimension reduction (DR) techniques by means of probabili...

Feature Learning for Dimensionality Reduction toward Maximal Extraction of Hidden Patterns

Dimensionality reduction (DR) plays a vital role in the visual analysis ...

Joint Characterization of Spatiotemporal Data Manifolds

Spatiotemporal (ST) image data are increasingly common and often high-di...

Inversion using a new low-dimensional representation of complex binary geological media based on a deep neural network

Efficient and high-fidelity prior sampling and inversion for complex geo...

On genetic programming representations and fitness functions for interpretable dimensionality reduction

Dimensionality reduction (DR) is an important technique for data explora...

Dimensionality Reduction has Quantifiable Imperfections: Two Geometric Bounds

In this paper, we investigate Dimensionality reduction (DR) maps in an i...

ZADU: A Python Library for Evaluating the Reliability of Dimensionality Reduction Embeddings

Dimensionality reduction (DR) techniques inherently distort the original...

Please sign up or login with your details

Forgot password? Click here to reset