VARCLUST: clustering variables using dimensionality reduction

by   Piotr Sobczyk, et al.

VARCLUST algorithm is proposed for clustering variables under the assumption that variables in a given cluster are linear combinations of a small number of hidden latent variables, corrupted by the random noise. The entire clustering task is viewed as the problem of selection of the statistical model, which is defined by the number of clusters, the partition of variables into these clusters and the 'cluster dimensions', i.e. the vector of dimensions of linear subspaces spanning each of the clusters. The optimal model is selected using the approximate Bayesian criterion based on the Laplace approximations and using a non-informative uniform prior on the number of clusters. To solve the problem of the search over a huge space of possible models we propose an extension of the ClustOfVar algorithm which was dedicated to subspaces of dimension only 1, and which is similar in structure to the K-centroid algorithm. We provide a complete methodology with theoretical guarantees, extensive numerical experimentations, complete data analyses and implementation. Our algorithm assigns variables to appropriate clusterse based on the consistent Bayesian Information Criterion (BIC), and estimates the dimensionality of each cluster by the PEnalized SEmi-integrated Likelihood Criterion (PESEL), whose consistency we prove. Additionally, we prove that each iteration of our algorithm leads to an increase of the Laplace approximation to the model posterior probability and provide the criterion for the estimation of the number of clusters. Numerical comparisons with other algorithms show that VARCLUST may outperform some popular machine learning tools for sparse subspace clustering. We also report the results of real data analysis including TCGA breast cancer data and meteorological data. The proposed method is implemented in the publicly available R package varclust.


Adaptive Bayesian Variable Clustering via Structural Learning of Breast Cancer Data

Clustering of proteins is of interest in cancer cell biology. This artic...

Hierarchical clustering with discrete latent variable models and the integrated classification likelihood

In this paper, we introduce a two step methodology to extract a hierarch...

Robust Bayesian Cluster Enumeration

A major challenge in cluster analysis is that the number of data cluster...

Subspace Clustering through Sub-Clusters

The problem of dimension reduction is of increasing importance in modern...

A Novel Bayesian Cluster Enumeration Criterion for Unsupervised Learning

The Bayesian Information Criterion (BIC) has been widely used for estima...

Measuring Cluster Stability for Bayesian Nonparametrics Using the Linear Bootstrap

Clustering procedures typically estimate which data points are clustered...

Robust Discriminative Clustering with Sparse Regularizers

Clustering high-dimensional data often requires some form of dimensional...

Please sign up or login with your details

Forgot password? Click here to reset