Self-learning Local Supervision Encoding Framework to Constrict and Disperse Feature Distribution for Clustering

by   Jielei Chu, et al.

To obtain suitable feature distribution is a difficult task in machine learning, especially for unsupervised learning. In this paper, we propose a novel self-learning local supervision encoding framework based on RBMs, in which the self-learning local supervisions from visible layer are integrated into the contrastive divergence (CD) learning of RBMs to constrict and disperse the distribution of the hidden layer features for clustering tasks. In the framework, we use sigmoid transformation to obtain hidden layer and reconstructed hidden layer features from visible layer and reconstructed visible layer units during sampling procedure. The self-learning local supervisions contain local credible clusters which stem from different unsupervised learning and unanimous voting strategy. They are fused into hidden layer features and reconstructed hidden layer features. For the same local clusters, the hidden features and reconstructed hidden layer features of the framework tend to constrict together. Furthermore, the hidden layer features of different local clusters tend to disperse in the encoding process. Under such framework, we present two instantiation models with the reconstruction of two different visible layers. One is self-learning local supervision GRBM (slsGRBM) model with Gaussian linear visible units and binary hidden units using linear transformation for visible layer reconstruction. The other is self-learning local supervision RBM (slsRBM) model with binary visible and hidden units using sigmoid transformation for visible layer reconstruction.


page 3

page 4

page 8

page 9

page 10

page 11

page 12

page 13


DCEF: Deep Collaborative Encoder Framework for Unsupervised Clustering

Collaborative representation is a popular feature learning approach, whi...

Occam learning

We discuss probabilistic neural network models for unsupervised learning...

Complex-Valued Restricted Boltzmann Machine for Direct Speech Parameterization from Complex Spectra

This paper describes a novel energy-based probabilistic distribution tha...

Hierarchical Models as Marginals of Hierarchical Models

We investigate the representation of hierarchical models in terms of mar...

Using Inherent Structures to design Lean 2-layer RBMs

Understanding the representational power of Restricted Boltzmann Machine...

Data Mapping for Restricted Boltzmann Machine

Restricted Boltzmann machine (RBM) is interpreted as a data mapping betw...

Learning Latent Representations in Neural Networks for Clustering through Pseudo Supervision and Graph-based Activity Regularization

In this paper, we propose a novel unsupervised clustering approach explo...

Please sign up or login with your details

Forgot password? Click here to reset