DCEF: Deep Collaborative Encoder Framework for Unsupervised Clustering

by   Jielei Chu, et al.

Collaborative representation is a popular feature learning approach, which encoding process is assisted by variety types of information. In this paper, we propose a collaborative representation restricted Boltzmann Machine (CRRBM) for modeling binary data and a collaborative representation Gaussian restricted Boltzmann Machine (CRGRBM) for modeling realvalued data by applying a collaborative representation strategy in the encoding procedure. We utilize Locality Sensitive Hashing (LSH) to generate similar sample subsets of the instance and observed feature set simultaneously from input data. Hence, we can obtain some mini blocks, which come from the intersection of instance and observed feature subsets. Then we integrate Contrastive Divergence and Bregman Divergence methods with mini blocks to optimize our CRRBM and CRGRBM models. In their training process, the complex collaborative relationships between multiple instances and features are fused into the hidden layer encoding. Hence, these encodings have dual characteristics of concealment and cooperation. Here, we develop two deep collaborative encoder frameworks (DCEF) based on the CRRBM and CRGRBM models: one is a DCEF with Gaussian linear visible units (GDCEF) for modeling real-valued data, and the other is a DCEF with binary visible units (BDCEF) for modeling binary data. We explore the collaborative representation capability of the hidden features in every layer of the GDCEF and BDCEF framework, especially in the deepest hidden layer. The experimental results show that the GDCEF and BDCEF frameworks have more outstanding performances than the classic Autoencoder framework for unsupervised clustering task on the MSRA-MM2.0 and UCI datasets, respectively.


page 1

page 2

page 3

page 6

page 8

page 10

page 12

page 13


Self-learning Local Supervision Encoding Framework to Constrict and Disperse Feature Distribution for Clustering

To obtain suitable feature distribution is a difficult task in machine l...

Minor Constraint Disturbances for Deep Semi-supervised Learning

In high-dimensional data space, semi-supervised feature learning based o...

Complex-Valued Restricted Boltzmann Machine for Direct Speech Parameterization from Complex Spectra

This paper describes a novel energy-based probabilistic distribution tha...

Unsupervised prototype learning in an associative-memory network

Unsupervised learning in a generalized Hopfield associative-memory netwo...

Data Mapping for Restricted Boltzmann Machine

Restricted Boltzmann machine (RBM) is interpreted as a data mapping betw...

Theta-RBM: Unfactored Gated Restricted Boltzmann Machine for Rotation-Invariant Representations

Learning invariant representations is a critical task in computer vision...

Learning Gaussian-Bernoulli RBMs using Difference of Convex Functions Optimization

The Gaussian-Bernoulli restricted Boltzmann machine (GB-RBM) is a useful...

Please sign up or login with your details

Forgot password? Click here to reset