Sparse K-Means with ℓ_∞/ℓ_0 Penalty for High-Dimensional Data Clustering
share
Sparse clustering, which aims to find a proper partition of an extremely high-dimensional data set with redundant noise features, has been attracted more and more interests in recent years. The existing studies commonly solve the problem in a framework of maximizing the weighted feature contributions subject to a ℓ_2/ℓ_1 penalty. Nevertheless, this framework has two serious drawbacks: One is that the solution of the framework unavoidably involves a considerable portion of redundant noise features in many situations, and the other is that the framework neither offers intuitive explanations on why this framework can select relevant features nor leads to any theoretical guarantee for feature selection consistency. In this article, we attempt to overcome those drawbacks through developing a new sparse clustering framework which uses a ℓ_∞/ℓ_0 penalty. First, we introduce new concepts on optimal partitions and noise features for the high-dimensional data clustering problems, based on which the previously known framework can be intuitively explained in principle. Then, we apply the suggested ℓ_∞/ℓ_0 framework to formulate a new sparse k-means model with the ℓ_∞/ℓ_0 penalty (ℓ_0-k-means for short). We propose an efficient iterative algorithm for solving the ℓ_0-k-means. To deeply understand the behavior of ℓ_0-k-means, we prove that the solution yielded by the ℓ_0-k-means algorithm has feature selection consistency whenever the data matrix is generated from a high-dimensional Gaussian mixture model. Finally, we provide experiments with both synthetic data and the Allen Developing Mouse Brain Atlas data to support that the proposed ℓ_0-k-means exhibits better noise feature detection capacity over the previously known sparse k-means with the ℓ_2/ℓ_1 penalty (ℓ_1-k-means for short).
READ FULL TEXT