Image as Set of Points

by   Xu Ma, et al.

What is an image and how to extract latent features? Convolutional Networks (ConvNets) consider an image as organized pixels in a rectangular shape and extract features via convolutional operation in local region; Vision Transformers (ViTs) treat an image as a sequence of patches and extract features via attention mechanism in a global range. In this work, we introduce a straightforward and promising paradigm for visual representation, which is called Context Clusters. Context clusters (CoCs) view an image as a set of unorganized points and extract features via simplified clustering algorithm. In detail, each point includes the raw feature (e.g., color) and positional information (e.g., coordinates), and a simplified clustering algorithm is employed to group and extract deep features hierarchically. Our CoCs are convolution- and attention-free, and only rely on clustering algorithm for spatial interaction. Owing to the simple design, we show CoCs endow gratifying interpretability via the visualization of clustering process. Our CoCs aim at providing a new perspective on image and visual representation, which may enjoy broad applications in different domains and exhibit profound insights. Even though we are not targeting SOTA performance, COCs still achieve comparable or even better results than ConvNets or ViTs on several benchmarks. Codes are available at:


page 2

page 7

page 15

page 16

page 17

page 18


Very Compact Clusters with Structural Regularization via Similarity and Connectivity

Clustering algorithms have significantly improved along with Deep Neural...

Unsupervised Data Fusion with Deeper Perspective: A Novel Multi-Sensor Deep Clustering Algorithm

The ever-growing developments in technology to capture different types o...

Interpretable Visualizations with Differentiating Embedding Networks

We present a visualization algorithm based on a novel unsupervised Siame...

Color Image Clustering using Block Truncation Algorithm

With the advancement in image capturing device, the image data been gene...

Git: Clustering Based on Graph of Intensity Topology

Accuracy, Robustness to noises and scales, Interpretability, Speed, and ...

Rethinking Image Forgery Detection via Contrastive Learning and Unsupervised Clustering

Image forgery detection aims to detect and locate forged regions in an i...

Discovering Features in Sr_14Cu_24O_41 Neutron Single Crystal Diffraction Data by Cluster Analysis

To address the SMC'18 data challenge, "Discovering Features in Sr_14Cu_2...

Please sign up or login with your details

Forgot password? Click here to reset