Cleora: A Simple, Strong and Scalable Graph Embedding Scheme

by   Barbara Rychalska, et al.

The area of graph embeddings is currently dominated by contrastive learning methods, which demand formulation of an explicit objective function and sampling of positive and negative examples. This creates a conceptual and computational overhead. Simple, classic unsupervised approaches like Multidimensional Scaling (MSD) or the Laplacian eigenmap skip the necessity of tedious objective optimization, directly exploiting data geometry. Unfortunately, their reliance on very costly operations such as matrix eigendecomposition make them unable to scale to large graphs that are common in today's digital world. In this paper we present Cleora: an algorithm which gets the best of two worlds, being both unsupervised and highly scalable. We show that high quality embeddings can be produced without the popular step-wise learning framework with example sampling. An intuitive learning objective of our algorithm is that a node should be similar to its neighbors, without explicitly pushing disconnected nodes apart. The objective is achieved by iterative weighted averaging of node neigbors' embeddings, followed by normalization across dimensions. Thanks to the averaging operation the algorithm makes rapid strides across the embedding space and usually reaches optimal embeddings in just a few iterations. Cleora runs faster than other state-of-the-art CPU algorithms and produces embeddings of competitive quality as measured on downstream tasks: link prediction and node classification. We show that Cleora learns a data abstraction that is similar to contrastive methods, yet at much lower computational cost. We open-source Cleora under the MIT license allowing commercial use under


Role action embeddings: scalable representation of network positions

We consider the question of embedding nodes with similar local neighborh...

SCE: Scalable Network Embedding from Sparsest Cut

Large-scale network embedding is to learn a latent representation for ea...

Node Embedding over Temporal Graphs

In this work, we present a method for node embedding in temporal graphs....

Fast and Accurate Network Embeddings via Very Sparse Random Projection

We present FastRP, a scalable and performant algorithm for learning dist...

Efficient distributed representations beyond negative sampling

This article describes an efficient method to learn distributed represen...

Maximizing Cohesion and Separation in Graph Representation Learning: A Distance-aware Negative Sampling Approach

The objective of unsupervised graph representation learning (GRL) is to ...

Understanding Negative Sampling in Graph Representation Learning

Graph representation learning has been extensively studied in recent yea...

Please sign up or login with your details

Forgot password? Click here to reset