Transitivity-Preserving Graph Representation Learning for Bridging Local Connectivity and Role-based Similarity

by   Van Thuy Hoang, et al.

Graph representation learning (GRL) methods, such as graph neural networks and graph transformer models, have been successfully used to analyze graph-structured data, mainly focusing on node classification and link prediction tasks. However, the existing studies mostly only consider local connectivity while ignoring long-range connectivity and the roles of nodes. In this paper, we propose Unified Graph Transformer Networks (UGT) that effectively integrate local and global structural information into fixed-length vector representations. First, UGT learns local structure by identifying the local substructures and aggregating features of the k-hop neighborhoods of each node. Second, we construct virtual edges, bridging distant nodes with structural similarity to capture the long-range dependencies. Third, UGT learns unified representations through self-attention, encoding structural distance and p-step transition probability between node pairs. Furthermore, we propose a self-supervised learning task that effectively learns transition probability to fuse local and global structural features, which could then be transferred to other downstream tasks. Experimental results on real-world benchmark datasets over various downstream tasks showed that UGT significantly outperformed baselines that consist of state-of-the-art models. In addition, UGT reaches the expressive power of the third-order Weisfeiler-Lehman isomorphism test (3d-WL) in distinguishing non-isomorphic graph pairs. The source code is available at


Companion Animal Disease Diagnostics based on Literal-aware Medical Knowledge Graph Representation Learning

Knowledge graph (KG) embedding has been used to benefit the diagnosis of...

Relational Self-Supervised Learning on Graphs

Over the past few years, graph representation learning (GRL) has been a ...

Self-supervised Graph-level Representation Learning with Local and Global Structure

This paper studies unsupervised/self-supervised whole-graph representati...

Fuse Local and Global Semantics in Representation Learning

We propose Fuse Local and Global Semantics in Representation Learning (F...

Maximizing Cohesion and Separation in Graph Representation Learning: A Distance-aware Negative Sampling Approach

The objective of unsupervised graph representation learning (GRL) is to ...

Towards Better Dynamic Graph Learning: New Architecture and Unified Library

We propose DyGFormer, a new Transformer-based architecture for dynamic g...

The Influence of Network Structural Preference on Node Classification and Link Prediction

Recent advances in complex network analysis opened a wide range of possi...

Please sign up or login with your details

Forgot password? Click here to reset