TPGNN: Learning High-order Information in Dynamic Graphs via Temporal Propagation

by   Zehong Wang, et al.

Temporal graph is an abstraction for modeling dynamic systems that consist of evolving interaction elements. In this paper, we aim to solve an important yet neglected problem – how to learn information from high-order neighbors in temporal graphs? – to enhance the informativeness and discriminativeness for the learned node representations. We argue that when learning high-order information from temporal graphs, we encounter two challenges, i.e., computational inefficiency and over-smoothing, that cannot be solved by conventional techniques applied on static graphs. To remedy these deficiencies, we propose a temporal propagation-based graph neural network, namely TPGNN. To be specific, the model consists of two distinct components, i.e., propagator and node-wise encoder. The propagator is leveraged to propagate messages from the anchor node to its temporal neighbors within k-hop, and then simultaneously update the state of neighborhoods, which enables efficient computation, especially for a deep model. In addition, to prevent over-smoothing, the model compels the messages from n-hop neighbors to update the n-hop memory vector preserved on the anchor. The node-wise encoder adopts transformer architecture to learn node representations by explicitly learning the importance of memory vectors preserved on the node itself, that is, implicitly modeling the importance of messages from neighbors at different layers, thus mitigating the over-smoothing. Since the encoding process will not query temporal neighbors, we can dramatically save time consumption in inference. Extensive experiments on temporal link prediction and node classification demonstrate the superiority of TPGNN over state-of-the-art baselines in efficiency and robustness.


page 1

page 2

page 3

page 4


Hop-Aware Dimension Optimization for Graph Neural Networks

In Graph Neural Networks (GNNs), the embedding of each node is obtained ...

PathMLP: Smooth Path Towards High-order Homophily

Real-world graphs exhibit increasing heterophily, where nodes no longer ...

Hierarchical and Contrastive Representation Learning for Knowledge-aware Recommendation

Incorporating knowledge graph into recommendation is an effective way to...

Spatial Autoregressive Coding for Graph Neural Recommendation

Graph embedding methods including traditional shallow models and deep Gr...

Graph Pointer Neural Networks

Graph Neural Networks (GNNs) have shown advantages in various graph-base...

APAN: Asynchronous Propagation Attention Network for Real-time Temporal Graph Embedding

Limited by the time complexity of querying k-hop neighbors in a graph da...

HOPF: Higher Order Propagation Framework for Deep Collective Classification

Given a graph wherein every node has certain attributes associated with ...

Please sign up or login with your details

Forgot password? Click here to reset