Equivariant Polynomials for Graph Neural Networks

by   Omri Puny, et al.

Graph Neural Networks (GNN) are inherently limited in their expressive power. Recent seminal works (Xu et al., 2019; Morris et al., 2019b) introduced the Weisfeiler-Lehman (WL) hierarchy as a measure of expressive power. Although this hierarchy has propelled significant advances in GNN analysis and architecture developments, it suffers from several significant limitations. These include a complex definition that lacks direct guidance for model improvement and a WL hierarchy that is too coarse to study current GNNs. This paper introduces an alternative expressive power hierarchy based on the ability of GNNs to calculate equivariant polynomials of a certain degree. As a first step, we provide a full characterization of all equivariant graph polynomials by introducing a concrete basis, significantly generalizing previous results. Each basis element corresponds to a specific multi-graph, and its computation over some graph data input corresponds to a tensor contraction problem. Second, we propose algorithmic tools for evaluating the expressiveness of GNNs using tensor contraction sequences, and calculate the expressive power of popular GNNs. Finally, we enhance the expressivity of common GNN architectures by adding polynomial features or additional operations / aggregations inspired by our theory. These enhanced GNNs demonstrate state-of-the-art results in experiments across multiple graph learning benchmarks.


page 1

page 2

page 3

page 4


Graph Neural Networks Are More Powerful Than we Think

Graph Neural Networks (GNNs) are powerful convolutional architectures th...

From Graph Low-Rank Global Attention to 2-FWL Approximation

Graph Neural Networks (GNNs) are known to have an expressive power bound...

Rethinking the Expressive Power of GNNs via Graph Biconnectivity

Designing expressive Graph Neural Networks (GNNs) is a central topic in ...

On the Ability of Graph Neural Networks to Model Interactions Between Vertices

Graph neural networks (GNNs) are widely used for modeling complex intera...

Snowflake: Scaling GNNs to High-Dimensional Continuous Control via Parameter Freezing

Recent research has shown that Graph Neural Networks (GNNs) can learn po...

Towards Arbitrarily Expressive GNNs in O(n^2) Space by Rethinking Folklore Weisfeiler-Lehman

Message passing neural networks (MPNNs) have emerged as the most popular...

SE(3)-equivariant Graph Neural Networks for Learning Glassy Liquids Representations

Within the glassy liquids community, the use of Machine Learning (ML) to...

Please sign up or login with your details

Forgot password? Click here to reset