Graph Convolutional Neural Networks as Parametric CoKleisli morphisms
share
We define the bicategory of Graph Convolutional Neural Networks 𝐆𝐂𝐍𝐍_n for an arbitrary graph with n nodes. We show it can be factored through the already existing categorical constructions for deep learning called 𝐏𝐚𝐫𝐚 and 𝐋𝐞𝐧𝐬 with the base category set to the CoKleisli category of the product comonad. We prove that there exists an injective-on-objects, faithful 2-functor 𝐆𝐂𝐍𝐍_n →𝐏𝐚𝐫𝐚(𝖢𝗈𝖪𝗅(ℝ^n × n× -)). We show that this construction allows us to treat the adjacency matrix of a GCNN as a global parameter instead of a a local, layer-wise one. This gives us a high-level categorical characterisation of a particular kind of inductive bias GCNNs possess. Lastly, we hypothesize about possible generalisations of GCNNs to general message-passing graph neural networks, connections to equivariant learning, and the (lack of) functoriality of activation functions.
READ FULL TEXT
