Fully Hyperbolic Neural Networks

by   Weize Chen, et al.

Hyperbolic neural networks have shown great potential for modeling complex data. However, existing hyperbolic networks are not completely hyperbolic, as they encode features in a hyperbolic space yet formalize most of their operations in the tangent space (a Euclidean subspace) at the origin of the hyperbolic space. This hybrid method greatly limits the modeling ability of networks. In this paper, we propose a fully hyperbolic framework to build hyperbolic networks based on the Lorentz model by adapting the Lorentz transformations (including boost and rotation) to formalize essential operations of neural networks. Moreover, we also prove that linear transformation in tangent spaces used by existing hyperbolic networks is a relaxation of the Lorentz rotation and does not include the boost, implicitly limiting the capabilities of existing hyperbolic networks. The experimental results on four NLP tasks show that our method has better performance for building both shallow and deep networks. Our code will be released to facilitate follow-up research.


page 1

page 2

page 3

page 4


Pseudo-Poincaré: A Unification Framework for Euclidean and Hyperbolic Graph Neural Networks

Hyperbolic neural networks have recently gained significant attention du...

Poincaré ResNet

This paper introduces an end-to-end residual network that operates entir...

Hyperbolic Convolution via Kernel Point Aggregation

Learning representations according to the underlying geometry is of vita...

Hyperbolic Attention Networks

We introduce hyperbolic attention networks to endow neural networks with...

Hyperbolic Convolutional Neural Networks

Deep Learning is mostly responsible for the surge of interest in Artific...

Nested Hyperbolic Spaces for Dimensionality Reduction and Hyperbolic NN Design

Hyperbolic neural networks have been popular in the recent past due to t...

Stable Geodesic Update on Hyperbolic Space and its Application to Poincare Embeddings

A hyperbolic space has been shown to be more capable of modeling complex...

Please sign up or login with your details

Forgot password? Click here to reset