Convolutional Neural Networks on 3D Surfaces Using Parallel Frames

by   Hao Pan, et al.

We extend Convolutional Neural Networks (CNNs) on flat and regular domains (e.g. 2D images) to curved surfaces embedded in 3D Euclidean space that are discretized as irregular meshes and widely used to represent geometric data in Computer Vision and Graphics. We define surface convolution on tangent spaces of a surface domain, where the convolution has two desirable properties: 1) the distortion of surface domain signals is locally minimal when being projected to the tangent space, and 2) the translation equi-variance property holds locally, by aligning tangent spaces with the canonical parallel transport that preserves metric. For computation, we rely on a parallel N-direction frame field on the surface that minimizes field variation and therefore is as compatible as possible to and approximates the parallel transport. On the tangent spaces equipped with parallel frames, the computation of surface convolution becomes standard routine. The frames have rotational symmetry which we disambiguate by constructing the covering space of surface induced by the parallel frames and grouping the feature maps into N sets accordingly; convolution is computed on the N branches of the cover space with respective feature maps while the kernel weights are shared. To handle irregular points of a discrete mesh while sharing kernel weights, we make the convolution semi-discrete, i.e. the convolution kernels are polynomial functions, and their convolution with discrete surface points becomes sampling and weighted summation. Pooling and unpooling operations are computed along a mesh hierarchy built through simplification. The presented surface CNNs allow effective deep learning on meshes. We show that for tasks of classification, segmentation and non-rigid registration, surface CNNs using only raw input signals achieve superior performances than previous models using sophisticated input features.


CNNs on Surfaces using Rotation-Equivariant Features

This paper is concerned with a fundamental problem in geometric deep lea...

Spherical U-Net on Cortical Surfaces: Methods and Applications

Convolutional Neural Networks (CNNs) have been providing the state-of-th...

3D object classification and retrieval with Spherical CNNs

3D object classification and retrieval presents many challenges that are...

DualConvMesh-Net: Joint Geodesic and Euclidean Convolutions on 3D Meshes

We propose DualConvMesh-Nets (DCM-Net) a family of deep hierarchical con...

FPNN: Field Probing Neural Networks for 3D Data

Building discriminative representations for 3D data has been an importan...

Field Convolutions for Surface CNNs

We present a novel surface convolution operator acting on vector fields ...

Multi-directional Geodesic Neural Networks via Equivariant Convolution

We propose a novel approach for performing convolution of signals on cur...

Please sign up or login with your details

Forgot password? Click here to reset