PVNet: A Joint Convolutional Network of Point Cloud and Multi-View for 3D Shape Recognition

by   Haoxuan You, et al.

3D object recognition has attracted wide research attention in the field of multimedia and computer vision. With the recent proliferation of deep learning, various deep models with different representations have achieved the state-of-the-art performance. Among them, point cloud and multi-view based 3D shape representations are promising recently, and their corresponding deep models have shown significant performance on 3D shape recognition. However, there is little effort concentrating point cloud data and multi-view data for 3D shape representation, which is, in our consideration, beneficial and compensated to each other. In this paper, we propose the Point-View Network (PVNet), the first framework integrating both the point cloud and the multi-view data towards joint 3D shape recognition. More specifically, an embedding attention fusion scheme is proposed that could employ high-level features from the multi-view data to model the intrinsic correlation and discriminability of different structure features from the point cloud data. In particular, the discriminative descriptions are quantified and leveraged as the soft attention mask to further refine the structure feature of the 3D shape. We have evaluated the proposed method on the ModelNet40 dataset for 3D shape classification and retrieval tasks. Experimental results and comparisons with state-of-the-art methods demonstrate that our framework can achieve superior performance.


PVRNet: Point-View Relation Neural Network for 3D Shape Recognition

Three-dimensional (3D) shape recognition has drawn much research attenti...

MANet: Multimodal Attention Network based Point- View fusion for 3D Shape Recognition

3D shape recognition has attracted more and more attention as a task of ...

MeshNet: Mesh Neural Network for 3D Shape Representation

Mesh is an important and powerful type of data for 3D shapes and widely ...

Auto-MVCNN: Neural Architecture Search for Multi-view 3D Shape Recognition

In 3D shape recognition, multi-view based methods leverage human's persp...

Invariant Training 2D-3D Joint Hard Samples for Few-Shot Point Cloud Recognition

We tackle the data scarcity challenge in few-shot point cloud recognitio...

Superquadric Object Representation for Optimization-based Semantic SLAM

Introducing semantically meaningful objects to visual Simultaneous Local...

Multi-View Vision-to-Geometry Knowledge Transfer for 3D Point Cloud Shape Analysis

As two fundamental representation modalities of 3D objects, 2D multi-vie...

Please sign up or login with your details

Forgot password? Click here to reset