3D ShapeNets: A Deep Representation for Volumetric Shapes

by   Zhirong Wu, et al.

3D shape is a crucial but heavily underutilized cue in today's computer vision systems, mostly due to the lack of a good generic shape representation. With the recent availability of inexpensive 2.5D depth sensors (e.g. Microsoft Kinect), it is becoming increasingly important to have a powerful 3D shape representation in the loop. Apart from category recognition, recovering full 3D shapes from view-based 2.5D depth maps is also a critical part of visual understanding. To this end, we propose to represent a geometric 3D shape as a probability distribution of binary variables on a 3D voxel grid, using a Convolutional Deep Belief Network. Our model, 3D ShapeNets, learns the distribution of complex 3D shapes across different object categories and arbitrary poses from raw CAD data, and discovers hierarchical compositional part representations automatically. It naturally supports joint object recognition and shape completion from 2.5D depth maps, and it enables active object recognition through view planning. To train our 3D deep learning model, we construct ModelNet -- a large-scale 3D CAD model dataset. Extensive experiments show that our 3D deep representation enables significant performance improvement over the-state-of-the-arts in a variety of tasks.


page 1

page 2

page 3

page 4

page 5

page 6

page 7

page 8


VConv-DAE: Deep Volumetric Shape Learning Without Object Labels

With the advent of affordable depth sensors, 3D capture becomes more and...

CIGMO: Categorical invariant representations in a deep generative framework

Data of general object images have two most common structures: (1) each ...

Mask2CAD: 3D Shape Prediction by Learning to Segment and Retrieve

Object recognition has seen significant progress in the image domain, wi...

Pixels, voxels, and views: A study of shape representations for single view 3D object shape prediction

The goal of this paper is to compare surface-based and volumetric 3D obj...

Learning Discriminative 3D Shape Representations by View Discerning Networks

In view-based 3D shape recognition, extracting discriminative visual rep...

Learning Compositional Representation for 4D Captures with Neural ODE

Learning based representation has become the key to the success of many ...

One-Shot Concept Learning by Simulating Evolutionary Instinct Development

Object recognition has become a crucial part of machine learning and com...

Please sign up or login with your details

Forgot password? Click here to reset