Novel View Synthesis from Single Images via Point Cloud Transformation

by   Hoang-An Le, et al.

In this paper the argument is made that for true novel view synthesis of objects, where the object can be synthesized from any viewpoint, an explicit 3D shape representation isdesired. Our method estimates point clouds to capture the geometry of the object, which can be freely rotated into the desired view and then projected into a new image. This image, however, is sparse by nature and hence this coarse view is used as the input of an image completion network to obtain the dense target view. The point cloud is obtained using the predicted pixel-wise depth map, estimated from a single RGB input image,combined with the camera intrinsics. By using forward warping and backward warpingbetween the input view and the target view, the network can be trained end-to-end without supervision on depth. The benefit of using point clouds as an explicit 3D shape for novel view synthesis is experimentally validated on the 3D ShapeNet benchmark. Source code and data will be available at


page 6

page 10

page 14

page 15

page 16

page 17

page 18

page 19


Leveraging Single-View Images for Unsupervised 3D Point Cloud Completion

Point clouds captured by scanning devices are often incomplete due to oc...

Learning A Unified 3D Point Cloud for View Synthesis

3D point cloud representation-based view synthesis methods have demonstr...

Method for the generation of depth images for view-based shape retrieval of 3D CAD model from partial point cloud

A laser scanner can easily acquire the geometric data of physical enviro...

3D Point Cloud Enhancement using Graph-Modelled Multiview Depth Measurements

A 3D point cloud is often synthesized from depth measurements collected ...

NeuralEditor: Editing Neural Radiance Fields via Manipulating Point Clouds

This paper proposes NeuralEditor that enables neural radiance fields (Ne...

Learning to Generate Dense Point Clouds with Textures on Multiple Categories

3D reconstruction from images is a core problem in computer vision. With...

CloudAAE: Learning 6D Object Pose Regression with On-line Data Synthesis on Point Clouds

It is often desired to train 6D pose estimation systems on synthetic dat...

Please sign up or login with your details

Forgot password? Click here to reset