3D-aware Conditional Image Synthesis

02/16/2023
by   Kangle Deng, et al.
0

We propose pix2pix3D, a 3D-aware conditional generative model for controllable photorealistic image synthesis. Given a 2D label map, such as a segmentation or edge map, our model learns to synthesize a corresponding image from different viewpoints. To enable explicit 3D user control, we extend conditional generative models with neural radiance fields. Given widely-available monocular images and label map pairs, our model learns to assign a label to every 3D point in addition to color and density, which enables it to render the image and pixel-aligned label map simultaneously. Finally, we build an interactive system that allows users to edit the label map from any viewpoint and generate outputs accordingly.

READ FULL TEXT

page 1

page 3

page 5

page 6

page 8

page 14

research
04/03/2019

Conditional Adversarial Generative Flow for Controllable Image Synthesis

Flow-based generative models show great potential in image synthesis due...
research
08/24/2020

CA-GAN: Weakly Supervised Color Aware GAN for Controllable Makeup Transfer

While existing makeup style transfer models perform an image synthesis w...
research
11/10/2021

A Multi-attribute Controllable Generative Model for Histopathology Image Synthesis

Generative models have been applied in the medical imaging domain for va...
research
08/21/2023

Patternshop: Editing Point Patterns by Image Manipulation

Point patterns are characterized by their density and correlation. While...
research
09/09/2023

VeRi3D: Generative Vertex-based Radiance Fields for 3D Controllable Human Image Synthesis

Unsupervised learning of 3D-aware generative adversarial networks has la...
research
06/21/2021

Understanding Object Dynamics for Interactive Image-to-Video Synthesis

What would be the effect of locally poking a static scene? We present an...
research
01/13/2022

VoLux-GAN: A Generative Model for 3D Face Synthesis with HDRI Relighting

We propose VoLux-GAN, a generative framework to synthesize 3D-aware face...

Please sign up or login with your details

Forgot password? Click here to reset