Pluralistic Image Completion

by   Chuanxia Zheng, et al.

Most image completion methods produce only one result for each masked input, although there may be many reasonable possibilities. In this paper, we present an approach for pluralistic image completion - the task of generating multiple and diverse plausible solutions for image completion. A major challenge faced by learning-based approaches is that usually only one ground truth training instance per label. As such, sampling from conditional VAEs still leads to minimal diversity. To overcome this, we propose a novel and probabilistically principled framework with two parallel paths. One is a reconstructive path that extends the VAE through a latent space that covers all partial images with different mask sizes, and imposes priors that adapt to the number of pixels. The other is a generative path for which the conditional prior is coupled to distributions obtained in the reconstructive path. Both are supported by GANs. We also introduce a new short+long term attention layer that exploits distant relations among decoder and encoder features, improving appearance consistency. When tested on datasets with buildings (Paris), faces (CelebAHQ), and natural images (ImageNet), our method not only generated higher-quality completion results, but also with multiple and diverse plausible outputs.


page 7

page 8

page 11

page 12

page 13

page 14

page 15

page 16


Diverse Image Synthesis from Semantic Layouts via Conditional IMLE

Most existing methods for conditional image synthesis are only able to g...

Multimodal Image Outpainting With Regularized Normalized Diversification

In this paper, we study the problem of generating a set ofrealistic and ...

Grounded and Controllable Image Completion by Incorporating Lexical Semantics

In this paper, we present an approach, namely Lexical Semantic Image Com...

Pluralistic Image Completion with Probabilistic Mixture-of-Experts

Pluralistic image completion focuses on generating both visually realist...

ForkNet: Multi-branch Volumetric Semantic Completion from a Single Depth Image

We propose a novel model for 3D semantic completion from a single depth ...

Learning Object Placement via Dual-path Graph Completion

Object placement aims to place a foreground object over a background ima...

A model of semantic completion in generative episodic memory

Many different studies have suggested that episodic memory is a generati...

Please sign up or login with your details

Forgot password? Click here to reset