SIEDOB: Semantic Image Editing by Disentangling Object and Background

by   Wuyang Luo, et al.

Semantic image editing provides users with a flexible tool to modify a given image guided by a corresponding segmentation map. In this task, the features of the foreground objects and the backgrounds are quite different. However, all previous methods handle backgrounds and objects as a whole using a monolithic model. Consequently, they remain limited in processing content-rich images and suffer from generating unrealistic objects and texture-inconsistent backgrounds. To address this issue, we propose a novel paradigm, Semantic Image Editing by Disentangling Object and Background (SIEDOB), the core idea of which is to explicitly leverages several heterogeneous subnetworks for objects and backgrounds. First, SIEDOB disassembles the edited input into background regions and instance-level objects. Then, we feed them into the dedicated generators. Finally, all synthesized parts are embedded in their original locations and utilize a fusion network to obtain a harmonized result. Moreover, to produce high-quality edited images, we propose some innovative designs, including Semantic-Aware Self-Propagation Module, Boundary-Anchored Patch Discriminator, and Style-Diversity Object Generator, and integrate them into SIEDOB. We conduct extensive experiments on Cityscapes and ADE20K-Room datasets and exhibit that our method remarkably outperforms the baselines, especially in synthesizing realistic and diverse objects and texture-consistent backgrounds.


page 1

page 3

page 4

page 5

page 6

page 7

page 8


PFB-Diff: Progressive Feature Blending Diffusion for Text-driven Image Editing

Diffusion models have showcased their remarkable capability to synthesiz...

Generating Object Stamps

We present an algorithm to generate diverse foreground objects and compo...

Scene Style Text Editing

In this work, we propose a task called "Scene Style Text Editing (SSTE)"...

SESAME: Semantic Editing of Scenes by Adding, Manipulating or Erasing Objects

Recent advances in image generation gave rise to powerful tools for sema...

Auto-Retoucher(ART) - A framework for Background Replacement and Image Editing

Replacing the background and simultaneously adjusting foreground objects...

Unconstrained Foreground Object Search

Many people search for foreground objects to use when editing images. Wh...

Towards High-Quality Specular Highlight Removal by Leveraging Large-Scale Synthetic Data

This paper aims to remove specular highlights from a single object-level...

Please sign up or login with your details

Forgot password? Click here to reset