Geometry Aligned Variational Transformer for Image-conditioned Layout Generation

by   Yunning Cao, et al.

Layout generation is a novel task in computer vision, which combines the challenges in both object localization and aesthetic appraisal, widely used in advertisements, posters, and slides design. An accurate and pleasant layout should consider both the intra-domain relationship within layout elements and the inter-domain relationship between layout elements and the image. However, most previous methods simply focus on image-content-agnostic layout generation, without leveraging the complex visual information from the image. To this end, we explore a novel paradigm entitled image-conditioned layout generation, which aims to add text overlays to an image in a semantically coherent manner. Specifically, we propose an Image-Conditioned Variational Transformer (ICVT) that autoregressively generates various layouts in an image. First, self-attention mechanism is adopted to model the contextual relationship within layout elements, while cross-attention mechanism is used to fuse the visual information of conditional images. Subsequently, we take them as building blocks of conditional variational autoencoder (CVAE), which demonstrates appealing diversity. Second, in order to alleviate the gap between layout elements domain and visual domain, we design a Geometry Alignment module, in which the geometric information of the image is aligned with the layout representation. In addition, we construct a large-scale advertisement poster layout designing dataset with delicate layout and saliency map annotations. Experimental results show that our model can adaptively generate layouts in the non-intrusive area of the image, resulting in a harmonious layout design.


page 1

page 4

page 8

page 11

page 12

page 13

page 14

page 15


Variational Transformer Networks for Layout Generation

Generative models able to synthesize layouts of different kinds (e.g. do...

Relation-Aware Diffusion Model for Controllable Poster Layout Generation

Poster layout is a crucial aspect of poster design. Prior methods primar...

LayoutGAN: Generating Graphic Layouts with Wireframe Discriminators

Layout is important for graphic design and scene generation. We propose ...

Layout Generation and Completion with Self-attention

We address the problem of layout generation for diverse domains such as ...

Intelligent Home 3D: Automatic 3D-House Design from Linguistic Descriptions Only

Home design is a complex task that normally requires architects to finis...

Building-GAN: Graph-Conditioned Architectural Volumetric Design Generation

Volumetric design is the first and critical step for professional buildi...

Reverse-Engineering Information Presentations: Recovering Hierarchical Grouping from Layouts of Visual Elements

Visual elements in an information presentation are often spatially and s...

Please sign up or login with your details

Forgot password? Click here to reset