A Survey and Taxonomy of Adversarial Neural Networks for Text-to-Image Synthesis

by   Jorge Agnese, et al.

Text-to-image synthesis refers to computational methods which translate human written textual descriptions, in the form of keywords or sentences, into images with similar semantic meaning to the text. In earlier research, image synthesis relied mainly on word to image correlation analysis combined with supervised methods to find best alignment of the visual content matching to the text. Recent progress in deep learning (DL) has brought a new set of unsupervised deep learning methods, particularly deep generative models which are able to generate realistic visual images using suitably trained neural network models. In this paper, we review the most recent development in the text-to-image synthesis research domain. Our survey first introduces image synthesis and its challenges, and then reviews key concepts such as generative adversarial networks (GANs) and deep convolutional encoder-decoder neural networks (DCNN). After that, we propose a taxonomy to summarize GAN based text-to-image synthesis into four major categories: Semantic Enhancement GANs, Resolution Enhancement GANs, Diversity Enhancement GANS, and Motion Enhancement GANs. We elaborate the main objective of each group, and further review typical GAN architectures in each group. The taxonomy and the review outline the techniques and the evolution of different approaches, and eventually provide a clear roadmap to summarize the list of contemporaneous solutions that utilize GANs and DCNNs to generate enthralling results in categories such as human faces, birds, flowers, room interiors, object reconstruction from edge maps (games) etc. The survey will conclude with a comparison of the proposed solutions, challenges that remain unresolved, and future developments in the text-to-image synthesis domain.


page 7

page 13

page 16

page 17

page 20

page 23

page 24

page 25


Generative Adversarial Text to Image Synthesis

Automatic synthesis of realistic images from text would be interesting a...

Adversarial Text-to-Image Synthesis: A Review

With the advent of generative adversarial networks, synthesizing images ...

Generative Adversarial Networks (GANs): Challenges, Solutions, and Future Directions

Generative Adversarial Networks (GANs) is a novel class of deep generati...

Game of GANs: Game Theoretical Models for Generative Adversarial Networks

Generative Adversarial Network, as a promising research direction in the...

Image Synthesis under Limited Data: A Survey and Taxonomy

Deep generative models, which target reproducing the given data distribu...

Video Generative Adversarial Networks: A Review

With the increasing interest in the content creation field in multiple s...

Evaluation and Comparison of Deep Learning Methods for Pavement Crack Identification with Visual Images

Compared with contact detection techniques, pavement crack identificatio...

Please sign up or login with your details

Forgot password? Click here to reset