ComicGAN: Text-to-Comic Generative Adversarial Network

by   Ben Proven-Bessel, et al.

Drawing and annotating comic illustrations is a complex and difficult process. No existing machine learning algorithms have been developed to create comic illustrations based on descriptions of illustrations, or the dialogue in comics. Moreover, it is not known if a generative adversarial network (GAN) can generate original comics that correspond to the dialogue and/or descriptions. GANs are successful in producing photo-realistic images, but this technology does not necessarily translate to generation of flawless comics. What is more, comic evaluation is a prominent challenge as common metrics such as Inception Score will not perform comparably, as they are designed to work on photos. In this paper: 1. We implement ComicGAN, a novel text-to-comic pipeline based on a text-to-image GAN that synthesizes comics according to text descriptions. 2. We describe an in-depth empirical study of the technical difficulties of comic generation using GAN's. ComicGAN has two novel features: (i) text description creation from labels via permutation and augmentation, and (ii) custom image encoding with Convolutional Neural Networks. We extensively evaluate the proposed ComicGAN in two scenarios, namely image generation from descriptions, and image generation from dialogue. Our results on 1000 Dilbert comic panels and 6000 descriptions show synthetic comic panels from text inputs resemble original Dilbert panels. Novel methods for text description creation and custom image encoding brought improvements to Frechet Inception Distance, detail, and overall image quality over baseline algorithms. Generating illustrations from descriptions provided clear comics including characters and colours that were specified in the descriptions.


page 4

page 5

page 6

page 7

page 8


Fine-Grained Image Generation from Bangla Text Description using Attentional Generative Adversarial Network

Generating fine-grained, realistic images from text has many application...

Text to artistic image generation

Painting is one of the ways for people to express their ideas, but what ...

Sequential Attention GAN for Interactive Image Editing via Dialogue

In this paper, we introduce a new task - interactive image editing via c...

Video Generation From Text

Generating videos from text has proven to be a significant challenge for...

TiVGAN: Text to Image to Video Generation with Step-by-Step Evolutionary Generator

Advances in technology have led to the development of methods that can c...

Cycle Text-To-Image GAN with BERT

We explore novel approaches to the task of image generation from their r...

Multiple Description Convolutional Neural Networks for Image Compression

Multiple description coding (MDC) is able to stably transmit the signal ...

Please sign up or login with your details

Forgot password? Click here to reset