SkrGAN: Sketching-rendering Unconditional Generative Adversarial Networks for Medical Image Synthesis

by   Tianyang Zhang, et al.

Generative Adversarial Networks (GANs) have the capability of synthesizing images, which have been successfully applied to medical image synthesis tasks. However, most of existing methods merely consider the global contextual information and ignore the fine foreground structures, e.g., vessel, skeleton, which may contain diagnostic indicators for medical image analysis. Inspired by human painting procedure, which is composed of stroking and color rendering steps, we propose a Sketching-rendering Unconditional Generative Adversarial Network (SkrGAN) to introduce a sketch prior constraint to guide the medical image generation. In our SkrGAN, a sketch guidance module is utilized to generate a high quality structural sketch from random noise, then a color render mapping is used to embed the sketch-based representations and resemble the background appearances. Experimental results show that the proposed SkrGAN achieves the state-of-the-art results in synthesizing images for various image modalities, including retinal color fundus, X-Ray, Computed Tomography (CT) and Magnetic Resonance Imaging (MRI). In addition, we also show that the performances of medical image segmentation method have been improved by using our synthesized images as data augmentation.


page 2

page 3

page 6


Generation of Artificial CT Images using Patch-based Conditional Generative Adversarial Networks

Deep learning has a great potential to alleviate diagnosis and prognosis...

Structure Unbiased Adversarial Model for Medical Image Segmentation

Generative models have been widely proposed in image recognition to gene...

Medical Image Segmentation on MRI Images with Missing Modalities: A Review

Dealing with missing modalities in Magnetic Resonance Imaging (MRI) and ...

Synthesis and Edition of Ultrasound Images via Sketch Guided Progressive Growing GANs

Ultrasound (US) is widely accepted in clinic for anatomical structure in...

ResViT: Residual vision transformers for multi-modal medical image synthesis

Multi-modal imaging is a key healthcare technology in the diagnosis and ...

An Attentive-based Generative Model for Medical Image Synthesis

Magnetic resonance (MR) and computer tomography (CT) imaging are valuabl...

Please sign up or login with your details

Forgot password? Click here to reset