Paired-Unpaired Unsupervised Attention Guided GAN with Transfer Learning for Bidirectional Brain MR-CT Synthesis

by   Omar S. Al-Kadi, et al.

Medical image acquisition plays a significant role in the diagnosis and management of diseases. Magnetic Resonance (MR) and Computed Tomography (CT) are considered two of the most popular modalities for medical image acquisition. Some considerations, such as cost and radiation dose, may limit the acquisition of certain image modalities. Therefore, medical image synthesis can be used to generate required medical images without actual acquisition. In this paper, we propose a paired–unpaired Unsupervised Attention Guided Generative Adversarial Network (uagGAN) model to translate MR images to CT images and vice versa. The uagGAN model is pre-trained with a paired dataset for initialization and then retrained on an unpaired dataset using a cascading process. In the paired pre-training stage, we enhance the loss function of our model by combining the Wasserstein GAN adversarial loss function with a new combination of non-adversarial losses (content loss and L1) to generate fine structure images. This will ensure global consistency, and better capture of the high and low frequency details of the generated images. The uagGAN model is employed as it generates more accurate and sharper images through the production of attention masks. Knowledge from a non-medical pre-trained model is also transferred to the uagGAN model for improved learning and better image translation performance. Quantitative evaluation and qualitative perceptual analysis by radiologists indicate that employing transfer learning with the proposed paired-unpaired uagGAN model can achieve better performance as compared to other rival image-to-image translation models.


page 2

page 4

page 7

page 8

page 10

page 11


Structurally aware bidirectional unpaired image to image translation between CT and MR

Magnetic Resonance (MR) Imaging and Computed Tomography (CT) are the pri...

Bridging the gap between paired and unpaired medical image translation

Medical image translation has the potential to reduce the imaging worklo...

Unsupervised Medical Image Translation Using Cycle-MedGAN

Image-to-image translation is a new field in computer vision with multip...

An Attentive-based Generative Model for Medical Image Synthesis

Magnetic resonance (MR) and computer tomography (CT) imaging are valuabl...

MR to X-Ray Projection Image Synthesis

Hybrid imaging promises large potential in medical imaging applications....

DC-cycleGAN: Bidirectional CT-to-MR Synthesis from Unpaired Data

Magnetic resonance (MR) and computer tomography (CT) images are two typi...

Empirical Bayesian Mixture Models for Medical Image Translation

Automatically generating one medical imaging modality from another is kn...

Please sign up or login with your details

Forgot password? Click here to reset