MRI Image-to-Image Translation for Cross-Modality Image Registration and Segmentation

by   Qianye Yang, et al.

We develop a novel cross-modality generation framework that learns to generate predicted modalities from given modalities in MR images without real acquisition. Our proposed method performs image-to-image translation by means of a deep learning model that leverages conditional generative adversarial networks (cGANs). Our framework jointly exploits the low-level features (pixel-wise information) and high-level representations (e.g. brain tumors, brain structure like gray matter, etc.) between cross modalities which are important for resolving the challenging complexity in brain structures. Based on our proposed framework, we first propose a method for cross-modality registration by fusing the deformation fields to adopt the cross-modality information from predicted modalities. Second, we propose an approach for MRI segmentation, translated multichannel segmentation (TMS), where given modalities, along with predicted modalities, are segmented by fully convolutional networks (FCN) in a multi-channel manner. Both these two methods successfully adopt the cross-modality information to improve the performance without adding any extra data. Experiments demonstrate that our proposed framework advances the state-of-the-art on five MRI datasets. We also observe encouraging results in cross-modality registration and segmentation on some widely adopted datasets. Overall, our work can serve as an auxiliary method in clinical diagnosis and be applied to various tasks in medical fields. Keywords: Image-to-image, cross-modality, registration, segmentation, MRI


page 3

page 14

page 22

page 25

page 27

page 29

page 30

page 31


Self-Attentive Spatial Adaptive Normalization for Cross-Modality Domain Adaptation

Despite the successes of deep neural networks on many challenging vision...

Unsupervised Multi-Modal Image Registration via Geometry Preserving Image-to-Image Translation

Many applications, such as autonomous driving, heavily rely on multi-mod...

Cycle-guided Denoising Diffusion Probability Model for 3D Cross-modality MRI Synthesis

This study aims to develop a novel Cycle-guided Denoising Diffusion Prob...

Robust Image Reconstruction with Misaligned Structural Information

Multi-modality (or multi-channel) imaging is becoming increasingly impor...

Generative Adversarial Networks for MR-CT Deformable Image Registration

Deformable Image Registration (DIR) of MR and CT images is one of the mo...

A Fully-Automatic Framework for Parkinson's Disease Diagnosis by Multi-Modality Images

Background: Parkinson's disease (PD) is a prevalent long-term neurodegen...

Please sign up or login with your details

Forgot password? Click here to reset