Cross-Modality Image Registration using a Training-Time Privileged Third Modality

by   Qianye Yang, et al.

In this work, we consider the task of pairwise cross-modality image registration, which may benefit from exploiting additional images available only at training time from an additional modality that is different to those being registered. As an example, we focus on aligning intra-subject multiparametric Magnetic Resonance (mpMR) images, between T2-weighted (T2w) scans and diffusion-weighted scans with high b-value (DWI_high-b). For the application of localising tumours in mpMR images, diffusion scans with zero b-value (DWI_b=0) are considered easier to register to T2w due to the availability of corresponding features. We propose a learning from privileged modality algorithm, using a training-only imaging modality DWI_b=0, to support the challenging multi-modality registration problems. We present experimental results based on 369 sets of 3D multiparametric MRI images from 356 prostate cancer patients and report, with statistical significance, a lowered median target registration error of 4.34 mm, when registering the holdout DWI_high-b and T2w image pairs, compared with that of 7.96 mm before registration. Results also show that the proposed learning-based registration networks enabled efficient registration with comparable or better accuracy, compared with a classical iterative algorithm and other tested learning-based methods with/without the additional modality. These compared algorithms also failed to produce any significantly improved alignment between DWI_high-b and T2w in this challenging application.


page 1

page 2

page 4

page 7

page 8


Self-supervised iRegNet for the Registration of Longitudinal Brain MRI of Diffuse Glioma Patients

Reliable and accurate registration of patient-specific brain magnetic re...

Unsupervised MMRegNet based on Spatially Encoded Gradient Information

Multi-modality medical images can provide relevant and complementary ana...

Elastic registration based on compliance analysis and biomechanical graph matching

An automatic elastic registration method suited for vascularized organs ...

Collaborative Quantization Embeddings for Intra-Subject Prostate MR Image Registration

Image registration is useful for quantifying morphological changes in lo...

Flexible Bayesian Modelling for Nonlinear Image Registration

We describe a diffeomorphic registration algorithm that allows groups of...

Synth-by-Reg (SbR): Contrastive learning for synthesis-based registration of paired images

Nonlinear inter-modality registration is often challenging due to the la...

Combiner and HyperCombiner Networks: Rules to Combine Multimodality MR Images for Prostate Cancer Localisation

One of the distinct characteristics in radiologists' reading of multipar...

Please sign up or login with your details

Forgot password? Click here to reset