Joint Registration and Segmentation via Multi-Task Learning for Adaptive Radiotherapy of Prostate Cancer

by   Mohamed S. Elmahdy, et al.

Medical image registration and segmentation are two of the most frequent tasks in medical image analysis. As these tasks are complementary and correlated, it would be beneficial to apply them simultaneously in a joint manner. In this paper, we formulate registration and segmentation as a joint problem via a Multi-Task Learning (MTL) setting, allowing these tasks to leverage their strengths and mitigate their weaknesses through the sharing of beneficial information. We propose to merge these tasks not only on the loss level, but on the architectural level as well. We studied this approach in the context of adaptive image-guided radiotherapy for prostate cancer, where planning and follow-up CT images as well as their corresponding contours are available for training. The study involves two datasets from different manufacturers and institutes. The first dataset was divided into training (12 patients) and validation (6 patients), and was used to optimize and validate the methodology, while the second dataset (14 patients) was used as an independent test set. We carried out an extensive quantitative comparison between the quality of the automatically generated contours from different network architectures as well as loss weighting methods. Moreover, we evaluated the quality of the generated deformation vector field (DVF). We show that MTL algorithms outperform their Single-Task Learning (STL) counterparts and achieve better generalization on the independent test set. The best algorithm achieved a mean surface distance of 1.06 ± 0.3 mm, 1.27 ± 0.4 mm, 0.91 ± 0.4 mm, and 1.76 ± 0.8 mm on the validation set for the prostate, seminal vesicles, bladder, and rectum, respectively. The high accuracy of the proposed method combined with the fast inference speed, makes it a promising method for automatic re-contouring of follow-up scans for adaptive radiotherapy.


page 1

page 3

page 11

page 12


A Cross-Stitch Architecture for Joint Registration and Segmentation in Adaptive Radiotherapy

Recently, joint registration and segmentation has been formulated in a d...

JSSR: A Joint Synthesis, Segmentation, and Registration System for 3D Multi-Modal Image Alignment of Large-scale Pathological CT Scans

Multi-modal image registration is a challenging problem yet important cl...

Three-dimensional reconstruction and characterization of bladder deformations

Background and Objective: Pelvic floor disorders are prevalent diseases ...

NeurReg: Neural Registration and Its Application to Image Segmentation

Registration is a fundamental task in medical image analysis which can b...

Multi-task Semi-supervised Learning for Pulmonary Lobe Segmentation

Pulmonary lobe segmentation is an important preprocessing task for the a...

Uncertainty in multitask learning: joint representations for probabilistic MR-only radiotherapy planning

Multi-task neural network architectures provide a mechanism that jointly...

3D Convolutional Neural Networks Image Registration Based on Efficient Supervised Learning from Artificial Deformations

We propose a supervised nonrigid image registration method, trained usin...

Please sign up or login with your details

Forgot password? Click here to reset