Image-level supervision and self-training for transformer-based cross-modality tumor segmentation

by   Malo de Boisredon, et al.

Deep neural networks are commonly used for automated medical image segmentation, but models will frequently struggle to generalize well across different imaging modalities. This issue is particularly problematic due to the limited availability of annotated data, making it difficult to deploy these models on a larger scale. To overcome these challenges, we propose a new semi-supervised training strategy called MoDATTS. Our approach is designed for accurate cross-modality 3D tumor segmentation on unpaired bi-modal datasets. An image-to-image translation strategy between imaging modalities is used to produce annotated pseudo-target volumes and improve generalization to the unannotated target modality. We also use powerful vision transformer architectures and introduce an iterative self-training procedure to further close the domain gap between modalities. MoDATTS additionally allows the possibility to extend the training to unannotated target data by exploiting image-level labels with an unsupervised objective that encourages the model to perform 3D diseased-to-healthy translation by disentangling tumors from the background. The proposed model achieves superior performance compared to other methods from participating teams in the CrossMoDA 2022 challenge, as evidenced by its reported top Dice score of 0.87+/-0.04 for the VS segmentation. MoDATTS also yields consistent improvements in Dice scores over baselines on a cross-modality brain tumor segmentation task composed of four different contrasts from the BraTS 2020 challenge dataset, where 95 supervised model performance is reached. We report that 99 maximum performance can be attained if 20 additionally annotated, which further demonstrates that MoDATTS can be leveraged to reduce the annotation burden.


page 3

page 5

page 6

page 8

page 9

page 10

page 13

page 14


M-GenSeg: Domain Adaptation For Target Modality Tumor Segmentation With Annotation-Efficient Supervision

Automated medical image segmentation using deep neural networks typicall...

Frequency Disentangled Learning for Segmentation of Midbrain Structures from Quantitative Susceptibility Mapping Data

One often lacks sufficient annotated samples for training deep segmentat...

Self-Attentive Spatial Adaptive Normalization for Cross-Modality Domain Adaptation

Despite the successes of deep neural networks on many challenging vision...

CrossMoDA 2021 challenge: Benchmark of Cross-Modality Domain Adaptation techniques for Vestibular Schwnannoma and Cochlea Segmentation

Domain Adaptation (DA) has recently raised strong interests in the medic...

Unsupervised image segmentation via maximum a posteriori estimation of continuous max-flow

Recent thrust in imaging capabilities in medical as well as emerging are...

Benefits of Linear Conditioning for Segmentation using Metadata

Medical images are often accompanied by metadata describing the image (v...

Please sign up or login with your details

Forgot password? Click here to reset