Disentangling Human Error from the Ground Truth in Segmentation of Medical Images

by   Le Zhang, et al.

Recent years have seen increasing use of supervised learning methods for segmentation tasks. However, the predictive performance of these algorithms depends on the quality of labels. This problem is particularly pertinent in the medical image domain, where both the annotation cost and inter-observer variability are high. In a typical label acquisition process, different human experts provide their estimates of the 'true' segmentation labels under the influence of their own biases and competence levels. Treating these noisy labels blindly as the ground truth limits the performance that automatic segmentation algorithms can achieve. In this work, we present a method for jointly learning, from purely noisy observations alone, the reliability of individual annotators and the true segmentation label distributions, using two coupled CNNs. The separation of the two is achieved by encouraging the estimated annotators to be maximally unreliable while achieving high fidelity with the noisy training data. We first define a toy segmentation dataset based on MNIST and study the properties of the proposed algorithm. We then demonstrate the utility of the method on three public medical imaging segmentation datasets with simulated (when necessary) and real diverse annotations: 1) MSLSC (multiple-sclerosis lesions); 2) BraTS (brain tumours); 3) LIDC-IDRI (lung abnormalities). In all cases, our method outperforms competing methods and relevant baselines particularly in cases where the number of annotations is small and the amount of disagreement is large. The experiments also show strong ability to capture the complex spatial characteristics of annotators' mistakes.


page 4

page 6

page 8

page 13

page 14

page 16

page 17

page 19


Learning From Noisy Labels By Regularized Estimation Of Annotator Confusion

The predictive performance of supervised learning algorithms depends on ...

When the Ground Truth is not True: Modelling Human Biases in Temporal Annotations

In supervised learning, low quality annotations lead to poorly performin...

Drawing the Same Bounding Box Twice? Coping Noisy Annotations in Object Detection with Repeated Labels

The reliability of supervised machine learning systems depends on the ac...

Combining Heterogeneously Labeled Datasets For Training Segmentation Networks

Accurate segmentation of medical images is an important step towards ana...

How Bad is Good enough: Noisy annotations for instrument pose estimation

Though analysis of Medical Images by Deep Learning achieves unprecedente...

Learning to segment images with classification labels

Two of the most common tasks in medical imaging are classification and s...

Deep Learning for Earth Image Segmentation based on Imperfect Polyline Labels with Annotation Errors

In recent years, deep learning techniques (e.g., U-Net, DeepLab) have ac...

Please sign up or login with your details

Forgot password? Click here to reset