Efficient human-in-loop deep learning model training with iterative refinement and statistical result validation

by   Manuel Zahn, et al.

Annotation and labeling of images are some of the biggest challenges in applying deep learning to medical data. Current processes are time and cost-intensive and, therefore, a limiting factor for the wide adoption of the technology. Additionally validating that measured performance improvements are significant is important to select the best model. In this paper, we demonstrate a method for creating segmentations, a necessary part of a data cleaning for ultrasound imaging machine learning pipelines. We propose a four-step method to leverage automatically generated training data and fast human visual checks to improve model accuracy while keeping the time/effort and cost low. We also showcase running experiments multiple times to allow the usage of statistical analysis. Poor quality automated ground truth data and quick visual inspections efficiently train an initial base model, which is refined using a small set of more expensive human-generated ground truth data. The method is demonstrated on a cardiac ultrasound segmentation task, removing background data, including static PHI. Significance is shown by running the experiments multiple times and using the student's t-test on the performance distributions. The initial segmentation accuracy of a simple thresholding algorithm of 92 complicated algorithms can be matched or beaten by pre-training with the poorer performing algorithms and a small quantity of high-quality data. The introduction of statistic significance analysis for deep learning models helps to validate the performance improvements measured. The method offers a cost-effective and fast approach to achieving high-accuracy models while minimizing the cost and effort of acquiring high-quality training data.


page 4

page 5

page 8

page 10

page 16


Training Deep Learning based Denoisers without Ground Truth Data

Recent deep learning based denoisers are trained to minimize the mean sq...

A Novel Deep Learning Pipeline for Retinal Vessel Detection in Fluorescein Angiography

While recent advances in deep learning have significantly advanced the s...

ArSDM: Colonoscopy Images Synthesis with Adaptive Refinement Semantic Diffusion Models

Colonoscopy analysis, particularly automatic polyp segmentation and dete...

Improving NER's Performance with Massive financial corpus

Training large deep neural networks needs massive high quality annotatio...

Distributional Ground Truth: Non-Redundant Crowdsourcing Data Quality Control in UI Labeling Tasks

HCI increasingly employs Machine Learning and Image Recognition, in part...

A System-driven Automatic Ground Truth Generation Method for DL Inner-City Driving Corridor Detectors

Data-driven perception approaches are well-established in automated driv...

Please sign up or login with your details

Forgot password? Click here to reset