Transfer Learning from an Artificial Radiograph-landmark Dataset for Registration of the Anatomic Skull Model to Dual Fluoroscopic X-ray Images

by   Chaochao Zhou, et al.

Registration of 3D anatomic structures to their 2D dual fluoroscopic X-ray images is a widely used motion tracking technique. However, deep learning implementation is often impeded by a paucity of medical images and ground truths. In this study, we proposed a transfer learning strategy for 3D-to-2D registration using deep neural networks trained from an artificial dataset. Digitally reconstructed radiographs (DRRs) and radiographic skull landmarks were automatically created from craniocervical CT data of a female subject. They were used to train a residual network (ResNet) for landmark detection and a cycle generative adversarial network (GAN) to eliminate the style difference between DRRs and actual X-rays. Landmarks on the X-rays experiencing GAN style translation were detected by the ResNet, and were used in triangulation optimization for 3D-to-2D registration of the skull in actual dual-fluoroscope images (with a non-orthogonal setup, point X-ray sources, image distortions, and partially captured skull regions). The registration accuracy was evaluated in multiple scenarios of craniocervical motions. In walking, learning-based registration for the skull had angular/position errors of 3.9 +- 2.1 deg / 4.6 +- 2.2 mm. However, the accuracy was lower during functional neck activity, due to overly small skull regions imaged on the dual fluoroscopic images at end-range positions. The methodology to strategically augment artificial training data can tackle the complicated skull registration scenario, and has potentials to extend to widespread registration scenarios.


page 20

page 22

page 24

page 25

page 27

page 29

page 30

page 31


Pose-dependent weights and Domain Randomization for fully automatic X-ray to CT Registration

Fully automatic X-ray to CT registration requires a solid initialization...

KidneyRegNet: A Deep Learning Method for 3DCT-2DUS Kidney Registration during Breathing

This work proposed a novel deep registration pipeline for 3D CT and 2D U...

Representing Ambiguity in Registration Problems with Conditional Invertible Neural Networks

Image registration is the basis for many applications in the fields of m...

Combining Transfer Learning And Segmentation Information with GANs for Training Data Independent Image Registration

Registration is an important task in automated medical image analysis. A...

Automatic 2D-3D Registration without Contrast Agent during Neurovascular Interventions

Fusing live fluoroscopy images with a 3D rotational reconstruction of th...

Automatic Annotation of Hip Anatomy in Fluoroscopy for Robust and Efficient 2D/3D Registration

Fluoroscopy is the standard imaging modality used to guide hip surgery a...

3D Convolutional Neural Networks Image Registration Based on Efficient Supervised Learning from Artificial Deformations

We propose a supervised nonrigid image registration method, trained usin...

Please sign up or login with your details

Forgot password? Click here to reset