CNN-based real-time 2D-3D deformable registration from a single X-ray projection

by   François Lecomte, et al.

Purpose: The purpose of this paper is to present a method for real-time 2D-3D non-rigid registration using a single fluoroscopic image. Such a method can find applications in surgery, interventional radiology and radiotherapy. By estimating a three-dimensional displacement field from a 2D X-ray image, anatomical structures segmented in the preoperative scan can be projected onto the 2D image, thus providing a mixed reality view. Methods: A dataset composed of displacement fields and 2D projections of the anatomy is generated from the preoperative scan. From this dataset, a neural network is trained to recover the unknown 3D displacement field from a single projection image. Results: Our method is validated on lung 4D CT data at different stages of the lung deformation. The training is performed on a 3D CT using random (non domain-specific) diffeomorphic deformations, to which perturbations mimicking the pose uncertainty are added. The model achieves a mean TRE over a series of landmarks ranging from 2.3 to 5.5 mm depending on the amplitude of deformation. Conclusion: In this paper, a CNN-based method for real-time 2D-3D non-rigid registration is presented. This method is able to cope with pose estimation uncertainties, making it applicable to actual clinical scenarios, such as lung surgery, where the C-arm pose is planned before the intervention.


page 5

page 9


Per-Pixel Lung Thickness and Lung Capacity Estimation on Chest X-Rays using Convolutional Neural Networks

Estimating the lung depth on x-ray images could provide both an accurate...

Real-Time Patient-Specific Lung Radiotherapy Targeting using Deep Learning

Radiation therapy has presented a need for dynamic tracking of a target ...

Multi-scale Image Fusion Between Pre-operative Clinical CT and X-ray Microtomography of Lung Pathology

Computational anatomy allows the quantitative analysis of organs in medi...

Analysis of Heterogeneity of Pneumothorax-associated Deformation using Model-based Registration

Recent advances in imaging techniques have enabled us to visualize lung ...

Airway Label Prediction in Video Bronchoscopy: Capturing Temporal Dependencies Utilizing Anatomical Knowledge

Purpose: Navigation guidance is a key requirement for a multitude of lun...

DeepOrganNet: On-the-Fly Reconstruction and Visualization of 3D / 4D Lung Models from Single-View Projections by Deep Deformation Network

This paper introduces a deep neural network based method, i.e., DeepOrga...

Kernel-based framework to estimate deformations of pneumothorax lung using relative position of anatomical landmarks

In video-assisted thoracoscopic surgeries, successful procedures of nodu...

Please sign up or login with your details

Forgot password? Click here to reset