Translating Simulation Images to X-ray Images via Multi-Scale Semantic Matching

by   Jingxuan Kang, et al.

Endovascular intervention training is increasingly being conducted in virtual simulators. However, transferring the experience from endovascular simulators to the real world remains an open problem. The key challenge is the virtual environments are usually not realistically simulated, especially the simulation images. In this paper, we propose a new method to translate simulation images from an endovascular simulator to X-ray images. Previous image-to-image translation methods often focus on visual effects and neglect structure information, which is critical for medical images. To address this gap, we propose a new method that utilizes multi-scale semantic matching. We apply self-domain semantic matching to ensure that the input image and the generated image have the same positional semantic relationships. We further apply cross-domain matching to eliminate the effects of different styles. The intensive experiment shows that our method generates realistic X-ray images and outperforms other state-of-the-art approaches by a large margin. We also collect a new large-scale dataset to serve as the new benchmark for this task. Our source code and dataset will be made publicly available.


page 2

page 8

page 9


NeBLa: Neural Beer-Lambert for 3D Reconstruction of Oral Structures from Panoramic Radiographs

Panoramic radiography (panoramic X-ray, PX) is a widely used imaging mod...

Learning Unsupervised Cross-domain Image-to-Image Translation Using a Shared Discriminator

Unsupervised image-to-image translation is used to transform images from...

Deep Image Harmonization by Bridging the Reality Gap

Image harmonization has been significantly advanced with large-scale har...

StereoGAN: Bridging Synthetic-to-Real Domain Gap by Joint Optimization of Domain Translation and Stereo Matching

Large-scale synthetic datasets are beneficial to stereo matching but usu...

Art2Real: Unfolding the Reality of Artworks via Semantically-Aware Image-to-Image Translation

The applicability of computer vision to real paintings and artworks has ...

DASGIL: Domain Adaptation for Semantic and Geometric-aware Image-based Localization

Long-Term visual localization under changing environments is a challengi...

Please sign up or login with your details

Forgot password? Click here to reset