PCNet: A Structure Similarity Enhancement Method for Multispectral and Multimodal Image Registration

by   Si-Yuan Cao, et al.

Multispectral and multimodal image processing is important in the community of computer vision and computational photography. As the acquired multispectral and multimodal data are generally misaligned due to the alternation or movement of the image device, the image registration procedure is necessary. The registration of multispectral or multimodal image is challenging due to the non-linear intensity and gradient variation. To cope with this challenge, we propose the phase congruency network (PCNet), which is able to enhance the structure similarity and alleviate the non-linear intensity and gradient variation. The images can then be aligned using the similarity enhanced features produced by the network. PCNet is constructed under the guidance of the phase congruency prior. The network contains three trainable layers accompany with the modified learnable Gabor kernels according to the phase congruency theory. Thanks to the prior knowledge, PCNet is extremely light-weight and can be trained on quite a small amount of multispectral data. PCNet can be viewed to be fully convolutional and hence can take input of arbitrary sizes. Once trained, PCNet is applicable on a variety of multispectral and multimodal data such as RGB/NIR and flash/no-flash images without additional further tuning. Experimental results validate that PCNet outperforms current state-of-the-art registration algorithms, including the deep-learning based ones that have the number of parameters hundreds times compared to PCNet. Thanks to the similarity enhancement training, PCNet outperforms the original phase congruency algorithm with two-thirds less feature channels.


page 1

page 2

page 3

page 4

page 7

page 9

page 12

page 13


Robust Registration of Multimodal Remote Sensing Images Based on Structural Similarity

Automatic registration of multimodal remote sensing data (e.g., optical,...

Unsupervised Multimodal Image Registration with Adaptative Gradient Guidance

Multimodal image registration (MIR) is a fundamental procedure in many i...

Mutual information neural estimation for unsupervised multi-modal registration of brain images

Many applications in image-guided surgery and therapy require fast and r...

Edge Preserving Multi-Modal Registration Based On Gradient Intensity Self-Similarity

Image registration is a challenging task in the world of medical imaging...

Multimodal Registration of Retinal Images Using Domain-Specific Landmarks and Vessel Enhancement

The analysis of different image modalities is frequently performed in op...

Impact of PCA-based preprocessing and different CNN structures on deformable registration of sonograms

Central venous catheters (CVC) are commonly inserted into the large vein...

Bayesian intrinsic groupwise registration via explicit hierarchical disentanglement

Previous methods on multimodal groupwise registration typically require ...

Please sign up or login with your details

Forgot password? Click here to reset