Fast computation of mutual information in the frequency domain with applications to global multimodal image alignment

by   Johan Öfverstedt, et al.

Multimodal image alignment is the process of finding spatial correspondences between images formed by different imaging techniques or under different conditions, to facilitate heterogeneous data fusion and correlative analysis. The information-theoretic concept of mutual information (MI) is widely used as a similarity measure to guide multimodal alignment processes, where most works have focused on local maximization of MI that typically works well only for small displacements; this points to a need for global maximization of MI, which has previously been computationally infeasible due to the high run-time complexity of existing algorithms. We propose an efficient algorithm for computing MI for all discrete displacements (formalized as the cross-mutual information function (CMIF)), which is based on cross-correlation computed in the frequency domain. We show that the algorithm is equivalent to a direct method while asymptotically superior in terms of run-time. Furthermore, we propose a method for multimodal image alignment for transformation models with few degrees of freedom (e.g. rigid) based on the proposed CMIF-algorithm. We evaluate the efficacy of the proposed method on three distinct benchmark datasets, of aerial images, cytological images, and histological images, and we observe excellent success-rates (in recovering known rigid transformations), overall outperforming alternative methods, including local optimization of MI as well as several recent deep learning-based approaches. We also evaluate the run-times of a GPU implementation of the proposed algorithm and observe speed-ups from 100 to more than 10,000 times for realistic image sizes compared to a GPU implementation of a direct method. Code is shared as open-source at <>.


Cross-Sim-NGF: FFT-Based Global Rigid Multimodal Alignment of Image Volumes using Normalized Gradient Fields

Multimodal image alignment involves finding spatial correspondences betw...

Improving Multimodal Fusion with Hierarchical Mutual Information Maximization for Multimodal Sentiment Analysis

In multimodal sentiment analysis (MSA), the performance of a model highl...

Mutual information neural estimation for unsupervised multi-modal registration of brain images

Many applications in image-guided surgery and therapy require fast and r...

FSMI: Fast computation of Shannon Mutual Information for information-theoretic mapping

Exploration tasks are embedded in many robotics applications, such as se...

Mutual Information-driven Triple Interaction Network for Efficient Image Dehazing

Multi-stage architectures have exhibited efficacy in image dehazing, whi...

Dependence Maximizing Temporal Alignment via Squared-Loss Mutual Information

The goal of temporal alignment is to establish time correspondence betwe...

Please sign up or login with your details

Forgot password? Click here to reset