Hyperspectral and Multispectral Image Fusion Using the Conditional Denoising Diffusion Probabilistic Model

by   Shuaikai Shi, et al.

Hyperspectral images (HSI) have a large amount of spectral information reflecting the characteristics of matter, while their spatial resolution is low due to the limitations of imaging technology. Complementary to this are multispectral images (MSI), e.g., RGB images, with high spatial resolution but insufficient spectral bands. Hyperspectral and multispectral image fusion is a technique for acquiring ideal images that have both high spatial and high spectral resolution cost-effectively. Many existing HSI and MSI fusion algorithms rely on known imaging degradation models, which are often not available in practice. In this paper, we propose a deep fusion method based on the conditional denoising diffusion probabilistic model, called DDPM-Fus. Specifically, the DDPM-Fus contains the forward diffusion process which gradually adds Gaussian noise to the high spatial resolution HSI (HrHSI) and another reverse denoising process which learns to predict the desired HrHSI from its noisy version conditioning on the corresponding high spatial resolution MSI (HrMSI) and low spatial resolution HSI (LrHSI). Once the training is completes, the proposed DDPM-Fus implements the reverse process on the test HrMSI and LrHSI to generate the fused HrHSI. Experiments conducted on one indoor and two remote sensing datasets show the superiority of the proposed model when compared with other advanced deep learningbased fusion methods. The codes of this work will be opensourced at this address: https://github.com/shuaikaishi/DDPMFus for reproducibility.


page 1

page 3

page 4

page 6

page 7

page 8

page 9


Unsupervised Hyperspectral and Multispectral Images Fusion Based on the Cycle Consistency

Hyperspectral images (HSI) with abundant spectral information reflected ...

SpectralDiff: Hyperspectral Image Classification with Spectral-Spatial Diffusion Models

Hyperspectral image (HSI) classification is an important topic in the fi...

FastHyMix: Fast and Parameter-free Hyperspectral Image Mixed Noise Removal

Hyperspectral imaging with high spectral resolution plays an important r...

Learning deep multiresolution representations for pansharpening

Retaining spatial characteristics of panchromatic image and spectral inf...

DiffUCD:Unsupervised Hyperspectral Image Change Detection with Semantic Correlation Diffusion Model

Hyperspectral image change detection (HSI-CD) has emerged as a crucial r...

Hyperspectral Pansharpening Based on Improved Deep Image Prior and Residual Reconstruction

Hyperspectral pansharpening aims to synthesize a low-resolution hyperspe...

Unsupervised Data Fusion with Deeper Perspective: A Novel Multi-Sensor Deep Clustering Algorithm

The ever-growing developments in technology to capture different types o...

Please sign up or login with your details

Forgot password? Click here to reset