Sparse Sampling Transformer with Uncertainty-Driven Ranking for Unified Removal of Raindrops and Rain Streaks

by   Sixiang Chen, et al.

In the real world, image degradations caused by rain often exhibit a combination of rain streaks and raindrops, thereby increasing the challenges of recovering the underlying clean image. Note that the rain streaks and raindrops have diverse shapes, sizes, and locations in the captured image, and thus modeling the correlation relationship between irregular degradations caused by rain artifacts is a necessary prerequisite for image deraining. This paper aims to present an efficient and flexible mechanism to learn and model degradation relationships in a global view, thereby achieving a unified removal of intricate rain scenes. To do so, we propose a Sparse Sampling Transformer based on Uncertainty-Driven Ranking, dubbed UDR-S2Former. Compared to previous methods, our UDR-S2Former has three merits. First, it can adaptively sample relevant image degradation information to model underlying degradation relationships. Second, explicit application of the uncertainty-driven ranking strategy can facilitate the network to attend to degradation features and understand the reconstruction process. Finally, experimental results show that our UDR-S2Former clearly outperforms state-of-the-art methods for all benchmarks.


page 1

page 2

page 3

page 4

page 5

page 6

page 8

page 9


Devignet: High-Resolution Vignetting Removal via a Dual Aggregated Fusion Transformer With Adaptive Channel Expansion

Vignetting commonly occurs as a degradation in images resulting from fac...

MSP-Former: Multi-Scale Projection Transformer for Single Image Desnowing

Image restoration of snow scenes in severe weather is a difficult task. ...

Towards Real-time High-Definition Image Snow Removal: Efficient Pyramid Network with Asymmetrical Encoder-decoder Architecture

In winter scenes, the degradation of images taken under snow can be pret...

Uncertainty-Driven Multi-Scale Feature Fusion Network for Real-time Image Deraining

Visual-based measurement systems are frequently affected by rainy weathe...

Learning Parallax Transformer Network for Stereo Image JPEG Artifacts Removal

Under stereo settings, the performance of image JPEG artifacts removal c...

Simultaneous Fidelity and Regularization Learning for Image Restoration

Most existing non-blind restoration methods are based on the assumption ...

Image Super-Resolution using Efficient Striped Window Transformer

Recently, transformer-based methods have made impressive progress in sin...

Please sign up or login with your details

Forgot password? Click here to reset