Residual Contrastive Learning for Joint Demosaicking and Denoising

by   Nanqing Dong, et al.

The breakthrough of contrastive learning (CL) has fueled the recent success of self-supervised learning (SSL) in high-level vision tasks on RGB images. However, CL is still ill-defined for low-level vision tasks, such as joint demosaicking and denoising (JDD), in the RAW domain. To bridge this methodological gap, we present a novel CL approach on RAW images, residual contrastive learning (RCL), which aims to learn meaningful representations for JDD. Our work is built on the assumption that noise contained in each RAW image is signal-dependent, thus two crops from the same RAW image should have more similar noise distribution than two crops from different RAW images. We use residuals as a discriminative feature and the earth mover's distance to measure the distribution divergence for the contrastive loss. To evaluate the proposed CL strategy, we simulate a series of unsupervised JDD experiments with large-scale data corrupted by synthetic signal-dependent noise, where we set a new benchmark for unsupervised JDD tasks with unknown (random) noise variance. Our empirical study not only validates that CL can be applied on distributions (c.f. features), but also exposes the lack of robustness of previous non-ML and SSL JDD methods when the statistics of the noise are unknown, thus providing some further insight into signal-dependent noise problems.


page 9

page 14

page 15


Contrastive Learning for Low-light Raw Denoising

Image/video denoising in low-light scenes is an extremely challenging pr...

Self-Supervised Reversed Image Signal Processing via Reference-Guided Dynamic Parameter Selection

Unprocessed sensor outputs (RAW images) potentially improve both low-lev...

Dirty Pixels: Optimizing Image Classification Architectures for Raw Sensor Data

Real-world sensors suffer from noise, blur, and other imperfections that...

NoiseTransfer: Image Noise Generation with Contrastive Embeddings

Deep image denoising networks have achieved impressive success with the ...

A Contrastive Learning Approach to Auroral Identification and Classification

Unsupervised learning algorithms are beginning to achieve accuracies com...

Optimizing the Noise in Self-Supervised Learning: from Importance Sampling to Noise-Contrastive Estimation

Self-supervised learning is an increasingly popular approach to unsuperv...

Model-Based Image Signal Processors via Learnable Dictionaries

Digital cameras transform sensor RAW readings into RGB images by means o...

Please sign up or login with your details

Forgot password? Click here to reset