Merging-ISP: Multi-Exposure High Dynamic Range Image Signal Processing

by   Prashant Chaudhari, et al.

The image signal processing pipeline (ISP) is a core element of digital cameras to capture high-quality displayable images from raw data. In high dynamic range (HDR) imaging, ISPs include steps like demosaicing of raw color filter array (CFA) data at different exposure times, alignment of the exposures, conversion to HDR domain, and exposure merging into an HDR image. Traditionally, such pipelines are built by cascading algorithms addressing the individual subtasks. However, cascaded designs suffer from error propagations since simply combining multiple processing steps is not necessarily optimal for the entire imaging task. This paper proposes a multi-exposure high dynamic range image signal processing pipeline (Merging-ISP) to jointly solve all subtasks for HDR imaging. Our pipeline is modeled by a deep neural network architecture. As such, it is end-to-end trainable, circumvents the use of complex, hand-crafted algorithms in its core, and mitigates error propagation. Merging-ISP enables direct reconstructions of HDR images from multiple differently exposed raw CFA images captured from dynamic scenes. We compared Merging-ISP against different alternative cascaded pipelines. End-to-end learning leads to HDR reconstructions of high perceptual quality and quantitatively outperforms competing ISPs by more than 1 dB in terms of PSNR.


page 2

page 4

page 6

page 7

page 8


FlexHDR: Modelling Alignment and Exposure Uncertainties for Flexible HDR Imaging

High dynamic range (HDR) imaging is of fundamental importance in modern ...

High Dynamic Range Imaging by Perceptual Logarithmic Exposure Merging

In this paper we emphasize a similarity between the Logarithmic-Type Ima...

Deep Snapshot HDR Imaging Using Multi-Exposure Color Filter Array

In this paper, we propose a deep snapshot high dynamic range (HDR) imagi...

Multi-Bracket High Dynamic Range Imaging with Event Cameras

Modern high dynamic range (HDR) imaging pipelines align and fuse multipl...

LenslessPiCam: A Hardware and Software Platform for Lensless Computational Imaging with a Raspberry Pi

Lensless imaging seeks to replace/remove the lens in a conventional imag...

Validation of image systems simulation technology using a Cornell Box

We describe and experimentally validate an end-to-end simulation of a di...

Rendering Nighttime Image Via Cascaded Color and Brightness Compensation

Image signal processing (ISP) is crucial for camera imaging, and neural ...

Please sign up or login with your details

Forgot password? Click here to reset