Neural Global Shutter: Learn to Restore Video from a Rolling Shutter Camera with Global Reset Feature

by   Zhixiang Wang, et al.

Most computer vision systems assume distortion-free images as inputs. The widely used rolling-shutter (RS) image sensors, however, suffer from geometric distortion when the camera and object undergo motion during capture. Extensive researches have been conducted on correcting RS distortions. However, most of the existing work relies heavily on the prior assumptions of scenes or motions. Besides, the motion estimation steps are either oversimplified or computationally inefficient due to the heavy flow warping, limiting their applicability. In this paper, we investigate using rolling shutter with a global reset feature (RSGR) to restore clean global shutter (GS) videos. This feature enables us to turn the rectification problem into a deblur-like one, getting rid of inaccurate and costly explicit motion estimation. First, we build an optic system that captures paired RSGR/GS videos. Second, we develop a novel algorithm incorporating spatial and temporal designs to correct the spatial-varying RSGR distortion. Third, we demonstrate that existing image-to-image translation algorithms can recover clean GS videos from distorted RSGR inputs, yet our algorithm achieves the best performance with the specific designs. Our rendered results are not only visually appealing but also beneficial to downstream tasks. Compared to the state-of-the-art RS solution, our RSGR solution is superior in both effectiveness and efficiency. Considering it is easy to realize without changing the hardware, we believe our RSGR solution can potentially replace the RS solution in taking distortion-free videos with low noise and low budget.


page 2

page 3

page 6

page 7

page 8


Rolling Shutter Inversion: Bring Rolling Shutter Images to High Framerate Global Shutter Video

A single rolling-shutter (RS) image may be viewed as a row-wise combinat...

Bringing Rolling Shutter Images Alive with Dual Reversed Distortion

Rolling shutter (RS) distortion can be interpreted as the result of pick...

Rolling-Shutter-Aware Differential SfM and Image Rectification

In this paper, we develop a modified differential Structure from Motion ...

Combining Internal and External Constraints for Unrolling Shutter in Videos

Videos obtained by rolling-shutter (RS) cameras result in spatially-dist...

Self-calibration-based Approach to Critical Motion Sequences of Rolling-shutter Structure from Motion

In this paper we consider critical motion sequences (CMSs) of rolling-sh...

Learning Adaptive Warping for Real-World Rolling Shutter Correction

This paper proposes the first real-world rolling shutter (RS) correction...

Deep Homography Estimation in Dynamic Surgical Scenes for Laparoscopic Camera Motion Extraction

Current laparoscopic camera motion automation relies on rule-based appro...

Please sign up or login with your details

Forgot password? Click here to reset