Minimalist and High-Quality Panoramic Imaging with PSF-aware Transformers

by   Qi Jiang, et al.

High-quality panoramic images with a Field of View (FoV) of 360-degree are essential for contemporary panoramic computer vision tasks. However, conventional imaging systems come with sophisticated lens designs and heavy optical components. This disqualifies their usage in many mobile and wearable applications where thin and portable, minimalist imaging systems are desired. In this paper, we propose a Panoramic Computational Imaging Engine (PCIE) to address minimalist and high-quality panoramic imaging. With less than three spherical lenses, a Minimalist Panoramic Imaging Prototype (MPIP) is constructed based on the design of the Panoramic Annular Lens (PAL), but with low-quality imaging results due to aberrations and small image plane size. We propose two pipelines, i.e. Aberration Correction (AC) and Super-Resolution and Aberration Correction (SR AC), to solve the image quality problems of MPIP, with imaging sensors of small and large pixel size, respectively. To provide a universal network for the two pipelines, we leverage the information from the Point Spread Function (PSF) of the optical system and design a PSF-aware Aberration-image Recovery Transformer (PART), in which the self-attention calculation and feature extraction are guided via PSF-aware mechanisms. We train PART on synthetic image pairs from simulation and put forward the PALHQ dataset to fill the gap of real-world high-quality PAL images for low-level vision. A comprehensive variety of experiments on synthetic and real-world benchmarks demonstrates the impressive imaging results of PCIE and the effectiveness of plug-and-play PSF-aware mechanisms. We further deliver heuristic experimental findings for minimalist and high-quality panoramic imaging. Our dataset and code will be available at


page 1

page 3

page 4

page 6

page 9

page 10

page 11

page 14


Annular Computational Imaging: Capture Clear Panoramic Images through Simple Lens

Panoramic Annular Lens (PAL), composed of few lenses, has great potentia...

Universal and Flexible Optical Aberration Correction Using Deep-Prior Based Deconvolution

High quality imaging usually requires bulky and expensive lenses to comp...

Neural Invertible Variable-degree Optical Aberrations Correction

Optical aberrations of optical systems cause significant degradation of ...

Image Quality Is Not All You Want: Task-Driven Lens Design for Image Classification

In computer vision, it has long been taken for granted that high-quality...

Chromatic Aberration Recovery on Arbitrary Images

Digital imaging sensor technology has continued to outpace development i...

High-quality Panorama Stitching based on Asymmetric Bidirectional Optical Flow

In this paper, we propose a panorama stitching algorithm based on asymme...

Karhunen-Loève Data Imputation in High Contrast Imaging

Detection and characterization of extended structures is a crucial goal ...

Please sign up or login with your details

Forgot password? Click here to reset