Are Visual Recognition Models Robust to Image Compression?

by   João Maria Janeiro, et al.

Reducing the data footprint of visual content via image compression is essential to reduce storage requirements, but also to reduce the bandwidth and latency requirements for transmission. In particular, the use of compressed images allows for faster transfer of data, and faster response times for visual recognition in edge devices that rely on cloud-based services. In this paper, we first analyze the impact of image compression using traditional codecs, as well as recent state-of-the-art neural compression approaches, on three visual recognition tasks: image classification, object detection, and semantic segmentation. We consider a wide range of compression levels, ranging from 0.1 to 2 bits-per-pixel (bpp). We find that for all three tasks, the recognition ability is significantly impacted when using strong compression. For example, for segmentation mIoU is reduced from 44.5 to 30.5 mIoU when compressing to 0.1 bpp using the best compression model we evaluated. Second, we test to what extent this performance drop can be ascribed to a loss of relevant information in the compressed image, or to a lack of generalization of visual recognition models to images with compression artefacts. We find that to a large extent the performance loss is due to the latter: by finetuning the recognition models on compressed training images, most of the performance loss is recovered. For example, bringing segmentation accuracy back up to 42 mIoU, i.e. recovering 82 of the original drop in accuracy.


page 1

page 3

page 4


Learning-based Compression for Material and Texture Recognition

Learning-based image compression was shown to achieve a competitive perf...

Exploiting Richness of Learned Compressed Representation of Images for Semantic Segmentation

Autonomous vehicles and Advanced Driving Assistance Systems (ADAS) have ...

Identity Preserving Loss for Learned Image Compression

Deep learning model inference on embedded devices is challenging due to ...

Distributed Learning and Inference with Compressed Images

Modern computer vision requires processing large amounts of data, both w...

To What Extent Does Downsampling, Compression, and Data Scarcity Impact Renal Image Analysis?

The condition of the Glomeruli, or filter sacks, in renal Direct Immunof...

Data-Efficient Sequence-Based Visual Place Recognition with Highly Compressed JPEG Images

Visual Place Recognition (VPR) is a fundamental task that allows a robot...

Data Efficient Visual Place Recognition Using Extremely JPEG-Compressed Images

Visual Place Recognition (VPR) is the ability of a robotic platform to c...

Please sign up or login with your details

Forgot password? Click here to reset