Discovering Limitations of Image Quality Assessments with Noised Deep Learning Image Sets

by   Wei Dai, et al.

Image quality is important, and it can affect overall performance in image processing and computer vision as well as for numerous other reasons. Image quality assessment (IQA) is consequently a vital task in different applications from aerial photography interpretation to object detection to medical image analysis. In previous research, the BRISQUE algorithm and the PSNR algorithm were evaluated with high resolution ( 512*384 pixels per image), but relatively small image sets (4,744 images). However, scientists have not evaluated IQA algorithms on low resolution (32*32 pixels per image), multi-perturbation, big image sets (for example, 60,000 different images not counting their perturbations). This study explores these two IQA algorithms through experimental investigation. We first chose two deep learning image sets, CIFAR-10 and MNIST. Then, we added 68 perturbations that add noise to the images in specific sequences and noise intensities. In addition, we tracked the performance outputs of the two IQA algorithms with singly and multiply noised images. After quantitatively analyzing experimental results, we report the limitations of the two IQAs with these noised CIFAR-10 and MNIST image sets. We also explain three potential root causes for performance degradation. These findings point out weaknesses of the two IQA algorithms. The research results provide guidance to scientists and engineers developing accurate, robust IQA algorithms. In addition to supporting future scientific research and industrial projects, all source codes are shared on the website:


page 1

page 2

page 3

page 4

page 6

page 7


PyTorch Image Quality: Metrics for Image Quality Assessment

Image Quality Assessment (IQA) metrics are widely used to quantitatively...

Shifted Windows Transformers for Medical Image Quality Assessment

To maintain a standard in a medical imaging study, images should have ne...

Benchmarking Robustness of Deep Learning Classifiers Using Two-Factor Perturbation

This paper adds to the fundamental body of work on benchmarking the robu...

CatSIM: A Categorical Image Similarity Metric

We introduce CatSIM, a new similarity metric for binary and multinary tw...

IQUAFLOW: A new framework to measure image quality

IQUAFLOW is a new image quality framework that provides a set of tools t...

Quantifying Visual Image Quality: A Bayesian View

Image quality assessment (IQA) models aim to establish a quantitative re...

Performance Analysis of Cone Detection Algorithms

Many algorithms have been proposed to help clinicians evaluate cone dens...

Please sign up or login with your details

Forgot password? Click here to reset