Analysis of Video Quality Datasets via Design of Minimalistic Video Quality Models

by   Wei Sun, et al.

Blind video quality assessment (BVQA) plays an indispensable role in monitoring and improving the end-users' viewing experience in various real-world video-enabled media applications. As an experimental field, the improvements of BVQA models have been measured primarily on a few human-rated VQA datasets. Thus, it is crucial to gain a better understanding of existing VQA datasets in order to properly evaluate the current progress in BVQA. Towards this goal, we conduct a first-of-its-kind computational analysis of VQA datasets via designing minimalistic BVQA models. By minimalistic, we restrict our family of BVQA models to build only upon basic blocks: a video preprocessor (for aggressive spatiotemporal downsampling), a spatial quality analyzer, an optional temporal quality analyzer, and a quality regressor, all with the simplest possible instantiations. By comparing the quality prediction performance of different model variants on eight VQA datasets with realistic distortions, we find that nearly all datasets suffer from the easy dataset problem of varying severity, some of which even admit blind image quality assessment (BIQA) solutions. We additionally justify our claims by contrasting our model generalizability on these VQA datasets, and by ablating a dizzying set of BVQA design choices related to the basic building blocks. Our results cast doubt on the current progress in BVQA, and meanwhile shed light on good practices of constructing next-generation VQA datasets and models.


A Comparative Evaluation of Temporal Pooling Methods for Blind Video Quality Assessment

Many objective video quality assessment (VQA) algorithms include a key s...

Audio-Visual Quality Assessment for User Generated Content: Database and Method

With the explosive increase of User Generated Content (UGC), UGC video q...

Visual Mechanisms Inspired Efficient Transformers for Image and Video Quality Assessment

Visual (image, video) quality assessments can be modelled by visual feat...

VMAF And Variants: Towards A Unified VQA

Video quality assessment (VQA) is now a fastgrowing subject, beginning t...

FAVER: Blind Quality Prediction of Variable Frame Rate Videos

Video quality assessment (VQA) remains an important and challenging prob...

Neighbourhood Representative Sampling for Efficient End-to-end Video Quality Assessment

The increased resolution of real-world videos presents a dilemma between...

FAST-VQA: Efficient End-to-end Video Quality Assessment with Fragment Sampling

Current deep video quality assessment (VQA) methods are usually with hig...

Please sign up or login with your details

Forgot password? Click here to reset