Assessing the Quality-of-Experience of Adaptive Bitrate Video Streaming

by   Zhengfang Duanmu, et al.

The diversity of video delivery pipeline poses a grand challenge to the evaluation of adaptive bitrate (ABR) streaming algorithms and objective quality-of-experience (QoE) models. Here we introduce so-far the largest subject-rated database of its kind, namely WaterlooSQoE-IV, consisting of 1350 adaptive streaming videos created from diverse source contents, video encoders, network traces, ABR algorithms, and viewing devices. We collect human opinions for each video with a series of carefully designed subjective experiments. Subsequent data analysis and testing/comparison of ABR algorithms and QoE models using the database lead to a series of novel observations and interesting findings, in terms of the effectiveness of subjective experiment methodologies, the interactions between user experience and source content, viewing device and encoder type, the heterogeneities in the bias and preference of user experiences, the behaviors of ABR algorithms, and the performance of objective QoE models. Most importantly, our results suggest that a better objective QoE model, or a better understanding of human perceptual experience and behaviour, is the most dominating factor in improving the performance of ABR algorithms, as opposed to advanced optimization frameworks, machine learning strategies or bandwidth predictors, where a majority of ABR research has been focused on in the past decade. On the other hand, our performance evaluation of 11 QoE models shows only a moderate correlation between state-of-the-art QoE models and subjective ratings, implying rooms for improvement in both QoE modeling and ABR algorithms. The database is made publicly available at: < zduanmu/waterloosqoe4/>.


page 1

page 3


A Knowledge-Driven Quality-of-Experience Model for Adaptive Streaming Videos

The fundamental conflict between the enormous space of adaptive streamin...

Subjective and Objective Analysis of Streamed Gaming Videos

The rising popularity of online User-Generated-Content (UGC) in the form...

Study on the Assessment of the Quality of Experience of Streaming Video

Dynamic adaptive streaming over HTTP provides the work of most multimedi...

A Survey on 360-Degree Video: Coding, Quality of Experience and Streaming

The commercialization of Virtual Reality (VR) headsets has made immersiv...

Optimizing Adaptive Video Streaming with Human Feedback

Quality of Experience (QoE)-driven adaptive bitrate (ABR) algorithms are...

A Subjective Dataset for Multi-Screen Video Streaming Applications

In modern-era video streaming systems, videos are streamed and displayed...

Top-down and bottom-up approaches to video Quality of Experience studies; overview and proposal of a new model

Modern video streaming services require quality assurance of the present...

Please sign up or login with your details

Forgot password? Click here to reset