Evaluation of objective quality metrics for multidimensional video scalability

•Objective metrics are evaluated on quality assessment of multidimensional video scalability.•Both generic image/video metrics and scalability-targeting metrics are included.•Evaluation is systematically performed on three different data sets.•In-depth analysis in terms of performance and complexity of metrics is provided.•Overall, scalability-targeting metrics show best performance.

Multidimensional video scalability refers to the possibility that a video sequence can be adapted according to given conditions of video consumption by adjusting one or more of its features such as frame size, frame rate, and spatial quality. An important issue in implementing an adaptive video distribution scheme using scalability is how to maximize the quality of experience for the delivered contents, which raises a more fundamental issue, that is, how to estimate perceived quality of scalable video contents. This paper evaluates existing state-of-the-art objective quality metrics, including both generic image/video metrics and ones particularly developed for scalable videos, on the problem of quality assessment of multidimensional video scalability. It is shown that, on the whole, some recently developed metrics targeting scalability perform best. The results are thoroughly discussed in relation to the nature of the problem in comparison to what has been reported in existing studies for other problems.