Additive models and separable pooling, a new look at structural similarity

Objective quality metrics predict perceived quality of image signals computationally and can: (i) benchmark and monitor compression and processing algorithms and (ii) optimise their performance for a given application (content, bandwidth, packet loss…). Structural similarity, represented by the well known SSIM index is a framework for objective assessment of image quality well known for its relative simplicity and robustness. Despite its practical appeal, SSIM's performance level, measured as agreement with subjective quality scores, lags more complex state-of-the-art metrics. We present a new look into structural similarity that uses an additive model and a spatial pooling approach that decouples individual structural comparisons and utilises the quality driven aggregation paradigm. We apply this new approach to both baseline intensity SSIM and gradient SSIM (GSSIM) frameworks and show, through extensive evaluation on four publicly available subjective datasets that it provides considerably more ordered (linear) relationship between objective and subjective quality for a variety of input conditions. We demonstrate that newly formulated structural similarity metrics using this approach are capable of equal or even better performance than more complex state-of-the-art objective metrics in the process lending support to the theory that humans base their opinion on the worst sections of the observed signal.