Learning a referenceless stereopair quality engine with deep nonnegativity constrained sparse autoencoder

This paper proposes a no-reference (NR)/referenceless quality evaluation method for stereoscopic three-dimensional (S3D) images based on deep nonnegativity constrained sparse autoencoder (DNCSAE). To address the quality issue of stereopairs whose perceived quality is not only determined by the individual left and right image qualities but also their interactions, a three-column DNCSAE framework is customized with individual DNCSAE module coping with the left image, the right image, and the cyclopean image, respectively. In the proposed framework, each individual DNCSAE module shares the same network architecture consisting of multiple stacked NCSAE layers and one Softmax regression layer at the end. The contribution of our model is that hierarchical feature evolution and nonlinear feature mapping are jointly optimized in a unified and perceptual-aware deep network (DNCSAE), which well resembles several important visual properties, i.e., hierarchy, sparsity, and non-negativity. To be more specific, for each DNCSAE, by taking a set of handcrafted natural scene statistic (NSS) features as inputs in the visible layer, the features in hidden layers are successively evolved to deeper levels producing increasingly discriminative quality-aware features (QAFs). Then, QAFs in the last NCSAE layer are summarized to their corresponding quality score by Softmax regression. Finally, three individual yet complementary quality scores estimated by each DNCSAE model are combined based on a Bayesian framework to obtain an overall 3D quality score. Experiments on three benchmark databases demonstrate the superiority of our method in terms of both prediction accuracy and generalization capability.