Video fusion performance evaluation based on structural similarity and human visual perception

In order to evaluate different video fusion algorithms in temporal stability and consistency as well as in spatial information transfer, a novel objective video fusion quality metric is proposed with the structural similarity (SSIM) index and the perception characteristics of human visual system (HVS) in this paper. Firstly, for each frame, two sub-indices, i.e., the spatial fusion quality index and the temporal fusion quality index, are defined by the weighted local SSIM indices. Secondly, for the current frame, an individual-frame fusion quality measure is obtained by integrating the above two sub-indices. Lastly, the proposed global video fusion metric is constructed as the weighted average of all the individual-frame fusion quality measures. In addition, according to the perception characteristics of HVS, some local and global spatial–temporal information, such as local variance, pixel movement, global contrast, background motion and so on, is employed to define the weights in the proposed metric. Several sets of experimental results demonstrate that the proposed metric can evaluate different video fusion algorithms accurately, and the evaluation results coincide with the subjective results well.

► The proposed metric evaluates a video fusion method in spatial and temporal aspects.
► With the SSIM index, the spatial–temporal texture information is employed.
► With the perception characteristics of HVS, local and global weights are defined.
► The metric is robust to noise.
► The metric's prediction result is consistent with the subjective assessment.