A novel video fusion framework using surfacelet transform

A novel video fusion framework based on the three-dimensional surfacelet transform (3D-ST) is proposed in this paper. Different from the traditional individual-frame based video fusion methods, the proposed framework fused multi-frame images of input videos as a whole rather than frame by frame independently with the 3D-ST. Furthermore, under the proposed framework, two ST-based video fusion algorithms are proposed. In the first algorithm, no special treatment is performed on the temporal motion information in input videos, and only a spatial-temporal region energy-based fusion rule is employed. While in the second algorithm, a modified z-score based motion detection is performed to distinguish the temporal motion information from the spatial geometry information, and then a motion-based fusion rule is present. Experimental results demonstrate that, with the motion selectivity of the 3D-ST, existing static image fusion rules can be extended to video fusion under the proposed framework. Both of the two proposed fusion algorithms significantly outperform some traditional individual-frame based and motion-based methods in spatial-temporal information extraction as well as in temporal stability and consistency. In addition, the second proposed algorithm is with high computation efficiency and can be applied to real-time video fusion.