Pixel convolutional neural network for multi-focus image fusion

This paper proposes a pixel-wise convolutional neural network (p-CNN) that can recognize the focused and defocused pixels in source images from its neighbourhood information for multi-focus image fusion. The proposed p-CNN can be thought of as a learned focus measure (FM) and provides more efficiency than conventional handcrafted FMs. To enable the p-CNN with the strong capability to discriminate focused and defocused pixels, a comprehensive training image set based on a public image database is created. Furthermore, by setting precise labels according to different focus levels and adding various defocus masks, the p-CNN can accurately measure the focus level of each pixel in source images in which the artefacts in the fused image can be efficiently avoided. We also propose a method to implement the p-CNN with a conventional image convolutional neural network (image-wised CNN), which is almost 25 times faster than directly using the p-CNN in multi-focus image fusion. Experimental results demonstrate that the proposed method is competitive with or even outperforms the state-of-the-art methods in terms of both subjective visual perception and objective evaluation metrics.