Confidence-driven image co-matting

• We present a novel framework for a new task called co-matting.
• Co-matting extracts alpha mattes in images with deformed foreground against backgrounds.
• We apply optimization to jointly improve the alpha mattes extracted from aligned foreground.
• Each alpha matte is evaluated using a matting confidence metric learned from a training dataset.
• Experiments show that the framework achieves noticeably higher quality results on an image stack.

Single image matting, the task of estimating accurate foreground opacity from a given image, is a severely ill-posed and challenging problem. Inspired by recent advances in image co-segmentation, in this paper, we present a novel framework for a new task called co-matting, which aims to simultaneously extract alpha mattes in multiple images that contain slightly deformed instances of the same foreground object against different backgrounds. Our system first generates trimaps for input images using co-segmentation, and an initial alpha matte for each image using single image matting. Each alpha matte is then locally evaluated using a novel matting confidence metric learned from a training dataset. In the co-matting step, we first align the foreground object instances using appearance and geometric features, then apply a global optimization on all input images to jointly improve their alpha mattes, which allows high confidence local regions to guide their corresponding low confidence ones in other images to achieve more accurate mattes all together. Experimental results show that this co-matting framework can achieve noticeably higher quality results on an image stack than applying state-of-the-art single image matting techniques individually on each image.

Overview of our system. Given two input images and their corresponding trimaps (indicated by green lines) in (a), we first generate an initial matte for each image in (b) using single image matting. Region-wise matting confidence shown in (c) is evaluated for each initial matte, and inter-image unknown region registration shown in (d) is established. All the information is then incorporated in a global optimization procedure to derive refined mattes of both images shown in (e).Figure optionsDownload high-quality image (335 K)Download as PowerPoint slide