Automatic flexible face replacement with no auxiliary data

•A novel framework free from human intervention providing automatic face replacement.•An energy function for more appealing final compositions with instant cloning.•A face contour optimization algorithm with cubic B-Spline curve fitting.•An algorithm for keeping the same hairstyle with guided inpainting and alpha matting.

In this paper, we propose a system for flexible face replacement in photos that does not require human intervention. The proposed replacement algorithm is able to efficiently paste one person׳s face into another׳s with different head poses and/or hairstyles. This work can enhance the quality of group photos that contain human faces, as they are seldom flawless. Previous methods require complex user interaction or auxiliary data such as a 3D head model library or a photo library, while our system does not. Given a photo, the faces are automatically detected and transformed into similar poses by fitting corresponding facial features. We then construct an energy function and optimize it for a better face contour between the transformed source face and the target one, followed by applying seamless cloning. Our system removes hair before face transfer and adds it back afterwards, maintaining the same hairstyles. We demonstrate that our system produces high-quality results without manual tuning of parameters.

Graphical abstractFigure optionsDownload full-size imageDownload high-quality image (83 K)Download as PowerPoint slide