3D face modeling based on structure optimization and surface reconstruction with B-Spline

•3D face structure optimization: frontal face structure optimization is considered as a sparse and low rank decomposition, and depth estimation is introduced as nonlinear programming based on constraints of multi-substructure.•Face surface reconstruction is conducted with B-spline control grid deforming by 3D structure transformation.•3D reconstruction solution is based on wild photos, instead of calibration.

How to reconstruct 3D face model from wild photos is such a difficult issue that camera calibration is necessary and the images must be from video sequences. In this paper, a face reconstruction model with structure optimization is proposed to build 3D face surface with individual geometry and physical features reservation through wild face images directly and without camera calibration. Low rank and B-Spline are employed to estimate the aligned 2D structure, to calculate the depth information with SSIM, and to reconstruct the 3D face surface from control points and their space transformation. Furthermore, LFW and Bosphorus datasets, as well as Young-to-Aged samples, are introduced to verify the proposed approach and the experimental results demonstrate the feasibility and effectiveness even with different poses, expressions and age-variety.