A robust real-time algorithm for facial shape recovery from a single image containing cast shadow under general, unknown lighting

It is difficult to find a non-iterative solution for statistical shape from shading (SFS) for a single image containing cast shadow under unknown, arbitrary light conditions. This is because it is not trivial to find a mapping from a convex cone in the image space to a point in the depth space. To find a non-iterative solution for the statistical SFS, which has not been done yet to our knowledge, we show that it is possible to approximate a cone of images (illumination cone) to a polytope by a nonlinear function, which is a transform from Cartesian coordinates to hyperspherical coordinates. The images of a subject form a convex cone in the image space, and it can be better to use the direction of an image in hyperspherical coordinates as an input feature for the mapping rather than the image itself. The maximum error occurs on one of the vertices of the polytope if we choose the objective function to be the squared error of a linear function in the hyperspherical space. Hence, we can solve the least square problem using the cost only for the vertices. In deriving the solution, we use the generalized Rayleigh quotient, canonical correlation analysis (CCA), and prior information on input images, to reduce the solution space and improve 3-D reconstruction performance. In experiments, the proposed scheme performs robustly under light variations, and is fast enough for real-time applications.

► We propose a robust facial shape recovery algorithm for a single image input.
► An input image is a frontal face under a general, unknown lighting with the cast shadow.
► A transformed face image in hyperspherical coordinates is used as an input feature.
► The method performs robustly even if there is a heavy shadow.
► The method has no iterative procedure and sufficient for real-time applications.