How to predict the global instantaneous feeling induced by a facial picture?

•A model of aesthetic quality assessment of frontal facial images based on low-level image statistics is proposed.•It is shown that combining 4 learning algorithms (SVM, ANN, RF, and GBT) enhances the prediction performance and robustness.•A model of likability evaluation based on high-level attributes is proposed.•Likability and aesthetic quality estimations are combined to select automatically good quality images depicting likable faces.

Picture selection is a time-consuming task for humans and a real challenge for machines, which have to retrieve complex and subjective information from image pixels. An automated system that infers human feelings from digital portraits would be of great help for profile picture selection, photo album creation or photo editing. In this work, two models of facial pictures evaluation are defined. The first one predicts the overall aesthetic quality of a facial image, and the second one answers the question “Among a set of facial pictures of a given person, on which picture does the person look like the most friendly?”. Aesthetic quality is evaluated by the computation of 15 features that encode low-level statistics in different image regions (face, eyes, and mouth). Relevant features are automatically selected by a feature ranking technique, and the outputs of 4 learning algorithms are fused in order to make a robust and accurate prediction of the image quality. Results are compared with recent works and the proposed algorithm obtains the best performance. The same pipeline is considered to evaluate the likability of a facial picture, with the difference that the estimation is based on high-level attributes such as gender, age, and smile. Performance of these attributes is compared with previous techniques that mostly rely on facial keypoint positions, and it is shown that it is possible to obtain likability predictions that are close to human perception. Finally, a combination of both models that selects a likable facial image of good quality for a given person is described.