Normal maps vs. visible images: Comparing classifiers and combining modalities

This work investigates face recognition based on normal maps, and the performance improvement that can be obtained when exploiting it within a multimodal system, where a further independent module processes visible images. We first propose a technique to align two 3D models of a face by means of normal maps, which is very fast while providing an accuracy comparable to well-known and more general techniques such as Iterative Closest Point (ICP). Moreover, we propose a matching criterion based on a technique which exploits difference maps. It does not reduce the dimension of the feature space, but performs a weighted matching between two normal maps. In the second place, we explore the range of performances offered by different linear and nonlinear classifiers, when applied to the normal maps generated from the above aligned models. Such experiments highlight the added value of chromatic information contained in normal maps. We analyse a solid list of classifiers which were selected due to their historical reference value (e.g. Principal Component Analysis) or to their good performances in the bidimensional setting (Linear Discriminant Analysis, Partitioned Iterated Function Systems). Last but not least, we perform experiments to measure how different ways of combining normal maps and visible images can enhance the results obtained by the single recognition systems, given that specific characteristics of the images are taken into account. For these last experiments we only consider the classifier giving the best average results in the preceding ones, namely the PIFS-based one.