Face recognition from multiple stylistic sketches: Scenarios, datasets, and evaluation

Due to the great texture discrepancies between photos and sketches, matching face sketches against mug shot photos is a challenging yet important topic in face recognition community, with potential applications in law enforcement and security. Despite the great progress achieved in recent years, existing works mainly focus on face recognition from SINGLE sketch. However, in real-world applications it is more practical to obtain MULTIPLE stylistic sketches for recognizing the suspect, such as clue from multiple eyewitnesses, cooperation with multiple forensic artists, even drawn with different kinds of pencils. It is challenging due to the difficulty of taking multiple sketches into consideration in these scenes. To this end, we present a comprehensive investigation of face recognition from multiple stylistic sketches. Considering that there are two types of sketches available, i.e. hand-drawn sketch and composite sketch, we propose three specific scenarios: (1) recognition from multiple hand-drawn sketches, (2) recognition from both hand-drawn sketch and composite sketches, and (3) recognition from multiple composite sketches. In additional to the definitions of the proposed scenarios, we also provide evaluation protocols and benchmark performance on public available datasets. Through extensive experiments on benchmarks, plenty of challenges in proposed scenarios are finally discussed and there are many possible future directions worth further investigation. The main purpose of this paper is to open a first fundamental study on the topic of face recognition from multiple stylistic sketches. Possible high technical and superior methodologies are encouraged in the future. Related materials of this paper are publicly available at http://chunleipeng.com/FRMSketches.html.