Unified framework for face sketch synthesis

•We designs a unified approach for face sketch synthesis.•We proposes a data-driven based high frequency information compensation strategy for face sketch synthesis.•The proposed method achieves superior performance in comparison to state-of-the-art face sketch synthesis methods.

Face sketch synthesis (FSS) has great significance to sketch based face retrieval or recognition and digital entertainment. Recently, great progress has been made in face sketch synthesis. Most state-of-the-art FSS methods work at patch level. However, these methods only consider either position constraint or global search when selecting candidate image patches. Furthermore, the common used weighting combination in these methods leads to the lost of reasonable high frequency details. We argue that all these factors are necessary for face sketch synthesis. To this end, we propose a simple yet effective unified approach considering position constraint, global search and high frequency compensation to infer pseudosketches from input test photos. Firstly, a nearest-neighbor search is conducted by combining both position constraint and global search. After obtaining the candidate image patches, a Markov network is applied to generate the pseudosketch. Secondly, the residue between an original sketch and the synthesized pseudosketch is modeled by the same Markov network to compensate the high frequency details. The effectiveness of the proposed method is demonstrated on the CUHK face sketch database by comparing with state-of-the-art FSS methods.