Does “lie to me” lie to you? An evaluation of facial clues to high-stakes deception

• A precedent for high-stakes deception detection using facial clues.
• Database consisting of real-world high-stakes deception videos is collected.
• Methods analyzing facial expressions in unconstrained environments are proposed.

During a forensic interview, high-stakes deception is very prevalent notwithstanding the heavy consequences that may result. Studies have shown that most untrained people cannot perform well in discerning liars and truth-tellers.Some psychological studies have stated that certain facial actions are more difficult to inhibit if the associated facial expressions are genuine. Similarly, these facial expressions are equally difficult to fake. This has cast light on the possibility that deception could be detected by analyzing these facial actions. However, to the best knowledge of the authors, there is no computer vision research that has attempted to discriminate high-stakes deception from truth using facial expressions. Therefore, this paper aims to test the validity of facial clues to deception detection in high-stakes situations using computer vision approaches.We also note that only a limited number of the existing databases have been collected specifically for deception detection studies and none of them were obtained from real-world situations. In this paper we present a video database of actual high-stakes situations, which we have created using YouTube.We have adopted 2D appearance-based methods as the methodology to characterize the 3D facial features. Instead of building a 3D head model as is the current trend, we have extracted invariant 2D features that are related to the 3D characteristic from nine separate facial regions by using dynamic facial analysis: eye blink, eyebrow motion, wrinkle occurrence and mouth motion. Then these cues are integrated to form a facial behavior pattern vector. A Random Forest was trained using the collected database and applied to classify the facial patterns into deceptive and truthful categories.Despite the many uncontrolled factors (illumination, head pose and facial occlusion) contained in the videos in our database, we have achieved an accuracy of 76.92% when discriminating liars from truth-tellers using methods based on both micro-expressions and “normal” facial expressions. The results have shown that using facial clues for automated lie detection is very promising from the point of view of practice.