Nonverbal social withdrawal in depression: Evidence from manual and automatic analyses

• We investigated the relation between nonverbal behavior and severity of depression.
• When symptoms were severe, participants showed less AU 12 and AU 15 and more AU 14.
• When symptoms were severe, participants' head motion was reduced in size and speed.
• The pattern of findings was highly consistent for automated and manual measurements.
• The findings support the hypothesis of nonverbal social withdrawal in depression.

The relationship between nonverbal behavior and severity of depression was investigated by following depressed participants over the course of treatment and video recording a series of clinical interviews. Facial expressions and head pose were analyzed from video using manual and automatic systems. Both systems were highly consistent for FACS action units (AUs) and showed similar effects for change over time in depression severity. When symptom severity was high, participants made fewer affiliative facial expressions (AUs 12 and 15) and more non-affiliative facial expressions (AU 14). Participants also exhibited diminished head motion (i.e., amplitude and velocity) when symptom severity was high. These results are consistent with the Social Withdrawal hypothesis: that depressed individuals use nonverbal behavior to maintain or increase interpersonal distance. As individuals recover, they send more signals indicating a willingness to affiliate. The finding that automatic facial expression analysis was both consistent with manual coding and revealed the same pattern of findings suggests that automatic facial expression analysis may be ready to relieve the burden of manual coding in behavioral and clinical science.

Figure optionsDownload high-quality image (220 K)Download as PowerPoint slide