Naming persons in video: Using the weak supervision of textual stories

• Application of naming faces in soap series.
• Using unsupervised and weak supervised methods.
• Exploiting the multiple co-occurrences between faces and names.
• The F1 values between 80% and 86% without any human supervision.

In this paper, we focus on the problem of automated video annotation. We report on the application of naming faces in soap series by using the weak supervision of narrative texts that describe the events in the video and that are drafted by fans. Several unsupervised methods that operate without any manual labeling of exemplar faces, and methods that use a limited number of labeled exemplars are presented and evaluated. All methods exploit the multiple co-occurrences between faces shown in the video and names mentioned in the texts to compute the strength of the linking and reinforce this coupling by means of an Expectation Maximization algorithm. We show that the unsupervised methods attain competitive results without any prior human effort. The results show F1F1 values between 80% and 86% for the recognition of the face-name pairs without any human supervision. These figures rise only slightly when a number of faces were manually labeled beforehand. The study gives insights in the benefits and bottlenecks of the proposed approaches, and an error analysis results in guidelines for the choice of a certain technique.