Collaborative filtering with facial expressions for online video recommendation

• We propose a procedure for online video recommendation based on users facial expressions.
• The proposed procedure addresses a user’s preference changes often observed while the user watches a video.
• The proposed procedure addresses the new user problem.
• Experiment results show that the proposed procedure produces better prediction accuracy than other benchmark systems.

Online video recommender systems help users find videos suitable for their preferences. However, they have difficulty in identifying dynamic user preferences. In this study, we propose a new recommendation procedure using changes of users’ facial expressions captured every moment. Facial expressions portray the users’ actual emotions about videos. We can utilize them to discover dynamic user preferences. Further, because the proposed procedure does not rely on historical rating or purchase records, it properly addresses the new user problem, that is, the difficulty in recommending products to users whose past rating or purchase records are not available. To validate the recommendation procedure, we conducted experiments with footwear commercial videos. Experiment results show that the proposed procedure outperforms benchmark systems including a random recommendation, an average rating approach, and a typical collaborative filtering approach for recommendation to both new and existing users. From the results, we conclude that facial expressions are a viable element in recommendation.