Long-term performance of collaborative filtering based recommenders in temporally evolving systems

Recommender systems benefit people at every moment in their daily life. Considerable attentions have been drawn by performance in one-step recommendation and static user-item network, while the performances of recommenders on temporally evolving systems remain unclear. To address this issue, this paper first describes an online commercial system by using a bipartite network. On this network, a recommendation-based evolution method is proposed to simulate the temporal dynamics between a recommender and its users. Then the long-term performance of three state-of-the-art collaborative filtering (CF)-based recommenders, i.e., the user-based CF (UCF), item-based CF (ICF) and latent factor-based model (LFM), is evaluated on the generated temporally evolving networks. Experimental results on two large, real datasets generated by industrial applications demonstrate that 1) optimization-based CF models like the LFM enjoy their high-prediction accuracy in one-step recommendation; and 2) entity relationship-based CF models like the ICF benefit the recommendation diversity, as well as the system health on a temporally evolving network. It turns out that in a temporally evolving system, an efficient recommender should consider both the one-step and long-term effects to generate satisfactory recommendations. Thus, it is necessary to adopt heterogeneous models, e.g., trade-off between optimization-based model and entity relationship-based model, in real systems to grasp various users' behavior patterns to improve their experiences.