Efficient global correlation measures for a collaborative filtering dataset

Recommender systems based on collaborative filtering (CF) rely on datasets containing users' taste preferences for various items. Accuracy of various prediction approaches depends on the amount of similarity between users and items in a dataset. As a heuristic estimate of this data quality aspect, which could serve as an indicator of the prediction ability, we define the Global User Correlation Measure (GUCM) and the Global Item Correlation Measure (GICM) of a dataset containing known user-item ratings. The proposed measures range from 0 to 1 and describe the quality of the dataset regarding the user-user and item-item similarities: a higher measure indicates more similar pairs and better prediction ability. The experiments show a correlation between the proposed measures and the accuracy of standard prediction models. The measures can be used to quickly estimate whether a dataset is suitable for collaborative filtering and whether we can expect high prediction accuracy of user-based or item-based CF approaches.