Analysis of ratings on trust inference in open environments

Ratings (also known as recommendations) provide an efficient and effective way to build trust relationship in the human society, by making use of the information from others rather than exclusively relying on one’s own direct observations. However, it is uncertain that whether the rating can play the same positive effect in the open computing environment because of differences between the computing world and human society. We envisage that there are two kinds of uncertainties: the uncertainty resulting from rating aggregation algorithms and the uncertainty resulting from other algorithm-independent design factors, which are coined as algorithm uncertainty and factor uncertainty in this paper. The algorithm uncertainty is related to such a problem: are the complex aggregating algorithms necessary? The factor uncertainty refers to how the performance of ratings is affected by all kinds of factors, including trust model design related factors and trust model design independent factors. In this paper, we take an initial step to answer these two uncertainties. First, we study the effect of all factors based on a simple averaging rating algorithm in terms of several proposed performance metrics. Then we compare different rating aggregation algorithms in the same context and platform, focusing on several relevant metrics. The simulation results show that ratings are not always as helpful as what we expected, especially when the system is facing malicious raters and highly dynamic peer behaviors. In certain circumstances, the simple average aggregation algorithm performs better than the complex ones, especially when there are considerable number of bad raters in the system. Considering the system dynamics, the cost of the algorithm design, and the system overhead, we argue that it is not worth putting too much energy on the design of complex rating aggregation schemes for trust inference in open computing environments.