Improving resource efficiency in data centers using reputation-based resource selection

Today data centers are consuming a lot of energy but not very efficiently. Much of energy is wasted. There are several types of energy waste at different levels including infrastructure-, machine- and system-level waste. The former two levels have been improved significantly in the last few years, however, few efforts have been put on the last level, especially the resource waste caused by failures in a data center. In this paper, we attack the problem proactively by leveraging a reputation-based resource selection scheme to reduce the number of resubmissions of tasks, resulting from the failure during the course of their execution. To capture the characteristics of resources, we introduce Opera, an OPEn ReputAtion model. Opera characterizes itself with two important novelties: a vector representation of the reputation and the just-in-time feature that represents the real-time system status, which, to our knowledge, has never been considered in conventional reputation systems. To demonstrate the effectiveness of Opera, we have integrated the Opera trust model into the scheduler of Hadoop. The experimental results showed that Opera enables the scheduler to select appropriate nodes which helped to reduce not only the number of re-executed tasks but also the execution time of Hadoop's jobs under the presence of failures and heavy workload up to 59% and 32%, respectively. This improvements, in turn, can improve the energy efficiency of the whole system and the network by 16.17% and 53.32% for the sort application respectively.