Design adaptive task allocation scheduler to improve MapReduce performance in heterogeneous clouds

This paper conducted a thorough research on one of the critical technologies in cloud computing, the MapReduce programming model. Some of the past research results showed that one of the methods for enhancing MapReduce performance is the execution through allocating identical tasks to each cloud node. However, such allocation is not applicable for the environment of a heterogeneous cloud. Due to the different computing power and system resources between the nodes, such uniform distribution of tasks will lower the performance between nodes, and hence this paper makes improvements on the original speculative execution method of Hadoop (called Hadoop Speculative) and LATE Scheduler by proposing a new scheduling scheme known as Adaptive Task Allocation Scheduler (ATAS). The ATAS adopts more accurate methods to determine the response time and backup tasks that affect the system, which is expected to enhance the success ratio of backup tasks and thereby effectively increase the system’s ability to respond. We conducted three different simulation experiments, in which execution time of the LATE Scheduler and ATAS could be reduced by a maximum of 12% and 30%, increased the average tasks throughput by a maximum of 13% and 33%, and reduced the average tasks latency by a maximum of 25% and 36% compared to Hadoop Speculative; the ATAS can effectively enhance the processing performance of MapReduce to be compared with LATE Scheduler in a heterogeneous cloud environment.