A new multi-objective bi-level programming model for energy and locality aware multi-job scheduling in cloud computing

•We propose a new multi-objective bi-level programming model based on MapReduce to improve energy efficiency of servers.•The relationship between performance and energy consumption of severs is taken into account in the proposed model.•Data locality can be adjusted dynamically according to current network state.•A new effective multi-objective genetic algorithm based on MOEA/D is proposed to solve the above large-scale scheduling model.

How to reduce power consumption of data centers has received worldwide attention. By combining the energy-aware data placement policy and locality-aware multi-job scheduling scheme, we propose a new multi-objective bi-level programming model based on MapReduce to improve the energy efficiency of servers. First, the variation of energy consumption with the performance of servers is taken into account; second, data locality can be adjusted dynamically according to current network state; last but not least, considering that task-scheduling strategies depend directly on data placement policies, we formulate the problem as an integer bi-level programming model. In order to solve the model efficiently, specific-design encoding and decoding methods are introduced. Based on these, a new effective multi-objective genetic algorithm based on MOEA/D is proposed. As there are usually tens of thousands of tasks to be scheduled in the cloud, this is a large-scale optimization problem and a local search operator is designed to accelerate convergent speed of the proposed algorithm. Finally, numerical experiments indicate the effectiveness of the proposed model and algorithm.