A new multi-objective bi-level programming model for energy and locality aware multi-job scheduling in cloud computing

• We propose a new multi-objective bi-level programming model based on MapReduce to improve energy efficiency of servers.
• The relationship between performance and energy consumption of severs is taken into account in the proposed model.
• Data locality can be adjusted dynamically according to current network state.
• A new effective multi-objective genetic algorithm based on MOEA/D is proposed to solve the above large-scale scheduling model.

How to reduce power consumption of data centers has received worldwide attention. By combining the energy-aware data placement policy and locality-aware multi-job scheduling scheme, we propose a new multi-objective bi-level programming model based on MapReduce to improve the energy efficiency of servers. First, the variation of energy consumption with the performance of servers is taken into account; second, data locality can be adjusted dynamically according to current network state; last but not least, considering that task-scheduling strategies depend directly on data placement policies, we formulate the problem as an integer bi-level programming model. In order to solve the model efficiently, specific-design encoding and decoding methods are introduced. Based on these, a new effective multi-objective genetic algorithm based on MOEA/D is proposed. As there are usually tens of thousands of tasks to be scheduled in the cloud, this is a large-scale optimization problem and a local search operator is designed to accelerate convergent speed of the proposed algorithm. Finally, numerical experiments indicate the effectiveness of the proposed model and algorithm.