QoS-guaranteed resource provisioning for cloud-based MapReduce in dynamical environments

- A new problem in cloud-based MapReduce computations is identified and formulated.
- An event-driven framework is proposed to tackle the problem.
- Two new resource scaling algorithms used in the framework are proposed.
- The proposed event-driven framework and new resource scaling algorithms are evaluated.

How to guarantee Quality of Service (QoS) with the minimum resource cost has become a new problem in cloud-based computation-intensive MapReduce computations. However, the new problem is challenging as the cloud-based MapReduce environment is dynamically changing. As a result, a static resource allocation scheme is not suitable for cloud-based MapReduce as resources may be under-provisioning, which leads to violations in the QoS of cloud-based MapReduce, or over-provisioning, which increases unnecessary resource cost. This paper abstracts the problem of cloud-based computation-intensive MapReduce computations as a dynamical optimization problem, and proposes an event-driven resource provisioning framework to solve that problem. This new event-driven framework has been compared with existing popular static resource provisioning frameworks and periodic resource provisioning frameworks by experiments. The experimental results have shown that the new event-driven resource provisioning framework not only guarantees the QoS of those MapReduce computations, but also reduces the running cost of MapReduce computations by 4.87−21.61 percent compared with those static resource provisioning frameworks, and by 1.70−16.12 percent compared with those periodic resource provisioning frameworks.