Analyzing Hadoop power consumption and impact on application QoS

•Power consumption characterization of Hadoop Clouds (with a social media use case).•Study of the QoS related to power consumption (in terms of processing time).•Experimentation on two different Cloud infrastructures (single node–multi node).•OpenNebula based private Cloud environments.

Energy efficiency is often identified as one of the key reasons for migrating to Cloud environments. It is stated that a data center hosting the Cloud environment is likely to achieve greater energy efficiency (at a reduced cost) compared to a local deployment. With increasing energy prices, it is also estimated that a large percentage of operational costs within a Cloud environment can be attributed to energy. In this work, we investigate and measure energy consumption of a number of virtual machines running the Hadoop system, over an OpenNebula Cloud. Our workload is based on sentiment analysis undertaken over Twitter messages. Our objective is to understand the tradeoff between energy efficiency and performance for such a workload. From our results we generalize and speculate on how such an analysis could be used as a basis to establish a Service Level Agreement (SLA) with a Cloud provider—especially where there is likely to be a high level of variability (both in performance and energy use) over multiple runs of the same application (at different times). Among the service level objectives that might be included in a SLA, Quality of Service (QoS) related metrics (i.e., latency) are one of the most challenging to support. This work provides some insight on the relationship between power consumption and QoS related metrics, describing how a combined consideration of these two metrics could be supported for a particular workload.