A transparent approach to performance analysis and comparison of infrastructure as a service providers

This study proposes a transparent approach to performance analysis and comparison of Infrastructure as a Service (IaaS) providers. Using established benchmarks, we focused on obtaining deeper insights into the performance of the public cloud infrastructure as reflected by the CPU, memory and disk I/O subsystems. We conducted experiments on a real public cloud environment and demonstrated how prospective cloud users can use our transparent methodology to discover how well virtualized public cloud resources meet their application requirements. Our transparent approach is unique in the sense that it helps prospective cloud users to decipher public cloud benchmarking data and appraise the performance of public cloud services relative to their application's performance objectives. Furthermore, we show workload correlations to the benchmarks using three real-life analytics applications. Our empirical results show significant performance differences for comparable instances on the public cloud, underscoring the need for thoughtful and transparent IaaS provider selection.