A self-organized resource provisioning for cloud block storage

Modern cloud deployments rely on multiple layers of microservices and a layer of one or more persistent data store systems. The latter has always received a lot of attention as it is the slowest part of the request of an online service. Its unpredictable performance due to workload interference, performance degradation, lack of control, and network partitions has made the cloud less attractive for some services, especially for the applications that require guaranteed throughput, latency, elasticity, and high availability. Motivated by those facts, we propose the Block Software Defined Storage (BSDS), a framework to automate the orchestration, deployment, and management of a storage system by separating the data layer from control. BSDS employs an SLA-aware (Service Level Agreement) and self-learning scheduler that treats the backend media as a black box and abstracts the resource allocation from the state of the storage cluster. Our proposed framework is a step toward the emergence of an intelligent cloud infrastructure that can adapt to unexpected failures, network traffic fluctuations, workload interference perturbations or even changes in the underlying infrastructure. We perform extensive simulations with workloads deduced from an enterprise block storage system and showcase that the proposed scheduler can provide guaranteed Quality of Service (QoS). We finally demonstrate a practical deployment of BSDS on the OpenStack platform.