Optimal metadata replications and request balancing strategy on cloud data centers

• In this paper we address an important problem pertaining to Metadata Server Clusters.
• We propose strategies to handle metadata replication and a load balancing strategy to minimize mean response time.
• We present rigorous theoretical analysis of the strategies proposed and present a practical algorithm.
• We compare our findings with most commonly used strategies that use hashing functions and show a significant gain.
• We demonstrate a trade-off relationship between makespan and the monetary cost.

In large-scale cloud data centers, metadata accesses will very likely become a severe performance bottleneck as metadata-based transactions account for over 50% of all file system operations. Clusters of Metadata Servers (MDS) that provide metadata searching service can improve the system performance significantly. For a data stored in cloud data centers, there may be several MDS storing the metadata replicas. Therefore, when a data request arrives, it has many potential metadata paths, one of which shall be chosen to obtain the best performance. In this paper, we attempt to determine the number of MDS that each data object in the system shall have and the request rates that each MDS shall serve, in order to achieve the minimum mean response time (MRT) of all the metadata requests. The target optimal constrained function has been formulated and a novel metadata request balancing algorithm based on request arrival rates has been proposed, which can find near-optimal solutions by a theoretical proof. In our experiments, we compare our algorithm with widely used hashing functions that have 0, 1, 2, 3 replicas, respectively. We validate our findings via simulations with respect to several influencing factors and prove that our proposed strategy is scalable, flexible and efficient for the real-life applications. Some interesting perspectives of the work are also presented at the end of this paper.