Using provenance to efficiently improve metadata searching performance in storage systems

• We propose a high-performance cost-effective provenance based metadata-search system.
• The usage of relationship graph can reduce the overhead of metadata searching.
• The approach exploits the time consumption of constructing relationships.
• We leverage the files’ weights to improve the accuracy of searching metadata.

In cloud storage systems, more than 50% of requests are metadata operations, and thus the file system metadata search performance has become increasingly important to different users. With the rapid growth of storage system scales in volume, traditional full-size index trees cannot offer high-performance metadata search due to hierarchical indexing bottleneck. In order to alleviate the long latency and improve the quality-of-service (QoS) in cloud storage service, we proposed a novel provenance based metadata-search system, called PROMES. The metadata search in PROMES is split into three phases: (i) leveraging correlation-aware metadata index tree to identify several files as seeds, most of which can satisfy the query requests, (ii) using the seeds to find the remaining query results via relationship graph search, (iii) continuing to refine and rerank the whole search results, and sending the final results to users. PROMES has the salient features of high query accuracy and low latency, due to files’ tight and lightweight indexing in relationship graph coming from provenance’s analysis. Compared with state-of-the-art metadata searching schemes, PROMES demonstrates its efficiency and efficacy in terms of query accuracy and response latency.