PBFilter: A flash-based indexing scheme for embedded systems

NAND Flash has become the most widely used electronic stable storage technology for embedded systems. As on-board storage capacity increases, the need for efficient indexing techniques arises. Such techniques are very challenging to design due to a combination of NAND Flash constraints (e.g., block-erase-before-page-rewrite constraint and limited number of erase cycles) and embedded system constraints (e.g., tiny RAM and resource consumption predictability). Previous work adapted traditional indexing methods to cope with Flash constraints by deferring index updates using a log and batching them to decrease the number of rewrite operations in Flash memory. However, these methods were not designed with embedded system constraints in mind and do not address them properly. In this paper, we propose a different alternative for indexing Flash-resident data that specifically addresses the embedded context. This approach, called PBFilter, organizes the index structure in a purely sequential way. Key lookups are sped up thanks to two principles called Summarization and Partitioning. We instantiate these principles with data structures and algorithms based on Bloom Filters and show the effectiveness of this approach through a comprehensive analytical performance study. Extensions of PBFilter on range queries and multi-criteria queries are also discussed. The proposed technique is integrated into a full-fledged embedded DBMS engine. We describe the complete design of the DBMS engine to illustrate the feasibility of adopting PBFilter technique in a real system. Finally, we show some performance measurements of the prototype on top of a real hardware platform, in order to validate the new technique in a practical manner.

► We introduce new metrics to capture requirements of Flash-based embedded systems.
► We propose PBFilter which organizes the index structure in a pure sequential way.
► Lookups are sped up thanks to Summarization and Partitioning.
► PBFilter technique also supports range queries and multi-criteria queries.
► PBFilter is integrated in an embedded DBMS and its effectiveness is proved.