A Multi-Staged Blackboard Query Optimization Framework for World-Spanning Distributed Database Resources

With the advent of distributed computing, particularly since the emergence of Grids, Clouds and other Service Oriented Computing paradigms, the querying of huge datasets of distributed databases or data repositories on a global scale has become a challenging research question. Currently, beside various other topics, two major concerns in this research area have to be addressed: data access & integration and query execution planning. Our research effort addresses the second issue, namely the query optimization of distributed database queries. Hereby we consider a variety of different heterogeneous and homogeneous infrastructures, parallel algorithms, and huge datasets, which span across several virtual organizations (VOs) with usually no centralized authority. This paper introduces a novel heuristic framework for the optimization of query execution plans (QEP) on a world-wide scale. Our work is based on a multi-staged blackboard mechanism to determine which available data, resources and operations have to be considered to perform a query optimally. Moreover, an evaluation scenario proves our findings that even small changes in the selection of e.g. sort operations for a query execution tree (QET) lead to significant performance improvements.