Case for dynamic deployment in a grid-based distributed query processor

Grid computing enables users to perform computationally expensive applications on distributed resources acquired dynamically. Users are allowed to combine structured data and analysis components into new applications from distributed sites into new applications. Distributed query processing offers an established way of structuring such computations, and well-known tools like OGSA-DAI and OGSA-DQP provide respectively a common interface to heterogeneous databases, and a way of exploiting distributed resources. Such significant benefits are however often undermined by high communication costs due to the need to move data between distributed resources. This paper describes an approach that addresses this by dynamically deploying query processing engines, analysis services and databases within virtual machines, on an internet-scale, so as to reduce communication costs. Results of internet-scale experiments are presented to demonstrate the performance benefits. Further, the use of dynamic deployment features based on requirements allows the creation of an ad-hoc runtime engine and thus opens up the possibility of creating a virtual marketplace for software and hardware resources.

► OGSA-DQP allows queries over globally distributed data and analysis services. ► Benefits are undermined by cost of data movement between resources. ► DynaSOAr framework allows on-demand deployment of services and virtual machines. ► OGSA-DQP is extended to allow dynamic deployment of data and query components. ► Internet-scale experiments demonstrate the performance benefits.