Load-aware inter-co-processor parallelism in database query processing

• Contribute heuristics to enhance performance by exploiting inter-device parallelism
• Heuristics consider load and speed on (co-)processors.
• Extensive evaluation on four use cases: aggregation, selection, sort, and join
• Assess the performance of best heuristic for systems with multiple co-processors
• Discuss how operator-stream-based scheduling can be used in a query processor

For a decade, the database community has been exploring graphics processing units and other co-processors to accelerate query processing. While the developed algorithms often outperform their CPU counterparts, it is not beneficial to keep processing devices idle while overutilizing others. Therefore, an approach is needed that efficiently distributes a workload on available (co-)processors while providing accurate performance estimates for the query optimizer. In this paper, we contribute heuristics that optimize query processing for response time and throughput simultaneously via inter-device parallelism. Our empirical evaluation reveals that the new approach achieves speedups up to 1.85 compared to state-of-the-art approaches while preserving accurate performance estimations. In a further series of experiments, we evaluate our approach on two new use cases: joining and sorting. Furthermore, we use a simulation to assess the performance of our approach for systems with multiple co-processors and derive some general rules that impact performance in those systems.