A holistic approach to build real-time stream processing system with GPU

• We propose a GPU-centric model for real-time GPU scheduling.
• A latency-driven GPU-based framework is designed for networking systems.
• A high-performance SRTP reverse proxy is built which utilizes GPU for real-time stream processing.
• The sufficient condition for real-time GPU scheduling is studied, and a mechanism is proposed for admission control.
• CPU limits system performance even the computation heavy jobs are offloaded to GPU.

Stream processing needs to process huge volume of data with strict deadline requirements. These applications generally consume large amount of network bandwidth and involve compute-intensive operations. Accelerating such operations with general purpose GPU has drawn a lot of attention from both academia and industry. However, GPU has not been applied to real-time stream processing due to its programming paradigm and unpredictable latency.In this paper, we study the problem of applying GPU to real-time processing and propose a holistic approach for building real-time stream processing system with GPU. Based on the proposed techniques, we build a GPU-accelerated SRTP reverse proxy that achieves more than 10Gbps overall throughput and guarantees low latency. Our work demonstrates that using GPU in high-speed real-time stream processing is both feasible and attractive.