Scalable and elastic total order in content-based publish/subscribe systems

Total order as a messaging guarantee scheme ensures that events sent to a set of subscribers are delivered by these subscribers in the same order. It has become increasingly important in content-based publish/subscribe (pub/sub) systems. Due to the large-scale live content and the churn workloads in the big data era, current emergency applications present a new challenge: how to provide a scalable and elastic total order service in content-based pub/sub systems. Most existing total order approaches cannot adapt to the churn workloads, and generate high delivery latency in the face of high arrival rate of live content. To this end, we propose a scalable and elastic total order service, called SETO, for content-based pub/sub systems in the cloud computing environment. SETO uses a two-layer pub/sub framework to decouple the event matching service and the total order service. In this framework, events are forwarded to their interested subscribers by multiple parallel servers. Through a preceding graph building technique, non-conflicting events in the same server are allowed to be delivered simultaneously, which greatly reduces the delivery latency. The performance-aware provisioning technique in SETO elastically adjusts the scale of servers to adapt to the churn workloads. To evaluate the performance of SETO, tens of servers and thousands of subscribers are deployed in our CloudStack testbed. Extensive experiments confirm that SETO can linearly reduce the delivery latency with the growth of servers, adaptively adjust the scale of servers in less than 5 s, and significantly outperforms the state-of-the-art approaches under diverse parameter settings.