Pyracanthus: A scalable solution for DHT-independent content-based publish/subscribe data networks

We study the problems associated with building large-scale, content-based, publish/subscribe networks. In particular, we focus on network-independent solutions, in an effort to isolate and bypass the difficult problems of ensuring the desirable underlying-network properties of scalability, fault tolerance, high performance, and adaptability to user and network dynamics. For this reason, we assume a popular overlay network infrastructure, based on Distributed Hash Tables, over which our pub/sub system will be built. We present Pyracanthus which embodies and can be configured using a suite of novel alternative components, which are categorized according to their ability to handle numerical and/or string attribute predicates and according to whether they are stateful/stateless. We present in detail each component consisting of algorithms for in-network indexing and storing of numerical and/or string-attribute subscription predicates, algorithms for matching events to relevant subscriptions, and any additional state. We analytically and experimentally analyze the performance of the alternative algorithms, exposing and quantifying related trade-offs.

Research Highlights►Pyracanthus is a DHT-independent content-based publish/subscribe system. ►Pyracanthus supports complex numerical- and string-attribute subscription predicates. ►Pyracanthus embodies stateful and stateless approaches for processing events and subscriptions.