Measuring stream processing systems adaptability under dynamic workloads

Data streaming belongs to the Big Data ecosystem, which generates high-frequency data streams featuring time-varying characteristics that challenge the traditional stream processing systems capacities. To deal with this problem, many self-adaptive stream processing systems have been proposed. Despite the evolution of self-adaptive systems, there is still a lack of standardized benchmarking systems to enable scientists to evaluate the autonomic capacities of their solutions. In this work, we propose an index called AI-SPS inspired by the human cerebral auto-regulation process. The index quantifies the capacity of an adaptive stream processing systems to self-adapt in the presence of highly dynamic workloads. An index of this nature will help the scientific community generate fair comparisons among literature with the aim of creating better solutions. We validate our proposal by evaluating the adaptive behavior of two state of the art self-adaptive stream processing systems. Tests were performed using real traffic datasets adapted specifically to stress the processing system. Results show that the proposed index quantifies the adaptation capacity of self-adaptive stream processing systems effectively.