Why wait? Let us start computing while the data is still on the wire

In this era of Big Data, computing useful and timely information from data is becoming increasingly complicated, particularly due to the ever increasing volumes of data that need to travel over the network to data centers to be stored and processed, all highly expensive operations in the long haul. This is a strong motivation to explore how to perform computing and analysis of data “on the wire”, i.e., while the data is still in transit. The nature of these computations include analysis, visualization, pattern recognition, and prediction on the streaming data. In this paper we present the idea of a framework capable of analyzing data in transit based on the principles of a service function chaining architecture. This framework can be deployed at any practical location within the network where computation on data flows is desirable. We further describe an all-virtual implementation of the framework as a worst-case scenario and present an early investigation of its capabilities with three examples - pattern recognition and data visualization on streaming Forex data, targeted advertising from clickstream data, and processing of monitoring data from solar sensors for maintenance decisions. Our results indicate that performing computations on live data flows to provide immediate perspective on the data is possible and attractive, but also that performance heavily depends on the amount and capabilities of the dedicated resources.