Enhancing redundant network traffic elimination

Protocol-independent redundant traffic elimination (RTE) is a method to detect and remove redundant chunks of data from network-layer packets by using caching at both ends of a network link or path. In this paper, we propose a set of techniques to improve the effectiveness of packet-level RTE. In particular, we consider two bypass techniques, with one based on packet size, and the other based on content type. The bypass techniques apply at the front-end of the RTE pipeline. Within the RTE pipeline, we propose chunk overlap and oversampling as techniques to improve redundancy detection, while obviating the need for chunk expansion at the network endpoints. Finally, we propose savings-based cache management at the back-end of the RTE pipeline, as an improvement over FIFO-based cache management. We evaluate our techniques on full-payload packet-level traces from university and enterprise environments. Our results show that the proposed techniques improve detected redundancy by up to 50% for university traffic, and up to 54% for enterprise Web server traffic.