CSTP: A parallel data transfer protocol using cross-stream coding

Transferring very high quality digital objects over optical networks is critical in many scientific applications, such as video streaming/conferencing, remote rendering on tiled display walls, or 3D virtual reality. Current data transfer protocols rely on UDP as well as a variety of compression techniques. None of the existing transfer protocols, however, scale well to many parallel data connections. Existing parallel streaming protocols have limited synchronization mechanisms for multiple streams, and they are prone to be slowed down significantly if one stream experiences significant packet loss. In this paper, we propose a new parallel streaming protocol which can stream many parallel data streams over optical networks: CSTP, the Cross-Stream Transfer Protocol. It not only tolerates random UDP packet loss, but also aims to tolerate unevenly distributed packet loss patterns across multiple streams to achieve synchronized parallel streams with limited coding overhead. We simulated the approach, and the results show that CSTP can generate steady throughput with fluctuating data streams of different data loss patterns, and can transfer data in parallel at a higher speed than multiple independent UDP streams.

Research highlights
► A new parallel cross-stream coding mechanism is proposed.
► We designed a data transfer protocol named CSTP.
► CSTP tolerates unevenly distributed packet loss patterns across multiple streams.
► CSTP can be scaled to large number of streams in our further study.