SDN-aware federation of distributed data

• Software defined networking (SDN) has created unprecedented opportunities for efficient data transfers.
• We present an observable and controllable network model to approach data transfers from multiple sources.
• We develop and implement two algorithms that take advantage of the programmability of the network.
• We introduce an architecture for transparent & adaptive data transfers and enable full exploitation of research infrastructures.
• We assess the improvement of data transfer rates resulting from SDN-enabling Distributed File Access Services (DFAS).

The introduction of software defined networking (SDN) has created an opportunity for file access services to get a view of the underlying network and to further optimize large data transfers. This opportunity is still unexplored while the amount of data that needs to be transferred is growing. Data transfers are also becoming more frequent as a result of interdisciplinary collaborations and the nature of research infrastructures. To address the needs for larger and more frequent data transfers, we propose an approach which enables file access services to use SDN. We extend the file access services developed in our earlier work by including network resources in the provisioning for large data transfers. A novel SDN-aware file transfer mechanism is prototyped for improving the performance and reliability of large data transfers on research infrastructure equipped with programmable network switches. Our results show that I/O and data-intensive scientific workflows benefit from SDN-aware file access services.