Towards a federated Metropolitan Area Grid environment: The SCoPE network-aware infrastructure

Grid computing offers us an effective approach, infrastructure and trend for coordinated resource sharing, problem solving and service integration into dynamic, multi-institutional, virtual organizations often spanning several distant sites in a large urban or regional area. In this paper, we discuss opportunities and challenges in the Grid resource management infrastructure and network control plane design, critical to the provision of network-assisted extensible Grid services on the metropolitan scale. Such services can empower a real high performance distributed computing system built on optical transport networks, administered within a single domain and offering plenty of cheap bandwidth to e-science applications. This approach makes the transport infrastructure the main enabling factor of a novel Grid vision, the “Metropolitan Area Grid” (MAG), aiming at unifying many geographically distributed federated computational and storage resources into a common “virtual site” abstraction, so that they can cooperate as if they were in the same Server Farm and Local Area Network. Simply stated, the MAG concept aims to make applications running on our metro Grid infrastructure aware of their complete computational and networking environment and capabilities, and able to make dynamic, adaptive and optimized use of heterogeneous network infrastructures connecting various high-end resources. As a proof of concept, we realized within the SCoPE High Performance Computing environment the prototype of a basic MAG architecture by implementing a novel centralized network resource management service supporting a flexible Grid-application interface and several effective network resource reservation facilities.