Achieving high job execution reliability using underutilized resources in a computational economy

In distributed shared resource environments, such as grids or recent clouds, one of the major challenges is how to meet users’ QoS requirements and rationally distribute resources at the same time. Computational economy has long been studied as an effective solution to address such resource-allocation problems. However, price alone has limitations in controlling supply-and-demand in computational markets. In particular, even though less reliable desktop PCs are the dominant resource in computation grids, they are often underutilized, regardless of their price, because they do not exhibit qualities required by typical scientific and business applications. To address this problem, we propose Highly Available Job Execution Service (HA-JES) that fosters the balanced resource consumption by dynamically virtualizing resources to meet QoS requirements from users. In particular, in HA-JES, jobs are replicated in a market-driven efficient way; underutilized and therefore cheap resources are exploited to build a virtualized high quality resource and hence facilitate balanced resource usage. Our evaluation results show that HA-JES benefits all actors in the market in terms of resource utilization, market capacity, and market stability.

► An economy-based brokering service, making balance between supply and demand. ► A grid simulator, based on a large scale monitoring study of real grid resources. ► Extensive experiments and analysis of market behavior, such as price stability.