Scientific computing with Google App Engine

We investigate the potential of Google App Engine (GAE) for scientific computing. We designed a generic master–slave framework that enables implementation and integration of new algorithms by instantiating one interface and two abstract classes. Applications are dynamically scheduled, executed, and monitored by the framework on the Google infrastructure, including a fault tolerant resubmission mechanism upon certain failures. We describe the implementation of a parallel rank sort algorithm using our framework and analyse the speedup, the execution overheads, and the cost of executing it on the Google Cloud infrastructure. Experimental results show that important speedup can be obtained from using GAE, especially for embarrassingly parallel Monte Carlo simulations. The two main obstacles that hinder the performance of applications on GAE are the resource quotas upon free use (especially the 30 s sequential execution time limit) and the middleware overheads. Cost-wise, GAE offers opportunities for cheaper computation due to the CPU cycle-based payment granularity, as opposed to the hourly billing intervals of other providers such as Amazon EC2.

► We designed a master–slave framework for writing parallel applications on GAE.
► GAE offers 2–3 times slower sequential performance than a 2.2 GHz Opteron.
► Monte Carlo simulations can benefit from GAE parallelisation.
► Middleware overheads and resource quotas upon free use lower the speedup.
► GAE is cheaper than Amazon EC2 for jobs shorter than one hour.