High performance solutions for big-data GWAS

• We consider mixed-models based genome-wide association studies of large scale.
• We address GWAS with only a single trait and with many traits.
• GWAS with arbitrarily large populations, and of arbitrarily many SNPs and traits are enabled.
• Distributed memory architectures, such as Cloud, clusters, and supercomputers are used.
• Scalability with respect to problem size and resources is demonstrated.

In order to associate complex traits with genetic polymorphisms, genome-wide association studies process huge datasets involving tens of thousands of individuals genotyped for millions of polymorphisms. When handling these datasets, which exceed the main memory of contemporary computers, one faces two distinct challenges: (1) millions of polymorphisms and thousands of phenotypes come at the cost of hundreds of gigabytes of data, which can only be kept in secondary storage; (2) the relatedness of the test population is represented by a relationship matrix, which, for large populations, can only fit in the combined main memory of a distributed architecture. In this paper, by using distributed resources such as Cloud or clusters, we address both challenges: the genotype and phenotype data is streamed from secondary storage using the double-buffering technique, while the relationship matrix is kept across the main memory of a distributed memory system. With the help of these solutions, we develop separate algorithms for studies involving only one or a multitude of traits. We show that these algorithms sustain high-performance and allow the analysis of enormous datasets.