The implementation of the finite-volume dynamical core in the community atmosphere model

A distributed memory message-passing parallel implementation of a finite-volume discretization of the primitive equations in the community atmosphere model (CAM) 3.0 is presented. These 3-D equations can be decoupled into a set of 2-D equations by the introduction of a floating vertical coordinate, resulting in considerable potential parallelism. Subsequent analysis of the data dependencies—in particular those arising from the polar singularity of the latitude-longitude coordinate system—suggests that two separate domain decompositions should be employed, each tailored for a different part of the model. The implementation requires that data be periodically redistributed between these two decompositions. Furthermore, data from nearest neighbors are kept in halo regions, which are updated between iterations. These data movements are optimized through one-sided communication primitives and multithreading. The resulting algorithm is shown to scale to very large machine configurations, even for relatively coarse resolutions.