Looking back at dense linear algebra software

• Growing gap of processor–memory communication affects linear algebra software.
• CPU/GPU hybridization creates challenging design combinations for legacy libraries.
• Loop restructuring methods include vectorization, chaining, blocking, and tiling.
• Scheduling techniques cover bulk-synchronous, SPMD, asynchronous, and dataflow.
• The effects of increased parallelism on numerical properties are briefly discussed.

Over the years, computational physics and chemistry served as an ongoing source of problems that demanded the ever increasing performance from hardware as well as the software that ran on top of it. Most of these problems could be translated into solutions for systems of linear equations: the very topic of numerical linear algebra. Seemingly then, a set of efficient linear solvers could be solving important scientific problems for years to come. We argue that dramatic changes in hardware designs precipitated by the shifting nature of the marketplace of computer hardware had a continuous effect on the software for numerical linear algebra. The extraction of high percentages of peak performance continues to require adaptation of software. If the past history of this adaptive nature of linear algebra software is any guide then the future theme will feature changes as well–changes aimed at harnessing the incredible advances of the evolving hardware infrastructure.