Exploiting Mixed Precision in Numerical Linear Algebra

Support for floating point arithmetic in multiple precisions is becoming increasingly common in emerging architectures. Mixed precision capabilities are already included in any machines on the TOP500 list and are expected to be a crucial hardware feature in coming exascale machines. From a computational scientists perspective, our goal is to determine how and where we can exploit mixed precision computation in our codes. This requires both an understanding of performance characteristics as well as an understanding of the numerical behavior of algorithms in finite precision arithmetic.

In this talk, we discuss recent and ongoing efforts in this area. In particular, we present and analyze a general algorithm for solving nxn nonsingular linear systems Ax = b based on iterative refinement in three precisions. From this, we develop GMRES-IR, a three-precision GMRES-based iterative refinement scheme that works for even ill-conditioned systems. We discuss performance results on modern GPU architectures and present the HPL-AI benchmark, based on our mixed precision iterative refinement algorithms. The world's top supercomputers already exceed exaflop performance on HPL-AI, achieving over 4x higher performance than on the standard HPL benchmark.