Exploring mixed and multi-precision SpMV for GPUs

Sparse Matrix-Vector Multiplication (SpMV) is one of the key memory-bound kernels commonly used in industrial and scientific applications. To improve its data movement and benefit from higher compute rates, there are several efforts to utilize mixed precision for SpMV. Most of the prior-art focus on performing the SpMV in different precisions throughout the entire application, such as an iterative solver (e.g., CG, GMRES) where certain steps can be done with lower precisions. More recently, methods of using mixed-precision within a single SpMV has been of consideration. For instance, one can decide precision for each non-zero value in the matrix, and then split the input into multiple matrices with their respective precisions; or, decide the precision for each block in a given block-diagonal matrix format. In this work, we are interested in this more fine-grained approach of mixedprecision SpMV. To this extent, we extend an existing entry-wise precision based approach by deciding precision for each row, motivated by the granularity of parallelism on a GPU where groups of threads process rows in row-compressed sparse matrices. We propose mixed-precision CSR storage methods with row permutations and describe its greater load-balance compared to the existing method. We also consider a multi-precision case where single and double-precision copies of the matrix are stored priorly, and further extend our mixed-precision SpMV approach to comply with it. To evaluate our methods, we apply them in two real-life applications: a multiprecision Jacobi method and a multi-precision Cardiac modeling application. We further extend our mixed-precision methodology to be used with the ELLPACK-R format. We demonstrate the effectiveness of the proposed SpMV methods on an extensive dataset of real-valued large sparse matrices from the SuiteSparse Matrix Collection using an NVIDIA V100 GPU.