CSDL Home I IPDPS 2011 25th IEEE International Parallel & Distributed Processing Symposium (IPDPS 2011)

Subscribe

Anchorage, AK

May 16, 2011 to May 20, 2011

ISBN: 978-1-61284-372-8

pp: 48-58

DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/IPDPS.2011.15

ABSTRACT

We describe an implementation of the Communication-Avoiding QR (CAQR) factorization that runs entirely on a single graphics processor (GPU). We show that the reduction in memory traffic provided by CAQR allows us to outperform existing parallel GPU implementations of QR for a large class of tall-skinny matrices. Other GPU implementations of QR handle panel factorizations by either sending the work to a general-purpose processor or using entirely bandwidth-bound operations, incurring data transfer overheads. In contrast, our QR is done entirely on the GPU using compute-bound kernels, meaning performance is good regardless of the width of the matrix. As a result, we outperform CULA, a parallel linear algebra library for GPUs by up to 17x for tall-skinny matrices and Intel's Math Kernel Library (MKL) by up to 12x. We also discuss stationary video background subtraction as a motivating application. We apply a recent statistical approach, which requires many iterations of computing the singular value decomposition of a tall-skinny matrix. Using CAQR as a first step to getting the singular value decomposition, we are able to get the answer 3x faster than if we use a traditional bandwidth-bound GPU QR factorization tuned specifically for that matrix size, and 30x faster than if we use Intel's Math Kernel Library (MKL) singular value decomposition routine on a multicore CPU.

CITATION

Michael Anderson,
Grey Ballard,
James Demmel,
Kurt Keutzer,
"Communication-Avoiding QR Decomposition for GPUs",

*IPDPS*, 2011, 25th IEEE International Parallel & Distributed Processing Symposium (IPDPS 2011), 25th IEEE International Parallel & Distributed Processing Symposium (IPDPS 2011) 2011, pp. 48-58, doi:10.1109/IPDPS.2011.15