Scalable task parallelism for NUMA: A uniform abstraction for coordinated scheduling and memory management
2016 International Conference on Parallel Architecture and Compilation Techniques (PACT) (2016)
Sept. 11, 2016 to Sept. 15, 2016
Andi Drebes , The University of Manchester, School of Computer Science, Oxford Road, M13 9PL, United Kingdom
Antoniu Pop , The University of Manchester, School of Computer Science, Oxford Road, M13 9PL, United Kingdom
Karine Heydemann , Sorbonne Universités, UPMC Univ Paris 06, CNRS, UMR 7606, LIP6, 4, Place Jussieu, F-75252 Cedex 05, France
Albert Cohen , INRIA and École Normale Supérieure, 45 rue d'Ulm, F-75005 Paris, France
Nathalie Drach , Sorbonne Universités, UPMC Univ Paris 06, CNRS, UMR 7606, LIP6, 4, Place Jussieu, F-75252 Cedex 05, France
Dynamic task-parallel programming models are popular on shared-memory systems, promising enhanced scalability, load balancing and locality. Yet these promises are undermined by non-uniform memory access (NUMA). We show that using NUMA-aware task and data placement, it is possible to preserve the uniform abstraction of both computing and memory resources for task-parallel programming models while achieving high data locality. Our data placement scheme guarantees that all accesses to task output data target the local memory of the accessing core. The complementary task placement heuristic improves the locality of task input data on a best effort basis. Our algorithms take advantage of data-flow style task parallelism, where the privatization of task data enhances scalability by eliminating false dependences and enabling fine-grained dynamic control over data placement. The algorithms are fully automatic, application-independent, performance-portable across NUMA machines, and adapt to dynamic changes. Placement decisions use information about inter-task data dependences readily available in the run-time system and placement information from the operating system. We achieve 94% of local memory accesses on a 192-core system with 24 NUMA nodes, up to 5× higher performance than NUMA-aware hierarchical work-stealing, and even 5.6× compared to static interleaved allocation. Finally, we show that state-of-the-art dynamic page migration by the operating system cannot catch up with frequent affinity changes between cores and data and thus fails to accelerate task-parallel applications.
Programming, Resource management, Parallel processing, Heuristic algorithms, Data models, Dynamic scheduling, Computational modeling
A. Drebes, A. Pop, K. Heydemann, A. Cohen and N. Drach, "Scalable task parallelism for NUMA: A uniform abstraction for coordinated scheduling and memory management," 2016 International Conference on Parallel Architecture and Compilation Techniques (PACT), Haifa, Israel, 2016, pp. 125-137.