Teaching Parallel Computing with Python: Strategies and Lessons

Neftali Watkinson, Aniket Shivam, Alexandru Nicolau, and Alexander V. Veidenbaum, all of the Department of Computer Science, University of California, Irvine, dove into the importance of teaching parallel computing to computer science students and the challenges of using languages with high levels of abstraction, such as Python. They used a lesson plan that uses data dependence analysis and loop transformations to teach parallel computing to Python programming students.

A 3-Session Series

The researchers’ work included three sessions to guide the students through their Python journey.

Lecture 1

This lecture introduced NumPy, a Python package for scientific computing, and its optimization possibilities. The concept of parallel computing was then introduced, including concurrency and vector instructions (SIMD), along with Python’s Global Interpreter Lock (GIL), which blocked automatic parallel execution and multithreading of Python code. Students learned about the threading and multiprocessing libraries, and that multiprocessing was the best option for exposing local parallelism on multi-core architectures due to the GIL.

Want More Tech News? Subscribe to ComputingEdge Newsletter Today!

Lecture 2

The second lecture covered data dependencies between statements and focused on three types of dependence relations. Three loop transformation techniques were also introduced, including loop distribution, loop interchange, and loop chunking. The lecture demonstrated an easily parallelizable function that computed the Sobel image filter for edge detection using the multiprocessing package for implementing parallelism in Python.

Practical Session

This session provided students with hands-on experience doing loop transformations in the computer lab. Examples and templates for implementing multiprocessing were provided, and students were asked to apply each transformation to a loop nest and try to run multiprocessing to measure performance improvement using Python’s time module. The goal was to successfully transform and optimize the code using multiprocessing to achieve the best possible performance.

Next Steps

The researchers plan to circumvent the drawbacks of using Python for parallel computing by implementing libraries that the students can import into their projects. These will allow for automatic dependence analysis that they can use to verify their transformations and for a more streamlined implementation of multiprocessing. This will enable instructors to focus on the high-level aspects of parallel computing without having to delve too deep into technical details.

Check out the full paper today to learn more.