HIPCL: From CUDA to OpenCL Execution
by Pekka Jääskeläinen and Michal Babej, Customized Parallel Computing group, Tampere University, Finland

Heterogeneous-Compute Interface for Portability (HIP) is a runtime API and a conversion tool to help make CUDA programs more portable. It was originally contributed by AMD to the open source community with the intention to ease the effort of making CUDA applications also work on AMD’s ROCm platform.

While AMD and NVIDIA share the vast majority of the discrete GPU market, it is useful to make this “CUDA portability enhancement route” available to an even wider set of platforms. Since the Khronos OpenCL standard remains the most widely adopted cross-platform heterogeneous programming API/middleware, it is interesting to study whether HIP could be ported on top of it, expanding its scope potentially to all OpenCL supported devices. Our CPC group has worked on this project, known as HIPCL, for some time; it’s now published and available in Github.

During the development of HIPCL, CPC aims to identify features lacking in OpenCL and other challenges when transitioning from CUDA applications to OpenCL-supported platforms. The main challenge we have identified so far is a lack of an Intermediate Representation (IR) that is widely adopted by multiple OpenCL implementations and that also has solid open source infrastructure available. While the main program of a heterogeneous application is typically compiled to a native instruction-set binary of the host CPU, using an IR target and just-in-time compilation for the kernel parts helps maintain portability of the application executable across multiple heterogeneous devices.

Standard Portable Intermediate Representation (SPIR) versions 1.2 and 2.0 are the first IRs defined by Khronos to be used with OpenCL. Because it’s based on the LLVM compiler framework IR, they have relatively good support in the upstream LLVM compiler, but lack wide adoption in vendor implementations: Nvidia does not seem to support SPIR at all. The AMD APP SDK CPU-only implementation supports SPIR, but AMD’s GPU implementations don’t support any SPIR version. The older versions of the Intel OpenCL implementations support SPIR, while the newer versions (proprietary CPU runtime and the new NEO runtime) support only SPIR-V. Overall, the current focus of the OpenCL community is now on SPIR-V. SPIR-V supports both OpenCL (since version 2.1) and also a now popular graphics and compute API, Vulkan. It also has LLVM to SPIR-V conversion tools available as open source. SPIR-V thus seemed the most “future proof” choice for the HIPCL’s fat binary output’s IR.

In the first release of HIPCL we focused on testing the output on the Portable Computing Language (POCL) open source OpenCL implementation framework as well as Intel OpenCL SDK “NEO” for GPUs, both of which had adequate SPIR-V support for our test cases. We plan to expand the support to other OpenCL platforms in the future releases as the SPIR-V support in the OpenCL implementations keep maturing.

The first release of HIPCL is a proof-of-concept, but is already useful for end-users. It can run most of the CUDA examples in the HIP repository and the list of supported CUDA applications will grow steadily as we add new features. That being said, we naturally welcome contributions from the community in the form of pull requests and issue reports with CUDA applications that do not port. However, when reporting issues on non-porting CUDA applications, please first make sure that your application works with the upstream HIP to ROCm path, since our primary focus is on getting the OpenCL support on par with it. The current status of HIPCL will be kept up to date in the README section on the front page of the Github repository. If you plan to work on a new feature, please let us know first by opening a feature request issue to avoid us working on it at the same time.

On behalf of the CPC group, we want to thank key funding sources that are helping us realize a more open and diverse heterogeneous computing ecosystem: The HSA Foundation, Academy of Finland (funding decision 297548) and ECSEL JU project FitOptiVis (project number 783162).

For additional information, contact us via pekka.jaaskelainen@tuni.fi.