GPU Programming Languages

Page maintainer: Alessio Sclocco @isazi

Learning Resources

• Carpentries GPU Programming course
  • Lesson material
• Introduction to CUDA C
  • Slides
• Introduction to OpenACC
  • Slides
• Introduction to HIP Programming
  • Video
• SYCL Introduction and Best Practices
  • Video
• CSCS GPU Programming with Julia
  • Course recordings

Documentation

• CUDA
  • C programming guide
  • Runtime API
  • Driver API
  • Fortran programming guide
• HIP
  • Kernel language syntax
  • Runtime API
• SYCL
  • Specification
  • Reference guide
• OpenCL
  • Guide
  • API
  • OpenCL C specification
  • Reference guide
• OpenACC
  • Programming guide
  • Reference guide
• OpenMP
  • Reference guide

Overview of Libraries

• CUDA
  • cuBLAS
  • NVBLAS
  • cuFFT
  • cuGRAPH
  • cuRAND
  • cuSPARSE
• HIP
  • hipBLAS
  • hipFFT
  • hipRAND
  • hipSPARSE
• SYCL
  • OneAPI BLAS
  • OneAPI FFT
  • OneAPI sparse
  • OneAPI random number generators
• OpenCL
  • CLBlast
  • clFFT

Source-to-source Translation

• CUDA to HIP
  • hipify
• CUDA to SYCL
  • SYCLomatic
• CUDA to OpenCL
  • cutocl

Foreign Function Interfaces

• C++
  • CUDA
    • cudawrappers
  • OpenCL
    • CLHPP
• Python
  • CUDA
    • PyCuda
    • CuPy
    • cuda-python
  • HIP
    • PyHIP
  • SYCL
    • dpctl
  • OpenCL
    • PyOpenCL
• Julia
  • CUDA
    • CUDA.jl
  • HIP
    • AMDGPU.jl
  • SYCL
    • oneAPI.jl
• Java
  • CUDA
    • JCuda
  • OpenCL
    • JOCL

High-Level Abstractions

• C++
  • Kokkos
  • Raja
• Python
  • Numba
  • pykokkos

Debugging and Profiling Tools

• CUDA
  • Nsight Systems
  • Nsight Compute
  • CUDA-GDB
  • compute-sanitizer
• HIP
  • omniperf
  • rocprof
• SYCL
  • oneprof
  • onetrace

Performance Optimization

• PRACE best practice guide on modern accelerators
• CUDA best practices
• OneAPI SYCL best practices

Auto-tuning

• Kernel Tuner
  • GitHub repository
  • Documentation
  • Tutorial