CI Overview

August 4, 2026 · View on GitHub

This document provides an overview of how Continuous Integration (CI) works in TheRock, including the build → artifact → test pipeline. If you're migrating from MathCI or Jenkins-based workflows, this guide will help you understand TheRock's GitHub Actions-based approach.

Quick Summary

TheRock CI follows this workflow:

  1. Build ROCm from source via TheRock on CPU machines
  2. Upload build artifacts to S3 (public-read buckets)
  3. Test downloads and tests artifacts on GPU machines for testing

Instead of Jenkins and Groovy pipelines, TheRock uses GitHub Actions workflows defined in YAML files under .github/workflows/.

CI Architecture

TheRock uses a multi-stage CI pipeline that splits the build into stages (compiler-runtime → runtime-tests/math-libs, etc.) with dependency chaining. Below is a general diagram of the CI flow

graph TD
    A[generic build] --> B1[arch build - gfx94X-dcgpu]
    A[generic build] --> B2[arch build - gfx110X-dgpu]
    B1 --> C[test - gfx94X-dcgpu]
    B2 --> D[test - gfx110X-dgpu]

Each stage runs as a separate job, uploads its artifacts and logs to S3, then downstream stages download and build on top of them. This allows for:

  • Parallelization: Multiple GPU families can build math-libs simultaneously once compiler-runtime completes
  • Incremental builds: Test-only runs can skip build stages by downloading pre-built artifacts
  • Flexibility: Different stages can run on different runner types (e.g., CPU-only for build, GPU for tests)

Key workflow files:

The WSL ROCDXG stage is a special case in the portable Linux multi-arch flow: the job starts on a Windows runner, then runs artifact fetch, configure, build, and upload steps inside a WSL Ubuntu shell. See WSL ROCDXG CI Stage for details.

Build Phase

TheRock builds ROCm components from source and produces artifacts - archive slices of key components.

graph LR
    A[Configure CMake] --> B[CMake Build]
    B --> D[Upload to S3]

What gets built: Compiler (LLVM, etc.), core runtime (HIP, ROCr, etc.), math libraries (rocBLAS, rocFFT, etc.), ML libraries (MIOpen, etc.), media libraries (rocDecode, rocJPEG), computer vision libraries (RPP), and more.

Artifact organization: Each component is packaged into separate archives by sub-components (lib, run, dev, doc, test). See artifacts.md for complete details on artifact structure and naming conventions.

Artifact Storage and Distribution

S3 Buckets

TheRock uses Amazon S3 for artifact storage. All artifact buckets are public-read, so no authentication is needed to download them.

See s3_buckets.md for the complete bucket list and authentication details for uploads.

Accessing Artifacts

Download artifacts using the fetch_artifacts.py script, which handles fetching from S3 and extracting to the correct locations.

See installing_artifacts.md for detailed instructions on how the CI system installs artifacts:

  • Finding GitHub run IDs
  • Selecting components to download
  • Installing from CI runs vs releases
  • Using with different GPU families

Test Phase

Tests are defined in fetch_test_configurations.py, which generates a test matrix for parallel execution across multiple runners.

graph LR
    A[Configure Test Matrix] --> C[Sanity Check]
    C --> D1[Test rocBLAS]
    C --> D2[Test rocFFT]
    C --> D3[Test MIOpen]
    C --> D4[Test...]

The test workflow downloads only the artifacts needed for the specific tests being run (e.g., --blas --tests for rocBLAS tests), installs them, and runs the test scripts in parallel.

Test configuration: Each test specifies which artifacts to download, timeout, platform support (Linux/Windows), and the test script to run.

Test scripts: Python scripts in build_tools/github_actions/test_executable_scripts/ that work on both Linux and Windows. (shortly, these scripts will be migrated to their respective monorepos)

See adding_tests.md for how to add new tests to the CI pipeline.

Common CI Tasks

Viewing logs

At the start of a CI run, you will be able to see a GitHub Step Summary in the Summary page of the CI run.

As each build completes and uploads, you are able to find the index pages for artifacts and logs.

Trigger a CI Run

CI runs on:

  • workflow_dispatch of multi_arch_ci
    • if a build is already available in CI, you are able to re-use those artifacts for test runs. See ci_behavior_manipulation.md doc below.
  • pull_request
  • push to main branch
  • schedule: nightly runs

See ci_behavior_manipulation.md on ways to trigger and manipulate the CI

Run Specific Tests

Use workflow dispatch with test labels to run only specific tests instead of the full suite.

See test_filtering.md for advanced filtering options.

Add a New Test

  1. Create test script in build_tools/github_actions/test_executable_scripts/
  2. Add entry to fetch_test_configurations.py
  3. Ensure artifact dependencies are configured in install_rocm_from_artifacts.py

See adding_tests.md for step-by-step instructions.

Reproduce CI Failures Locally

Use the automated reproduction script to download artifacts and set up the same environment as CI.

See test_environment_reproduction.md for detailed instructions.

Debug Build Failures

Build logs are uploaded to S3 and organized by stage and GPU family.

See workflow_outputs.md for the S3 layout structure and github_actions_debugging.md for debugging techniques.

Further Reading

Build System

Testing

Infrastructure

Getting Help