github-actions.md

system-tests expose an official reusable workflow that allow to create a simple workflow running system-tests

name: System Tests

on:
  pushes:
    branches:
      - main
  pull_request:
    branches:
      - "**"

jobs:
  # first, build the artifact to be tested. This part is related to your repo
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: ./build.sh
      - uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
        with:
          name: binaries
          path: path/to/your/binary


  system-tests:
    needs:
      - build
    uses:  DataDog/system-tests/.github/workflows/system-tests.yml@main
      secrets: inherit
      permissions:
        contents: read
        id-token: write
      with:
        library: java
        binaries_artifact: binaries
        desired_execution_time: 900
        scenarios_groups: tracer-release
        skip_empty_scenarios: true

Parameters

Name	Description	Type	Required	Default
artifact_retention_days	How long should artifacts produced by the workflow should be retained	number	false	14
binaries_artifact	Artifact name containing the binaries to test	string	false	empty
desired_execution_time	In seconds, system-tests will try to respect this time budget.	number	false	empty
excluded_scenarios	Comma-separated list of scenarios not to run	string	false	empty
force_execute	Comma-separated list of tests to run even if they are skipped by manifest or decorators	string	false	empty
library	Library to test	string	true	—
parametric_job_count	How many jobs should be used to run PARAMETRIC scenario	number	false	1
push_to_test_optimization	Push tests results to DataDog Test Optimization. Fetches credentials automatically via dd-sts	boolean	false	false
test_optimization_datadog_site	DataDog site to use for test optimization	string	false	datadoghq.com
ref	system-tests ref to run the tests on (can be any valid branch, tag or SHA in system-tests repo)	string	false	main
scenarios	Comma-separated list scenarios to run	string	false	DEFAULT
scenarios_groups	Comma-separated list of scenarios groups to run	string	false	empty
skip_empty_scenarios	Skip scenarios that contain only xfail or irrelevant tests	boolean	false	false

Note: if test_optimization_datadog_site is set to true, then the calling repo must be referenced in dd-sts.

Permissions

The following permissions are always required:

Permission	Reason
contents: read	Checkout the repository
id-token: write	Required by downstream workflows to fetch Datadog credentials via dd-sts. GitHub validates this upfront, so it must be granted even if test optimization is disabled

Secrets

For some purposes, secrets are used in the workflow:

Name	Description
DD_API_KEY	Some scenarios requires valid API and APP keys
DD_APPLICATION_KEY
DD_API_KEY_2
DD_APP_KEY_2
DD_API_KEY_3
DD_APP_KEY_3
DOCKERHUB_USERNAME and DOCKERHUB_TOKEN	If both are set, all docker pull are authenticated, which offer higher rate limit

You can sends them ,either by using secrets: inherit (doc), or use explicit secret ids

Private parameters

Those parameters are used only by system-tests own CI

Name	Description	Type	Required	Default
_system_tests_dev_mode	Shall we run system tests in dev mode (library and agent dev binary)	boolean	false	false
_system_tests_library_target_branch_map	If system-tests dev mode, JSON map of library to branch (e.g. {"java":"b1","dotnet":"b2"})	string	false
_build_buddies_images	Shall we build buddies images	boolean	false	false
_build_proxy_image	Shall we build proxy image	boolean	false	false
_build_lambda_proxy_image	Shall we build the lambda-proxy image	boolean	false	false
_build_php_base_images	Shall we build php base images for tests on python tracer	boolean	false	false
_build_python_base_images	Shall we build python base images for tests on python tracer	boolean	false	false
_enable_replay_scenarios	Enable replay scenarios, should only be used in system-tests CI	boolean	false	false