aap-integration-tests

July 28, 2026 · View on GitHub

The cross-repository integration test builder and runner for the AAP ecosystem. No other repo runs these tests.

See ARCHITECTURE.md for the full design and rationale. This README is just an entry map; the architecture doc is the source of truth.

Status: early integration harness. The CLI and GitHub Actions workflow are wired, but green runs still depend on Android device stability and fresh cross-repository artifact pins.

What it does

  1. Read a named setup catalog (catalogs/<name>.json) — a flat list of { repo, commit, artifacts, files? } download entries.
  2. Acquire the cataloged module/plugin APKs from each commit's GitHub Actions artifacts (PAT-authenticated), into a working dir, cached by repo@commit.
  3. Install them on an adb-connectable device. By default an already installed package is skipped; pass --reinstall to force adb install -r.
  4. Run test cases through the embedded JS automation surfaces (adb am broadcast) in plugin host apps and uapmd-app.
  5. Report per-case pass/fail details and a final suite summary. Some helper code for audio-golden verification exists, but the committed CI suite is currently focused on load/inspect/project behavior.

Layout

catalogs/            named setup catalogs (committed)
src/
  cli.js             entrypoint wiring the steps
  paths.js           .work/ directory layout
  catalog.js         load + validate a catalog
  acquire.js         resolve commit -> artifact -> download -> cache
  install.js         adb install, skipping installed packages unless forced
  verify.js          golden / spectrum tolerant comparison helpers
  device/            device providers (gmd | local | firebase)
tests/cases/         test-case definitions
.work/               working dir — NOT committed; Actions-cached

Usage

npm install

# Run the integration matrix with one command.
# Downloads catalog APKs, installs them once, then runs every committed CI case.
export GITHUB_TOKEN=<PAT with artifact-read scope>
npm test -- --device auto

# If the APKs are already installed on a connected device/emulator:
npm test -- --device auto --skip-acquire --skip-install

# Force reinstall staged/downloaded APKs, useful when a device has an outdated
# package with the same package name already installed:
npm test -- --device gmd --reinstall

# Pass runner options after npm's `--` delimiter:
npm run test -- --reinstall --device=gmd

# No device: download APKs and bring up an emulator (GMD).
export GITHUB_TOKEN=<PAT with artifact-read scope>
npm test -- --device gmd

Key options: --token (PAT; or $GITHUB_TOKEN), --device auto|local|gmd|firebase, --serial, --suite ci|all, --skip-acquire / --skip-install, --reinstall, --host-apk. Requires adb, aapt/aapt2, and the Android SDK emulator for gmd.

For single-case debugging, keep using --case:

npm test -- --case connectivity-mda --catalog mda-ci --device auto
npm run test -- --case=connectivity-mda --catalog=mda-ci --device=auto

# Requires aaphostsample built with offline test-graph support; see tests/cases/README.md.
npm test -- --case instrument-effect-audibility --catalog supported-plugins-ci \
  --host-apk /absolute/path/to/aaphostsample-debug.apk --device auto

Local APK overrides

To test a local APK instead of the downloaded artifact, copy it into .work/local/ using the catalog's final destination filename. Local APKs are preferred over same-named files in .work/downloaded/ during install.

Examples:

mkdir -p .work/local

# aap-core host app, replacing the artifact entry:
# samples/aaphostsample/build/outputs/apk/debug/aaphostsample-debug.apk -> aaphostsample.apk
cp ../aap-core/samples/aaphostsample/build/outputs/apk/debug/aaphostsample-debug.apk \
  .work/local/aaphostsample.apk

# BYOD plugin, replacing:
# BYOD-debug.apk -> aap-juce-byod.apk
cp ../aap-juce-byod/ignore/app/build/outputs/apk/debug/BYOD-debug.apk \
  .work/local/aap-juce-byod.apk

# uapmd app, replacing the extracted artifact filename used by uapmd catalogs:
cp ../uapmd/app/build/outputs/apk/debug/app-debug.apk \
  .work/local/app-debug.apk

npm test -- --case uapmd-byod-preset-values --skip-acquire --device auto

The filename after .work/local/ must match the destination name in the catalog's files map, for example catalogs/byod-ci.json uses aaphostsample.apk and aap-juce-byod.apk. If a catalog entry has no files map, use the APK basename extracted from the artifact; for uapmd that is app-debug.apk. If you do not pass --skip-acquire, the runner may still download missing catalog artifacts first; .work/local only overrides which APK is installed. The installer only consumes .apk files from .work/local; AARs are not installed by this harness.

CI (GitHub Actions)

  • integration-tests — downloads host + plugin APKs by commit from GitHub Actions artifacts, first runs npm run unit as a fast Node preflight, then boots an emulator (reactivecircus/android-emulator-runner), installs APKs, and runs the default integration suite (npm test -- --device auto). It uses API 35 because uapmd's Android app requires API 31+. Manual + nightly + on main.

    Prerequisites before it goes green:

    1. Secret AAP_ARTIFACTS_PAT — a PAT with artifact-read access to every source repo named by catalogs/*.json (default GITHUB_TOKEN can't read other repos' artifacts).
    2. Catalog pins whose GitHub Actions artifacts still exist, or cached copies in .work/cache. Otherwise acquire fails with a clear artifact error.
    3. Enough emulator capacity for uapmd and the larger plugin set; the workflow captures logcat and meminfo as integration-diagnostics on every run.

uapmd-based testing

A second on-device control surface: uapmd-app exposes the same broadcast protocol with a uapmd.* facade (project save/load, tracks, instancing). The uapmd-project case type (tests/cases/uapmd-project-mda.json) creates a new project, adds a track + plugin per entry, saves, reloads, and verifies — driving the uapmd sequencer/project stack on top of AAP.

node src/cli.js --case uapmd-project-mda --device auto --skip-acquire   # uapmd-app already installed

Catalog catalogs/uapmd-ci.json downloads the uapmd-app APK + MDA plugin. Prerequisite: uapmd's android.yml is workflow_dispatch-only and builds against a chosen aap-core ref — dispatch it against an aap-core commit with the current js-controller, then pin the uapmd commit in uapmd-ci.json (placeholder until then).