Releasing AgentOps

July 9, 2026 · View on GitHub

This document describes the release process for the ao CLI and AgentOps plugin.

Overview

Releases are triggered by git tags and use a publisher-only workflow:

git tag vX.Y.Z
    ↓
┌──────────────────────────────────────────────────────┐
│          Local gate (authoritative)                  │
├──────────────────────────────────────────────────────┤
│  ./scripts/ci-local-release.sh                       │
│  - validation + tests + smoke checks                 │
│  - bootstrap + pawl smoke                            │
│  - SBOM + security report artifacts                  │
│  - release readiness score + SIL/VIL/HIL evidence    │
└──────────────────────────────────────────────────────┘
                         ↓
┌──────────────────────────────────────────────────────┐
│           release.yml (publisher only)               │
├──────────────────────────────────────────────────────┤
│  - pre-publish SBOM + security + readiness evidence  │
│  - GoReleaser publish                                │
│  - GitHub Release notes + assets                     │
│  - Homebrew update                                   │
│  - SLSA provenance attestation                       │
└──────────────────────────────────────────────────────┘

Making a Release

1a. Release Size Check

Releases should contain at most 15 commits per minor version bump. v2.2.0 had 26 commits — too large to review, debug, or bisect effectively.

If the commit count since the last tag exceeds 15:

Split into multiple releases (e.g., vX.Y.Z for first batch, vX.Y.(Z+1) for remainder)
Or ensure each commit is well-scoped and the changelog covers all changes

Check commit count:

git log $(git describe --tags --match 'v[0-9]*.[0-9]*.[0-9]*' --abbrev=0)..HEAD --oneline | wc -l

The pre-flight validation script (scripts/validate-release.sh) will warn (not fail) when this threshold is exceeded.

1b. Historical Changelog Coverage

The maintained changelog coverage floor is v2.16.0. Earlier tags are treated as historical bootstrap tags and do not require backfilled curated notes or audit documents for current releases. If a future release needs a complete pre-v2.16.0 history, file explicit backfill beads for the missing versions instead of blocking the active release by default.

1c. Agent Metadata Mutation

Release validation must be read-only with respect to tracked finding metadata. scripts/ci-local-release.sh runs the release smoke path through scripts/check-release-agent-metadata-stable.sh, and scripts/release-smoke-test.sh uses ao inject --no-cite, ao lookup --no-cite, and ao flywheel close-loop --dry-run by default.

Intentional citation/finding metadata refreshes are opt-in:

AGENTOPS_RELEASE_ALLOW_AGENT_MUTATIONS=1 scripts/ci-local-release.sh

2. Update CHANGELOG

Follow Keep a Changelog format:

## [X.Y.Z] - YYYY-MM-DD

### Added
- Feature description

### Changed
- Change description

### Fixed
- Fix description

3. Create and Push Tag

# Ensure you're on main and up to date
git checkout main
git pull

# Create annotated tag
git tag -a vX.Y.Z -m "Release vX.Y.Z"

# Push commit and tag (triggers Validate and the release workflow)
git push origin main
git push origin vX.Y.Z

# Release is not done until full Validate is green for the exact tagged SHA
scripts/verify-release-ci.sh vX.Y.Z

4. Monitor the Publisher Workflow

Watch the release at: https://github.com/boshu2/agentops/actions

The workflow runs three jobs:

doc-release-gate - Confirms release docs and generated references are current
pre-publish-evidence - Generates SBOM, runs the full security gate, and writes release-readiness evidence before publish
publish - Runs only after both gates pass; publishes artifacts, updates Homebrew, uploads the pre-publish evidence bundle, and signs attestation

5. Verify the Release

After the workflow completes:

# Update Homebrew
brew update

# Upgrade ao
brew upgrade agentops

# Verify version
ao version
# Should show: ao version X.Y.Z

6. Verify Integrity (checksums + provenance)

# Download and verify checksums
curl -sL https://github.com/boshu2/agentops/releases/download/vX.Y.Z/checksums.txt
shasum -a 256 -c checksums.txt --ignore-missing

# Verify SLSA provenance (requires gh CLI)
gh attestation verify ao-darwin-arm64.tar.gz --repo boshu2/agentops

Release Artifacts

Each release produces:

Artifact	Description
`ao-darwin-amd64.tar.gz`	macOS Intel binary
`ao-darwin-arm64.tar.gz`	macOS Apple Silicon binary
`ao-linux-amd64.tar.gz`	Linux x86_64 binary
`ao-linux-arm64.tar.gz`	Linux ARM64 binary
`checksums.txt`	SHA256 checksums for all archives
`sbom-cyclonedx-go-mod.json`	Publishable CycloneDX SBOM for Go dependencies
`security-gate-summary.json`	Security scan summary (gitleaks/semgrep/gosec/trivy/etc.)
`release-readiness.json`	Release readiness score with SIL/VIL/HIL status
`eval-agentops-fast.json`	AgentOps eval proof summary for release readiness
`eval-baseline-audit.json`	Eval baseline drift audit; stale suite hashes block audit resolution
`digital-twin-evidence.json`	Local release digital-twin/VIL proof
SLSA attestation	Build provenance (verifiable via `gh attestation verify`)

Release Notes

Release notes are auto-generated by GoReleaser with:

Header with install/upgrade instructions and integrity verification
Changelog grouped by type (features, fixes, docs, other)
Footer with full changelog diff link

The template lives in .goreleaser.yml under release.header and release.footer.

Validation Checks

Release validation is local-first and enforced by:

./scripts/ci-local-release.sh

This local gate runs doc checks, manifest/schema checks, smoke/integration checks, bootstrap and pawl smoke paths, binary validation, SBOM generation, security scans, AgentOps eval evidence, digital-twin/VIL evidence, and the release readiness score. Official release audits require SIL/VIL evidence plus workflow-rich HIL evidence or an explicit HIL waiver. For command variants and expected release-E2E smoke markers, see Release E2E Checklist.

Failure Modes

Local Gate Fails

If ./scripts/ci-local-release.sh fails, do not tag or publish.

To fix:

Identify the issue from the workflow logs
Fix the code
Delete the tag: git tag -d vX.Y.Z && git push origin :refs/tags/vX.Y.Z
Re-run the local gate until all checks pass
Create and push the tag

Exact-SHA Validate Fails

If scripts/verify-release-ci.sh vX.Y.Z prints NO-GO release-ci, the release is not complete even if the tag exists locally or the publisher workflow starts.

To fix:

Open the reported run URL and repair the failing Validate job.
Commit the fix on main.
Move the release tag to the fixed commit with scripts/retag-release.sh vX.Y.Z, or delete and recreate the local tag before any public release exists.
Re-run scripts/verify-release-ci.sh vX.Y.Z and record the GO release-ci line in the handoff or release audit notes.

Publish Fails

If publisher CI fails after a local gate pass, the release may be in a partial state.

To fix:

Check if GitHub release was created (may need manual cleanup)
Check if Homebrew formula was pushed
If the original tag-push workflow is still running for that tag, wait for it to finish before starting anything else
For post-tag commits, use scripts/retag-release.sh vX.Y.Z
Use workflow_dispatch only to rerun an existing tag after the original tag-triggered workflow has already finished and you are not pushing a new tag in parallel
scripts/retag-release.sh already waits on the tag-push workflow. It should not launch a second manual publish run for the same tag.

Why this matters: a second publish for the same tag can collide on release assets and fail with GitHub 422 already_exists errors if the first publish already created or uploaded assets.

Homebrew Token Expired

The HOMEBREW_TAP_GITHUB_TOKEN secret is validated before publish. If it fails:

Generate a new PAT at https://github.com/settings/tokens
Scope: public_repo (for homebrew-agentops)
Update the secret in repository settings

Manual Release (workflow_dispatch)

Use workflow_dispatch only when you need to rerun an existing tag release after the original tag-triggered workflow has already completed or failed.

Do not use workflow_dispatch as the primary path for a fresh release. The primary path is still git push origin vX.Y.Z.

Do not launch a manual run in parallel with a tag-push run for the same tag.

If you have post-tag commits that should become part of the same version, use:

scripts/retag-release.sh vX.Y.Z

If you only need to retry the existing tag with no new commits:

Go to Actions → Release workflow
Click "Run workflow"
Enter the tag (e.g., vX.Y.Z)
Click "Run workflow"

Local Testing

Before tagging, you can test the build locally:

# Run CI-equivalent local release gate (required)
./scripts/ci-local-release.sh

# Once the target version is known, bind the artifacts to that version for the audit
./scripts/ci-local-release.sh --release-version X.Y.Z
./scripts/resolve-release-artifacts.sh X.Y.Z

# Publishable local artifacts will be written to:
# .agents/releases/local-ci/<timestamp>/
#   - sbom-vX.Y.Z.cyclonedx.json
#   - sbom-vX.Y.Z.spdx.json
#   - security-gate-full.json
#   - eval-agentops-fast.json
#   - eval-baseline-audit.json
#   - digital-twin-evidence.json
#   - hil-evidence.json
#   - release-readiness.json
#   - release-artifacts.json

# Install goreleaser
brew install goreleaser

# Validate config
goreleaser check

# Build snapshot (doesn't require tag)
goreleaser build --snapshot --clean --single-target

# Test the binary
./dist/ao_darwin_arm64/ao version
./dist/ao_darwin_arm64/ao --help
./dist/ao_darwin_arm64/ao status

# Run doc-release gate
./tests/docs/validate-doc-release.sh

Dependency Automation Policy

Dependency automation is managed by Dependabot and follows this policy:

Scope: Go modules (/cli/go.mod) and GitHub Actions (/.github/workflows/*.yml)
Cadence: Weekly on Monday (separate schedules for Go and Actions)
Grouping: Minor and patch updates are grouped per ecosystem
Major updates: Opened as separate PRs for explicit review
Security updates: Treated as priority work and merged outside normal cadence when validated
PR limits: Capped to keep queue manageable and avoid review overload

Configuration Files

File	Purpose
`.goreleaser.yml`	Build config, checksums, release notes, Homebrew formula
`.github/workflows/release.yml`	Publisher-only release workflow (publish + assets + attestation)
`.github/workflows/validate.yml`	CI validation (includes doc-release stabilization gate)
`.github/workflows/nightly.yml`	Nightly tests with failure alerts
`.github/dependabot.yml`	Dependabot policy and schedules for Go modules + GitHub Actions
`scripts/validate-release.sh`	Binary validation script
`tests/docs/validate-doc-release.sh`	Unified doc-release gate (links + skills + message freeze)

Homebrew Tap

The Homebrew formula is automatically pushed to: https://github.com/boshu2/homebrew-agentops

Users install with:

brew tap boshu2/agentops
brew install agentops

Security

Checksums: SHA256 checksums for all archives (checksums.txt)
SLSA provenance: Build attestation via actions/attest-build-provenance
Dependabot: Automated dependency updates for Go modules and GitHub Actions
Security scan: Secrets and dangerous pattern detection on every PR

See SECURITY.md for vulnerability reporting.

Troubleshooting

"ao version" shows "dev"

The ldflags version injection failed. Check .goreleaser.yml:

ldflags:
  - -s -w -X main.version={{ .Version }}

./scripts/ci-local-release.sh should catch this before tagging.

Binary artifact missing or renamed

GoReleaser archive naming doesn't match extraction pattern. Check:

.goreleaser.yml archives section
Release asset names in the GitHub Release page

Homebrew formula not updated

Check HOMEBREW_TAP_GITHUB_TOKEN is valid
Check workflow logs for push errors
Verify formula at homebrew-agentops repo

Attestation verification fails

# Ensure gh CLI is authenticated
gh auth status

# Try verification with verbose output
gh attestation verify <file> --repo boshu2/agentops --verbose