Deploy

Initial install of Ingero Echo (cluster-wide event store) and Fleet (custom OTEL Collector distribution) on a Kubernetes cluster.

This guide covers the Helm path. For binary-on-a-VM see ../single-node-deployment.md.

Prerequisites

• Kubernetes 1.27+ with a default StorageClass that supports ReadWriteOnce PVCs
• Helm 3.13+
• A namespace for Echo + Fleet (ingero is the convention)
• A ghcr.io pull secret if your cluster does not have anonymous GHCR access

Image verification (cosign)

v1.0.0 images and binaries are signed via cosign keyless OIDC. Verify before install:

cosign verify ghcr.io/ingero-io/ingero-echo:1.0.0 \
  --certificate-identity-regexp '^https://github.com/ingero-io/ingero-fleet/' \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com

cosign verify ghcr.io/ingero-io/ingero-fleet:1.0.0 \
  --certificate-identity-regexp '^https://github.com/ingero-io/ingero-fleet/' \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com

Binary archives carry the same signature on checksums.txt:

curl -fsSL -o checksums.txt \
  https://github.com/ingero-io/ingero-fleet/releases/download/v1.0.0/checksums.txt
curl -fsSL -o checksums.txt.sig \
  https://github.com/ingero-io/ingero-fleet/releases/download/v1.0.0/checksums.txt.sig

# cosign looks up the issuing certificate in the Rekor
# transparency log via --certificate-identity-regexp.
cosign verify-blob checksums.txt \
  --signature checksums.txt.sig \
  --certificate-identity-regexp '^https://github.com/ingero-io/ingero-fleet/' \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com

The cosign signature is over checksums.txt; the trust chain to the actual binary archive runs through the SHA-256 hash inside checksums.txt. Always run the second step:

# After the cosign verify succeeds, validate the archive matches.
curl -fsSL -O https://github.com/ingero-io/ingero-fleet/releases/download/v1.0.0/ingero-fleet_1.0.0_linux_amd64.tar.gz
sha256sum -c checksums.txt --ignore-missing

If sha256sum -c fails, the archive has been tampered with even though the signature on checksums.txt is valid. Do NOT install.

SBOMs are attached to each archive as *.cyclonedx.sbom.json. SBOM integrity is anchored on the cosign-signed checksums.txt (each SBOM's SHA-256 is in the same signed checksum file). After the cosign verify-blob checksums.txt step above succeeds, sha256sum -c validates the SBOMs:

SBOM=ingero-fleet_1.0.0_linux_amd64.tar.gz.cyclonedx.sbom.json
curl -fsSL -O "https://github.com/ingero-io/ingero-fleet/releases/download/v1.0.0/${SBOM}"
sha256sum -c checksums.txt --ignore-missing 2>&1 | grep "${SBOM}: OK"

Feed the verified SBOM to your scanner of choice (Trivy / Snyk / Grype).

Bearer token

Echo's HTTP+JSON, MCP, and OTLP listeners all require a bearer token. Mint one:

openssl rand -hex 32 > /tmp/echo-token

Create a Kubernetes Secret:

kubectl create namespace ingero
kubectl -n ingero create secret generic ingero-echo-auth \
  --from-file=token=/tmp/echo-token

ingero-echo-auth is the name the chart looks for by default. Override via auth.bearerTokenSecretName.

Echo install

helm install ingero-echo ./helm/ingero-echo \
  --namespace ingero \
  --set image.tag=v1.0.0 \
  --set 'extraEnv[0].name=AUTH_TOKEN' \
  --set 'extraEnv[0].valueFrom.secretKeyRef.name=ingero-echo-auth' \
  --set 'extraEnv[0].valueFrom.secretKeyRef.key=token' \
  --wait

Required: pick a StorageClass with encryption-at-rest

Echo's DuckDB store contains audit rows (bearer hashes, source IPs, key names, tool inputs). The default cluster StorageClass is usually unencrypted; this is a security gap. Pick a StorageClass with encryption-at-rest enabled.

Examples by cloud:

Cloud	StorageClass
AWS	Define a StorageClass referencing the ebs.csi.aws.com provisioner with encrypted: "true" and a KMS key
GCP	pd-balanced-csi or pd-ssd-csi (CSI driver default-encrypted)
Azure	managed-csi with kind: Managed and skuName: Premium_LRS against an encrypted disk set

Override the chart default:

--set persistence.storageClass=gp3-encrypted

Optional: ServiceMonitor for Prometheus scrape

--set serviceMonitor.enabled=true \
--set serviceMonitor.bearerTokenSecret.name=ingero-echo-auth \
--set serviceMonitor.bearerTokenSecret.key=token

The NetworkPolicy template automatically allows ingress from pods matching serviceMonitor.scraperSelector (default: kube-prometheus-stack app.kubernetes.io/name=prometheus).

Optional: Ingress

--set ingress.enabled=true \
--set ingress.className=nginx \
--set 'ingress.hosts[0].host=echo.example.com' \
--set 'ingress.hosts[0].paths[0].path=/api/' \
--set 'ingress.hosts[0].paths[0].pathType=Prefix' \
--set 'ingress.tls[0].hosts[0]=echo.example.com' \
--set 'ingress.tls[0].secretName=echo-tls'

TLS termination is the ingress controller's responsibility. For mTLS / client-cert verification see tls.md.

Fleet install

helm install ingero-fleet ./helm/ingero-fleet \
  --namespace ingero \
  --set image.tag=v1.0.0

Fleet's OTLP receiver listens on :4317 (gRPC) and :4318 (HTTP); the Helm chart wires the in-process processors (ingero, nccl, provider_lookup) and exposes a Prometheus self-metrics endpoint on :8888.

To forward Fleet's processed metrics to Echo, set the OTLP exporter in your Fleet config (the examples/lambda-e2e/scripts/22-deploy-echo.sh script does this automatically via a ConfigMap patch).

Verify

# Capability negotiation (unauthenticated)
kubectl -n ingero port-forward svc/ingero-echo 8081:8081 &
curl -fsS http://127.0.0.1:8081/api/versions | jq .

# Bearer-authed health
TOKEN=$(kubectl -n ingero get secret ingero-echo-auth -o jsonpath='{.data.token}' | base64 -d)
curl -fsS -H "Authorization: Bearer $TOKEN" http://127.0.0.1:8081/api/v2/health | jq .

# Tool catalog
curl -fsS -H "Authorization: Bearer $TOKEN" http://127.0.0.1:8081/api/v2/tools/list | jq '.tools | length'

supported should be ["v2"], preferred should be "v2". v1 paths return HTTP 410 Gone.

Next

• Upgrades
• Backup + restore
• Multi-tenant onboarding
• TLS
• Troubleshooting