k8s-runner

June 15, 2026 ยท View on GitHub

k8s-runner is the Kubernetes-native implementation of the RunnerService gRPC API.

Architecture: k8s-runner

Local Development

Full setup: Local Development

Prepare environment

git clone https://github.com/agynio/bootstrap.git
cd bootstrap
chmod +x apply.sh
./apply.sh -y

See bootstrap for details.

Run from sources

# Deploy once (exit when healthy)
devspace dev

# Watch mode (streams logs, re-syncs on changes)
devspace dev -w

E2E tests

E2E coverage runs from the centralized suite in agynio/e2e using the k8s_runner service tag. See E2E Testing.

Docker capability notes

The docker capability injects a Docker sidecar. For the rootless implementation, the sidecar runs nested runc and requires additional permissions and mounts to allow docker run to work:

  • allowPrivilegeEscalation: true for rootlesskit/newuidmap.
  • seccompProfile: Unconfined and appArmorProfile: Unconfined because default RuntimeDefault/AppArmor profiles block mount-related syscalls (for example mounting /proc) required by nested runc.
  • procMount: Unmasked to avoid /proc mount masking interfering with nested runc container setup.
  • pod.spec.hostUsers: false with an init container that writes /etc/subuid and /etc/subgid entries inside the pod user namespace.
  • HostPath mount for /dev/net/tun (type CharDevice).
  • docker-data emptyDir mounted at /home/rootless/.local/share so dockerd can create its own docker/ data root with correct ownership.

These settings can require Pod Security Admission exceptions for docker-capable workloads (baseline/restricted clusters may reject them).

Kata (microVM) docker runtimes

CAPABILITY_IMPLEMENTATIONS also supports docker: kata-qemu (and optionally docker: kata-fc). When enabled, k8s-runner keeps the privileged DinD sidecar behavior but sets pod.spec.runtimeClassName to match the selected Kata implementation. The cluster must provide the matching RuntimeClass and schedule onto KVM-capable nodes. This cannot be validated on local k3d/mac setups.

Workload egress NetworkPolicy

The Helm chart can install the static egress NetworkPolicy used by egress v1. Enable workloadEgressNetworkPolicy.enabled and set workloadNamespace to the namespace where runner-created workload pods run. The policy selects workload pods with agyn.dev/managed-by=agents-orchestrator, allows OpenZiti synthetic addresses (100.64.0.0/10), cluster DNS, and public internet, and excludes workloadEgressNetworkPolicy.clusterPodCIDR, workloadEgressNetworkPolicy.clusterServiceCIDR, and workloadEgressNetworkPolicy.additionalInternalCIDRs from public internet egress. blockedCIDRs remains as a deprecated compatibility alias.

The runner runtime does not create or update NetworkPolicy resources, and its ServiceAccount does not need networkpolicies RBAC.

Workload egress NetworkPolicy

The chart installs agent-workload-egress by default in the workload namespace. It selects pods labeled agyn.dev/managed-by: agents-orchestrator, allows DNS, allows the OpenZiti tunnel CIDR (100.64.0.0/10), and allows public internet egress while excluding configured cluster and internal CIDRs. Set workloadEgressNetworkPolicy.clusterPodCIDR, workloadEgressNetworkPolicy.clusterServiceCIDR, and workloadEgressNetworkPolicy.additionalInternalCIDRs for a production cluster.