nomad-driver-microsandbox

A Nomad task driver plugin that runs tasks as Microsandbox microVMs (libkrun) instead of containers — each task gets its own Linux kernel and a real VM boundary, booting in well under a second.

driver = "microsandbox" schedules a hardware-isolated microVM the same way driver = "docker" schedules a container.

How it works

The driver embeds the microsandbox Go SDK, which drives libkrun in-process via a runtime dlopen of the microsandbox FFI library — there is no daemon and no separate server. Sandboxes are created detached so the microVM outlives the plugin process, and their state is persisted on disk under MSB_HOME. That is what makes recovery work without a daemon: after a Nomad agent restart, RecoverTask re-attaches to the running sandbox by name (msb.GetSandbox(name)).

Nomad lifecycle	Microsandbox SDK
StartTask	CreateSandbox(name, WithDetached(), WithLabels{alloc}, WithImage/CPUs/Memory/Env/PortBindings) → Detach()
RecoverTask	GetSandbox(name) / ListSandboxesWith(label) — running ⇒ resume, gone ⇒ exited
WaitTask	monitor goroutine on GetSandbox(name).WaitUntilStopped()
StopTask	GetSandbox(name).Stop(WithStopTimeout)
DestroyTask	Kill() + RemoveSandbox(name)
ExecTask	GetSandbox(name).Connect().Exec(...) — backs nomad alloc exec
TaskStats	GetSandbox(name).Metrics()
Fingerprint	healthy iff /dev/kvm present and runtime installed in MSB_HOME

Requirements

• Linux with KVM enabled (/dev/kvm readable/writable by the Nomad client).
• The microsandbox runtime (msb + libkrunfw) staged under MSB_HOME (the deploy role pre-installs it; the driver reports unhealthy until present).
• Nomad 1.9.x (pinned in go.mod; the driver gRPC contract is stable across 1.9–1.11 but the Go structs must match the agent).

Build

cgo is required (the SDK's dlopen shim), but no Rust toolchain or libkrun is needed at build time:

make build              # linux/amd64 (default), CGO_ENABLED=1
make build-native       # host arch (e.g. on the Nomad client during deploy)

Configure (Nomad client)

Drop the binary in the client plugin_dir and enable it:

plugin "nomad-driver-microsandbox" {
  config {
    enabled           = true
    msb_home          = "/opt/microsandbox"
    default_cpus      = 1
    default_memory_mb = 512
  }
}

A Nomad client agent restart is required to load a new task driver (SIGHUP does not). Confirm with nomad node status -verbose <node> → driver.microsandbox = healthy.

Task config

task "agent" {
  driver = "microsandbox"
  config {
    image   = "registry.example/claude-code-sandbox:latest"
    cpus    = 2          # whole cores; falls back to resources / default
    memory  = 2048       # MiB; falls back to resources.memory / default
    network = "public-only"   # public-only | allow-all | none | non-local
    ports   = ["3000:3000"]   # "hostPort:guestPort", published on 127.0.0.1
    command = ["/bin/sh", "-c", "..."]  # runs as the primary (captured) session
    stop_on_exit = false      # true ⇒ stop the VM when command exits (batch)
    env     = [{ FOO = "bar" }]
    labels  = [{ team = "agents" }]
    workdir = "/work"
  }
  resources { cpu = 2000; memory = 2048 }
}

command runs as the sandbox's primary exec session (not the image entrypoint), so its stdout/stderr are captured to nomad alloc logs. By default the VM stays warm after the command exits, so you can nomad alloc exec in and take over; the task completes only when the VM is stopped. Set stop_on_exit = true for run-to-completion batch semantics: the VM is stopped as soon as the command exits and the task completes with the command's exit code.

Status

v0.1.0 — verified live on a KVM host (Nomad 1.9.5): boots a libkrun microVM, fingerprints healthy, runs the workload as the primary session with nomad alloc logs capturing stdout/stderr, nomad alloc exec attaches into the guest, and stop/destroy tears the VM down cleanly.

Since then: private OCI images are pulled with registry credentials from the plugin config (registry_user / registry_password), so headless hosts with no keyring can pull without the msb registry login / side-load hack.

Run-to-completion batch semantics (stop_on_exit) and orphan-sandbox reconciliation are implemented: a background reconciler reaps driver-managed sandboxes (identified by their nomad.alloc_id label) that Nomad no longer tracks, after a startup grace period so live tasks are recovered first.

Not yet implemented: a true PTY for fully interactive tty sessions (the attach path uses streaming exec) and snapshot/restore. See the deployment plan in the deployments repo.