2.2.22-Backend-Docker-Model-Runner.md

Docker Model Runner

Handle: dmr
URL: http://localhost:34920

Docker Model Runner is Docker's local model runtime (Docker Desktop on macOS/Windows, or the docker-model-plugin package on Linux). Harbor exposes it as a backend service by starting a small proxy container while Docker manages the host-native runner. This keeps Harbor's Compose workflow intact and lets containers call an OpenAI-compatible /v1 API.

DMR is especially useful on Apple Silicon because Metal inference runs on the macOS host while Harbor services consume it over HTTP.

Starting

harbor up dmr

When HARBOR_DMR_MANAGE_HOST=true, Harbor automatically installs or enables missing Docker Model Runner host components before starting the proxy. On Linux this uses the official docker-model-plugin package through apt or dnf when the docker model CLI is missing; on Docker Desktop systems Harbor uses Docker Desktop's Model Runner enablement command when it is available. Harbor then installs the runner runtime when supported, attempts to enable the TCP endpoint on Docker Desktop when HARBOR_DMR_ENABLE_TCP=true, and pulls HARBOR_DMR_MODEL when HARBOR_DMR_AUTO_PULL=true.

Start a frontend against DMR:

harbor up webui dmr

Use it with host tools:

harbor launch --backend dmr --model ai/smollm2 codex

Stopping

harbor down dmr

This stops the Harbor proxy container. harbor dmr stop only logs that Docker Model Runner remains managed by Docker; it does not shut down the host runner.

Configuration

Environment Variables

Following options can be set via harbor config:

# Harbor proxy port
HARBOR_DMR_HOST_PORT              34920

# Proxy image
HARBOR_DMR_IMAGE                  caddy
HARBOR_DMR_VERSION                2-alpine

# Host Docker Model Runner endpoint used by Harbor helpers and proxy
HARBOR_DMR_UPSTREAM_URL           http://model-runner.docker.internal:12434
HARBOR_DMR_RUNNER_PORT            12434

# Default model and host management
HARBOR_DMR_MODEL                  ai/smollm2
HARBOR_DMR_MANAGE_HOST            true
HARBOR_DMR_ENABLE_TCP             true
HARBOR_DMR_AUTO_PULL              true
HARBOR_DMR_API_KEY                sk-dmr

Volumes

The Harbor dmr service mounts only services/dmr/Caddyfile. Model files are managed by Docker Model Runner outside the Harbor service directory.

Model Management

harbor models ls --source dmr
harbor models pull --source dmr ai/smollm2
harbor models rm --source dmr ai/smollm2

The equivalent source-subcommand form is also supported:

harbor models dmr pull ai/smollm2

API

Harbor exposes DMR at:

http://localhost:34920/v1

Containers use:

http://dmr:8080/v1

The proxy rewrites Harbor's /v1/* requests to Docker Model Runner's /engines/v1/* API.

Integrations

dmr is exposed to Harbor containers at http://dmr:8080/v1 (integration API key sk-dmr). Harbor wires the same consumer set as oMLX through compose.x.*.dmr.yml overlays, including webui, chatui, aider, boost, litellm, bifrost, optillm, opint, astrbot, cognee, mindsdb, mi, ml-intern, npcsh, open-design, opennotebook, openterminal, anythingllm, cmdh, hermes, plandex, sillytavern, and traefik when enabled.

harbor up webui dmr

Troubleshooting

harbor dmr status
harbor logs dmr

Docker Model Runner CLI is missing

harbor up dmr attempts automatic setup when host management is enabled. If setup cannot proceed on your platform, install or update Docker Desktop, or install Docker's docker-model-plugin package on Linux, then verify:

docker model --help

Proxy is unhealthy

Check that Docker Model Runner is enabled and reachable at HARBOR_DMR_UPSTREAM_URL. If automatic TCP enablement fails, enable Model Runner from Docker Desktop or adjust HARBOR_DMR_UPSTREAM_URL to the endpoint Docker exposes on your machine.

Links

• Docker Model Runner
• Docker Model Runner API
• Docker Model Runner with Compose