Services Architecture and File Conventions

May 16, 2026 · View on GitHub

This directory contains all Docker Compose configurations and deployment artifacts for devbox services. This guide explains the layout, conventions, and how files are generated and deployed.

Tarball installs: The contents of this directory (services/), together with scripts/, setup.sh, docs/, README.md, and LICENSE, are bundled verbatim into the signed weekly release tarball (devbox.tar). Operators who install from a release tarball get the same lockfiles and setup.sh as those who clone the repo — no extra steps are needed. See docs/updating.md for the full verified download and install sequence.

Directory Layout

services/
├── README.md                       # This file
├── traefik/                        # Reverse proxy + docker-socket-proxy
│   ├── docker-compose.yml          # Base HTTP configuration
│   ├── docker-compose.https.yml    # HTTPS override (optional)
│   ├── traefik.yml                 # Static HTTP-only config (tracked)
│   ├── traefik.https.yml.template  # Install-time template (HTTPS-mode overwrites traefik.yml)
│   ├── .env.template               # Install-time template (rendered to .env, HTTPS mode only)
│   ├── dynamic/
│   │   ├── dashboard-auth.yml.template     # Basic auth for Traefik dashboard
│   │   ├── ollama-auth.yml.template        # Basic auth for ollama.internal
│   │   ├── middlewares-base.yml            # Security headers, rate limiting
│   │   ├── middlewares-https.yml           # HSTS (HTTPS only)
│   │   ├── middlewares-rate.yml            # Rate limit config
│   │   └── middlewares-allowlist.yml       # IP allowlist (Tailscale)
│   └── letsencrypt/                # ACME certs directory (operator-created at runtime)
└── ollama-openwebui/
    ├── docker-compose.yml          # Base, includes Ollama Traefik router
    ├── docker-compose.https.yml
    └── .env.template

# docker-compose.lock.yml files (digest pinning) are committed under
# services/<svc>/. setup.sh appends :docker-compose.lock.yml to the
# emitted COMPOSE_FILE_* chain when the lockfile is present in the source
# tree. Regenerate with scripts/update-images.sh --apply; verify with
# scripts/update-images.sh --check.

File Types

1. `docker-compose.yml` (Base, Tracked)

The base Compose file for HTTP-only deployment. Contains service definitions with mutable minor-pinned tags (e.g., traefik:v3.2).

Never: Pin by digest in the base file. Tags are intentionally mutable to allow weekly CI refresh.

2. `docker-compose.https.yml` (Override, Tracked)

Compose override file loaded only when ENABLE_HTTPS=true. Adds port 443, ACME configuration, and TLS settings.

Invoked via: COMPOSE_FILE=docker-compose.yml:docker-compose.https.yml:docker-compose.lock.yml

Rationale: ARCHITECTURE.md — eliminates duplication of security headers and middleware between HTTP and HTTPS modes.

3. `docker-compose.lock.yml` (Generated, Committed)

Generated by: docker compose config --lock-image-digests

Contains immutable image digests. Every image reference is rewritten from tag to digest (e.g., traefik@sha256:abc123...).

Generated twice on initial commit (Critic Issue #2):

First run: emits traefik:v3.2@sha256:...
Second run: emits traefik@sha256:... (stable, committed)

Updated by: ./scripts/update-images.sh --apply (runs weekly CI or manually)

Rationale: ARCHITECTURE.md — provides reproducible, auditable image deployment.

4. `.template` Files (Install-Time Templates)

Files with .template suffix undergo whitelisted install-time envsubst substitution. Only explicitly whitelisted variables are replaced.

Rendered by: render_env() in setup.sh

render_env "services/traefik/dynamic/dashboard-auth.yml.template" \
           "~/docker/traefik/dynamic/dashboard-auth.yml" \
           '$TRAEFIK_USER $TRAEFIK_HASH'

Only writes if destination absent: Existing files are never clobbered.

Rationale: ARCHITECTURE.md — prevents accidental baking of runtime variables at install time.

5. Non-Template Files (Runtime Interpolation)

Files WITHOUT .template suffix use Compose runtime interpolation from .env files. Variables remain as ${VAR} and are resolved at container-start time.

Critical: Do NOT run naked envsubst on non-template files.

Per-File X-Logging Anchor Pattern

Every docker-compose.yml contains a canonical 4-line x-logging anchor:

x-logging: &default-logging
  driver: json-file
  options:
    max-size: "10m"
    max-file: "3"

Duplication is by design:

YAML anchors are per-file; cross-file inclusion doesn't work (empirically tested)
4 lines × 3 files = 12 lines total; acceptable
CI drift check (scripts/ci/check-anchor-consistency.sh) enforces consistency

When updating: Change must be applied to all services/*/docker-compose.yml files in a single commit.

Rationale: ARCHITECTURE.md — per-file duplication with CI drift detection is the only YAML-spec-compliant approach.

Rendering Table (Install-Time Templates)

Every .template file is listed here with its whitelist.

Template File	Destination	Whitelist	Notes
`traefik/traefik.https.yml.template`	`~/docker/traefik/traefik.yml`	`$USER_EMAIL`	Traefik static config (HTTPS mode overwrites HTTP-only `traefik.yml`)
`traefik/dynamic/dashboard-auth.yml.template`	`~/docker/traefik/dynamic/dashboard-auth.yml`	`$TRAEFIK_USER $TRAEFIK_HASH`	Basic auth for Traefik dashboard
`traefik/dynamic/ollama-auth.yml.template`	`~/docker/traefik/dynamic/ollama-auth.yml`	`$OLLAMA_AUTH_USER $OLLAMA_AUTH_HASH`	Basic auth for `ollama.internal`; plaintext credential at `~/docker/.secrets/ollama-auth.txt`
`traefik/.env.template`	`~/docker/traefik/.env`	`$OVH_ENDPOINT $OVH_APPLICATION_KEY $OVH_APPLICATION_SECRET $OVH_CONSUMER_KEY`	OVH API credentials (HTTPS only)
`ollama-openwebui/.env.template`	`~/docker/ollama-openwebui/.env`	`$WEBUI_SECRET_KEY $ENABLE_SIGNUP`	Web UI session secret + signup toggle

Verification: bash scripts/ci/lint.sh validates this table against actual .template files.

HTTPS Base + Override Pattern

HTTP-only deployment:

COMPOSE_FILE=docker-compose.yml:docker-compose.lock.yml
docker compose up -d

HTTPS deployment:

COMPOSE_FILE=docker-compose.yml:docker-compose.https.yml:docker-compose.lock.yml
docker compose up -d

Rationale: ARCHITECTURE.md — override pattern eliminates duplication while maintaining DRY principle.

Image Digest Pinning Workflow

Initial Commit (Two-Pass Generation)

# First run
docker compose -f services/traefik/docker-compose.yml config --lock-image-digests > /tmp/lock1.yml

# Second run (stabilized, committed)
docker compose -f /tmp/lock1.yml config --lock-image-digests > services/traefik/docker-compose.lock.yml

# Verify stable
docker compose -f services/traefik/docker-compose.yml -f services/traefik/docker-compose.lock.yml config --lock-image-digests | diff -u services/traefik/docker-compose.lock.yml -
# Should be empty diff

Weekly Updates (CI Workflow)

./scripts/update-images.sh --check  # Check for updates
./scripts/update-images.sh --apply  # Apply updates
git add services/ && git commit -m "Refresh digests ($(date +%Y-%m-%d))"

Tag Pinning Strategy

Tags in base docker-compose.yml are minor-pinned (e.g., traefik:v3.2):

Weekly CI refresh pulls latest patch within the minor
Major bumps (v3 → v4) are manual edits
Inspect available majors manually via docker manifest inspect <image>:<tag> or upstream release notes

Rationale: ARCHITECTURE.md — automatic security patches, manual control of breaking changes.

Contributing to services/

Adding a New Service

Create directory: mkdir services/myservice
Write docker-compose.yml with mutable minor-pinned tag
Create .env.template if needed; add to rendering table
Create docker-compose.https.yml if HTTPS-specific config needed
Validate: docker compose -f services/myservice/docker-compose.yml config -q
Generate lockfile: docker compose config --lock-image-digests (twice)
Update this README's layout diagram and rendering table
Commit: one service per commit

Updating an Existing Service

For patch/minor updates:

./scripts/update-images.sh --apply
git add services/ && git commit -m "Refresh digests (YYYY-MM-DD)"

For major version bumps:

# Edit tag in docker-compose.yml
vim services/myservice/docker-compose.yml
./scripts/update-images.sh --apply
git add services/myservice && git commit -m "Bump myservice to v2.0"

Verification

# Validate Compose syntax (HTTP + HTTPS modes)
docker compose -f traefik/docker-compose.yml -f traefik/docker-compose.lock.yml config -q
docker compose -f traefik/docker-compose.yml -f traefik/docker-compose.https.yml -f traefik/docker-compose.lock.yml config -q

# Run all CI checks
./scripts/ci/lint.sh
./scripts/ci/check-anchor-consistency.sh
./scripts/ci/check-compose-config.sh

References

ARCHITECTURE.md: Services extraction rationale
ARCHITECTURE.md: HTTPS override pattern
ARCHITECTURE.md: Image digest pinning
ARCHITECTURE.md: Per-file logging anchor pattern
ARCHITECTURE.md: Tag pinning strategy
ARCHITECTURE.md: Template rendering discipline
Docker Compose: Specification