uCore Deployment Strategies

February 19, 2026 · View on GitHub

Overview

This document outlines approaches for deploying configuration changes to running uCore systems after initial provisioning. Since Ignition is provision-time only, ongoing management requires different tools.

Deployment Layers

uCore deployments have three distinct layers, each with different update mechanisms:

┌─────────────────────────────────────────┐
│ Layer 1: OS Image (Immutable Base)     │ → bootc/rpm-ostree
├─────────────────────────────────────────┤
│ Layer 2: Configuration (/etc)          │ → Git-based sync / Ansible
├─────────────────────────────────────────┤
│ Layer 3: Containers (Applications)     │ → Quadlet + auto-update
└─────────────────────────────────────────┘

Inspiration

Based on deuill/coreos-home-server — a production homelab using Git-based automatic configuration sync.

Key Concept: Git repository is the source of truth. Systems pull configuration updates periodically via systemd timer.

Implementation

1. Git Sync Service

Create /etc/systemd/system/home-ops-sync.service:

[Unit]
Description=Sync home-ops configuration from Git
After=network-online.target
Wants=network-online.target

[Service]
Type=oneshot
Environment=GIT_REPO=https://github.com/rwaltr/home-ops.git
Environment=GIT_BRANCH=main
ExecStartPre=/usr/bin/rm -rf /tmp/home-ops-sync
ExecStartPre=/usr/bin/git clone --depth=1 --branch=${GIT_BRANCH} ${GIT_REPO} /tmp/home-ops-sync
ExecStart=/usr/bin/bash /tmp/home-ops-sync/infra/ucore/scripts/apply-config.sh

[Install]
WantedBy=multi-user.target

Create /etc/systemd/system/home-ops-sync.timer:

[Unit]
Description=Sync home-ops configuration hourly

[Timer]
OnCalendar=hourly
RandomizedDelaySec=5m
Persistent=true

[Install]
WantedBy=timers.target

2. Deployment Script

Create infra/ucore/scripts/apply-config.sh:

#!/bin/bash
set -euo pipefail

REPO_ROOT="/tmp/home-ops-sync"
QUADLET_DIR="/etc/containers/systemd"

echo "Deploying configuration updates..."

# Sync Quadlet container definitions
rsync -av --delete "${REPO_ROOT}/infra/ucore/containers/" "${QUADLET_DIR}/"

# Reload systemd to pick up changes
systemctl daemon-reload

# Restart changed services (optional — may cause brief downtime)
for container in "${REPO_ROOT}/infra/ucore/containers/"*.container; do
    service_name=$(basename "$container" .container)
    if systemctl is-active "${service_name}.service" >/dev/null 2>&1; then
        echo "Restarting ${service_name}.service"
        systemctl restart "${service_name}.service"
    fi
done

echo "Configuration deployed successfully"

3. Enable on Host

# Copy units to /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable --now home-ops-sync.timer

# Verify
systemctl status home-ops-sync.timer

Daily Workflow

# 1. Edit configuration locally
vim infra/ucore/containers/rustfs.container

# 2. Commit and push
git add infra/ucore/containers/rustfs.container
git commit -m "Update RustFS configuration"
git push

# 3. Wait for automatic sync (within 1 hour)
# OR trigger manually:
ssh rwaltr@mouse "sudo systemctl start home-ops-sync.service"

# 4. Verify deployment
ssh rwaltr@mouse "systemctl status rustfs.service"

OS Updates

Configure automatic downloads with manual reboot control:

# Enable automatic staging
sudo sed -i 's/^#AutomaticUpdatePolicy=.*/AutomaticUpdatePolicy=stage/' /etc/rpm-ostreed.conf
sudo systemctl enable rpm-ostreed-automatic.timer --now

How it works:

  • Updates download and stage automatically
  • No automatic reboots
  • You control when to apply via systemctl reboot

Check for staged updates:

rpm-ostree status

Apply staged update:

sudo systemctl reboot

Manual Updates

# Check for updates
rpm-ostree upgrade --check

# Download and stage
rpm-ostree upgrade

# Reboot to apply
sudo systemctl reboot

Modern Alternative: bootc

Container-native OS management (available in newer uCore versions):

# One-time switch to bootc
sudo bootc switch ghcr.io/ublue-os/ucore:stable

# Future updates
sudo bootc upgrade
sudo systemctl reboot

Container Updates

Enable Podman's built-in auto-update timer:

sudo systemctl enable --now podman-auto-update.timer

Configure containers for auto-update (add label to Quadlet):

# infra/ucore/containers/rustfs.container
[Container]
Image=docker.io/rustfs/rustfs:latest
Label=io.containers.autoupdate=registry

How it works:

  • Timer runs daily
  • Checks registry for new image digest
  • Pulls updated images
  • Restarts systemd services

Manual trigger:

podman auto-update --dry-run  # Check for updates
podman auto-update            # Apply updates

Manual Container Updates

# Update single container
sudo podman pull docker.io/rustfs/rustfs:latest
sudo systemctl restart rustfs.service

# Update all containers
sudo podman auto-update

Rollback Procedures

OS Rollback

# List deployments
rpm-ostree status

# Rollback to previous deployment
sudo rpm-ostree rollback
sudo systemctl reboot

Container Rollback

# Pull specific version
sudo podman pull docker.io/rustfs/rustfs:0.1.0

# Update container file to pin version
vim infra/ucore/containers/rustfs.container
# Change: Image=docker.io/rustfs/rustfs:0.1.0

# Commit and push
git commit -am "Rollback RustFS to 0.1.0"
git push

# Wait for sync or trigger manually
ssh rwaltr@mouse "sudo systemctl start home-ops-sync.service"

Configuration Rollback

# Revert Git commit
git revert HEAD
git push

# Trigger sync
ssh rwaltr@mouse "sudo systemctl start home-ops-sync.service"

Monitoring & Verification

System Status

# OS deployment status
rpm-ostree status

# Check for staged updates
rpm-ostree upgrade --check

# Container auto-update status
systemctl status podman-auto-update.timer
journalctl -u podman-auto-update.service

# Config sync status
systemctl status home-ops-sync.timer
journalctl -u home-ops-sync.service

Service Health

# List all Quadlet services
systemctl list-units '*.service' | grep -E '(rustfs|netdata)'

# View service logs
journalctl -u rustfs.service -f

# Check container status
podman ps -a

Alternative Approaches

Ansible-Based Management

For multi-host fleets, consider Ansible:

# playbook.yml
- hosts: coreos
  tasks:
    - name: Layer packages
      ansible.builtin.command: rpm-ostree install htop

    - name: Deploy container configs
      ansible.builtin.copy:
        src: containers/
        dest: /etc/containers/systemd/

    - name: Reload systemd
      ansible.builtin.systemd:
        daemon_reload: yes

References:

Custom OCI Images

Build custom OS images with baked-in packages:

# Containerfile
FROM quay.io/fedora/fedora-coreos:stable
RUN rpm-ostree install htop tmux vim && \
    rpm-ostree cleanup -m && \
    ostree container commit

# Build and push
podman build -t ghcr.io/rwaltr/custom-ucore:latest .
podman push ghcr.io/rwaltr/custom-ucore:latest

# Rebase host
sudo rpm-ostree rebase ostree-unverified-registry:ghcr.io/rwaltr/custom-ucore:latest
sudo systemctl reboot

Best Practices

  1. Git as Source of Truth: All configuration changes go through Git
  2. Automatic Staging: Updates download automatically, apply manually
  3. Pin Critical Versions: Use specific tags for production services
  4. Test in VM First: Validate changes with mise run ucore:vm before production
  5. Enable Rollback: Keep previous deployment available (ostree admin pin 0)
  6. Monitor Logs: Use journalctl to track service health
  7. Immutability Matters: Never manually edit files — always update via Git

Implementation Checklist

  • Create infra/ucore/scripts/apply-config.sh
  • Add systemd units to Butane config or manually deploy
  • Enable home-ops-sync.timer on host
  • Enable rpm-ostreed-automatic.timer for OS updates
  • Enable podman-auto-update.timer for container updates
  • Add io.containers.autoupdate=registry label to containers
  • Test deployment workflow in VM
  • Document rollback procedure

References