Logs

August 12, 2026 · View on GitHub

The Logs tab streams Caddy's logs into the Ember TUI in real time. A left sidepanel lets you switch between Runtime logs (startup, reloads, TLS, admin API, modules), Access logs (HTTP requests), drill into a specific host within the access view, or pivot to By Route: an aggregated table that groups requests by their normalized URI pattern.

Each line is parsed on the fly and kept in an in-memory ring buffer of the last 10 000 entries per stream (access and runtime are held in separate buffers so a busy server's access traffic cannot evict rare runtime lines).

How it works (zero-config)

At startup, Ember:

  1. Binds a single TCP listener (by default on a free loopback port).
  2. Registers two sinks in Caddy pointing at that listener:
    • __ember__, scoped to http.log.access (access logs);
    • __ember_runtime__, excluding http.log.access (everything else).
  3. Enables access logging on every server that did not already have a logs block (runtime logs flow unconditionally, so no equivalent step is needed).
  4. Starts parsing incoming lines, routing each entry to the matching buffer by its logger field.

If Caddy is not yet reachable at startup, the listener stays open and a background watchdog retries every 30 seconds until it succeeds. The watchdog also re-registers both sinks if Caddy is reloaded (caddy reload, API-driven config push, etc.), so log streaming resumes without user intervention.

At clean shutdown, Ember unregisters both sinks and restores the access-logs config only on the servers it modified (a GET check prevents clobbering config the user or another tool may have added in the meantime).

The end result: a stock Caddyfile with no log directive still produces live access and runtime logs in Ember's TUI, and Caddy's persistent config ends the session exactly where it started.

Local vs remote Caddy

ScenarioCommand
Caddy on the same hostember
Caddy over a Unix socketember --addr unix//path/to/admin.sock
Caddy on a remote hostember --addr http://remote:2019 --log-listen :9210
Caddy in Docker (macOS/Windows)ember --log-listen host.docker.internal:9210
Caddy in Kubernetes Pod (CLI)kubectl logs -f <pod> | ember --stdin-logs --addr http://localhost:2019

When --addr points at a non-local host, Ember does not auto-bind a listener: a 127.0.0.1:<port> address would not be reachable from the remote Caddy process. In that case, pass --log-listen <addr> with an address Caddy can reach and the same behaviour applies. Set EMBER_LOG_LISTEN if you'd rather configure it via environment.

When the hostname in --log-listen cannot be resolved locally (e.g. host.docker.internal), Ember binds on 0.0.0.0:<port> instead and advertises the original address to Caddy. This lets a containerised Caddy reach the host without extra networking setup.

Reading from Stdin (Kubernetes CLI Monitoring)

In environments like Kubernetes where egress is restricted, or when you want to monitor a remote Pod locally without changing Caddy's configuration, you can use the --stdin-logs (or --from-stdin) flag.

This enables a completely unidirectional mode where:

  1. Ember does not attempt to register any network sinks in Caddy.
  2. Ember reads JSON logs directly from its standard input (stdin).

Step-by-Step Guide

  1. Port-forward Caddy's Admin API to your local machine so Ember can fetch metrics and configurations:

    kubectl port-forward pod/my-caddy-pod-abcde 2019:2019
    
  2. Stream Pod logs and pipe them into Ember on your local machine:

    kubectl logs -f pod/my-caddy-pod-abcde -c caddy | ember --stdin-logs --addr http://localhost:2019
    

    (Note: The -c caddy flag specifies the container name if you are running Caddy alongside other containers in a sidecar pattern).

Why use this approach?

  • No Side-Effects: Since Ember reads from stdin, it does not alter Caddy's log writers, making it perfectly safe for read-only production environments.
  • Full TUI Features: You still get the full terminal UI experience including real-time access logs, runtime logs, and the By Route aggregated statistics.

Hot-registered sinks

PUT /config/logging/logs/__ember__
{
  "writer":  { "output": "net", "address": "tcp/HOST:PORT", "soft_start": true },
  "encoder": { "format": "json" },
  "include": ["http.log.access"]
}

PUT /config/logging/logs/__ember_runtime__
{
  "writer":  { "output": "net", "address": "tcp/HOST:PORT", "soft_start": true },
  "encoder": { "format": "json" },
  "exclude": ["http.log.access"]
}

Notes:

  • Both sinks push to the same TCP listener; Caddy opens one connection per sink and Ember routes the entries by logger name.
  • soft_start: true means Caddy never blocks if the listener is briefly unavailable.
  • include on __ember__ and exclude on __ember_runtime__ keep the routing symmetric: every log line reaches exactly one Ember buffer.
  • The net writer reconnects on its own when Ember restarts.

Sidepanel

The left column is a tree:

Runtime
Access
  api.example.com
  static.example.com
  ...
By Route
  • Runtime shows everything Caddy logs that is not an HTTP access entry: startup, reload, TLS handshakes, admin API, plugin logs.
  • Access shows all HTTP access logs across every host.
  • The children under Access are the hosts actually seen in recent traffic, sorted alphabetically. Selecting one narrows the table to that host and drops the Host column so URIs get more room.
  • By Route swaps the log table for an aggregated view: one row per (host, method, normalized URI pattern) bucket. See the dedicated section below. Like Access, "By Route" has per-host children so you can drill into a single virtual host.

Selecting an entry resumes live-follow mode so you always see fresh data when you drill in. The filter (typed with /) composes with the sidepanel: type 500 while on api.example.com to see only 5xx responses for that host.

Reading the table

Access view (Access aggregate or per-host):

ColumnDescription
TimeLocal time the log entry was emitted, millisecond precision
CodeHTTP status code, color-coded (green 2xx, orange 4xx, red 5xx)
MethodHTTP method
HostValue of request.host (hidden in per-host view)
DurationServer-side processing time in milliseconds
URIValue of request.uri, truncated to fit

Runtime view:

ColumnDescription
TimeLocal time the log entry was emitted, millisecond precision
LevelLog level, color-coded (red ERROR/FATAL, orange WARN). A textual prefix doubles the cue so ERROR rows start with ! and WARN rows with *, keeping severity scannable when NO_COLOR is set
LoggerCaddy logger name (tls.handshake, admin.api, ...)
MessageThe log message

Lines that fail to parse as JSON are still shown in grey in the runtime view, so corrupt or mid-write lines never silently disappear.

By Route view

Selecting By Route in the sidepanel replaces the log table with an aggregated view of every request Ember has seen this session (the counts are kept independently of the access ring buffer, so they keep climbing even after the 10 000-entry cap is reached). Each row is one (host, method, pattern) bucket: GET /users/:id on api.localhost and GET /users/:id on app.localhost are different rows because they hit different handlers and have different latency profiles.

On the root By Route view, the host is folded into the Pattern column as a soft prefix (api.localhost /users/:id) so the two are distinguishable at a glance. Drilling into a per-host child filters the table on that host and drops the prefix.

ColumnDescription
CountNumber of requests in the bucket
MethodHTTP method
PatternThe normalized URI pattern (see below)
2xx / 3xxCounters per status class; 2xx is rendered green
4xx / 5xxCounters per status class; coloured (orange / red) and suffixed with * (4xx) or ! (5xx) so error rows stay scannable when NO_COLOR is set
AvgMean latency over the bucket
MaxSlowest single request
Avg MemMean PHP memory usage sampled from busy FrankenPHP threads serving this route (FrankenPHP only, see below)
Max MemHighest sampled PHP memory usage for this route (FrankenPHP only, see below)

Press s / S to cycle the sort field (Count -> Pattern -> Avg -> Max -> Avg Mem -> Max Mem), following the visual column order. The Avg Mem and Max Mem steps are skipped whenever those columns are off screen, so the cycle never sorts on a column you cannot see. The active column is marked with ▼ in the header, the same glyph the host and upstream tables use, so the cue is consistent across the app. / filters on method or pattern.

Memory columns

The Avg Mem and Max Mem columns aggregate the same per-thread memory usage the FrankenPHP tab shows live, keyed by (method, pattern): handy for spotting memory-hungry routes, catching leaks in worker mode, or sizing servers and pods from the max footprint.

They are drawn under two conditions. First, at least one route currently in view must carry a memory sample, which needs a FrankenPHP server running 1.12.2 or later (older versions report no per-thread memory usage, like the per-thread metrics of the FrankenPHP tab); a filter or a per-host drill-down matching only never-sampled routes hides them rather than showing two columns of dashes. Second, there must be room: the two columns cost 20 cells, and they are only drawn when that comes out of a wide terminal's slack rather than out of the Pattern column, which is the only one identifying a route. In practice that means a terminal of roughly 153 columns or more.

While the columns are off screen their two sort steps are skipped, and an Avg Mem / Max Mem sort already in effect is simply not applied: the table falls back to Count, the header marks it, and your choice comes back as soon as the columns do.

Four caveats stem from how the data is collected:

  • Values are sampled each time Ember polls a busy thread, not measured per request: a request that completes between two polls is never sampled (its row shows —), and a long request is sampled repeatedly.
  • The sample is the thread's PHP memory usage, not the request's. In worker mode a thread is reused across requests, so what it retains from earlier work is attributed to the route it happens to be serving when polled. Read the columns as "how much memory a thread holds while serving this route", and compare routes over many samples rather than trusting a single peak.
  • Thread states do not carry a host, so two buckets sharing the same (method, pattern) show identical memory values. This applies to the root view and to per-host drill-downs: a peak displayed under one host may have been reached while another vhost was being served.
  • Samples only arrive from the admin-API poll, so they freeze while that poll is paused or while Ember cannot reach the endpoint, whereas Count, Avg and Max keep advancing from the log stream. A By Route row can therefore pair live latency with stale memory figures. Note that p does not pause the poll from the Logs tab: it is a no-op in the By Route view, and freezes the tail in the other log views.

URL normalization

To keep the bucket count manageable, segments that look like dynamic identifiers are collapsed:

Segment shapeBecomes
550e8400-e29b-41d4-a716-446655440000 (UUID):uuid
0123456789abcdef0123… (16+ hex chars, sha-like):hash
12345 (all digits):id

Query strings and fragments are dropped before normalization (they belong on the request, not on the route).

The rules are intentionally narrow: anything that does not match leaves the segment alone (slugs, words, short hex strings…). A custom segment that collides with one of the rules, for example /dunning/{1,2,3,4}, will appear as :id, which is a known limitation of pattern-free heuristics. If you hit a case worth handling, open an issue with a representative URL and we will look into adding a rule or making the set configurable.

Scroll modes

The table has two states:

  • Following (default): pins the newest entry at the top and redraws as lines arrive.
  • Frozen: stops sliding so you can read a specific line without having it pushed down. The full buffer at freeze time is available, so you can scroll well past the initial viewport to inspect older entries. A pill on the right of the column header shows ● PAUSED plus how many new lines have been captured in the background.

Entering Frozen mode happens either implicitly when you scroll (↑, ↓, PgUp, PgDn, End) or explicitly by pressing p. Resume live follow with f (or Home, or p again). Switching sidepanel selection also resumes live mode so the frozen snapshot does not get out of sync with the visible buffer.

The header also surfaces a dropped: N chip once the in-memory ring buffer wraps. It is a reminder that the tail window holds the most recent 10 000 entries per scope, not the full history: any older lines have been evicted to keep the memory footprint bounded.

Keybindings

Focus is on the sidepanel by default when entering the tab.

KeyAction
← / hMove focus to the sidepanel
→ / l / EnterMove focus back to the table
↑ / ↓Navigate the focused panel. On the table, auto-freezes on first press from live
PgUp / PgDnPage up/down in the table (also auto-freezes)
EndJump to the oldest entry in the frozen snapshot (table), or the last sidepanel item
HomeFirst sidepanel item (sidepanel focus); resume follow (table focus)
fResume live follow (table focus)
/Filter: matches case-insensitively across all visible columns
pToggle pause: freezes or resumes the table
s / SCycle sort field (By Route view only)
cClear the current buffer (also resumes live follow)
TabSwitch tab
?Help overlay
qQuit

Jumping from the Caddy tab

While on the Caddy tab, press l on a host to switch to the Logs tab with the sidepanel pre-selected on that host's access entries. Go back to the aggregate view with ← then ↑ to navigate the sidepanel.

What happens if Ember crashes or is killed

Ember removes both sinks and the access-logs config it added at clean exit. Best-effort against unexpected exits:

EventBehaviour
q, Ctrl+C, normal quitDeferred cleanup runs ✓
Go panicDeferred cleanup runs (panics unwind defers) ✓
SIGTERM (systemctl stop)Trapped, forwarded to clean shutdown ✓
caddy reload / config pushWatchdog re-registers both sinks + access-logs within 30 s ✓
Caddy starts after EmberWatchdog activates streaming within 30 s ✓
SIGKILL, OOM kill, power lossSinks and auto-enabled access-logs blocks remain: see below

If a SIGKILL-class event leaves state behind, Caddy keeps the registrations but the writers use soft_start: true, so they do not block reloads or spam errors. To clean up:

  • Run ember again: Ember uses idempotent PUTs to register its sinks, overwriting any stale entries. The next clean quit then removes everything.

  • Or remove them manually:

    curl -X DELETE http://localhost:2019/config/logging/logs/__ember__
    curl -X DELETE http://localhost:2019/config/logging/logs/__ember_runtime__
    # plus, for each server Ember may have touched:
    curl -X DELETE http://localhost:2019/config/apps/http/servers/<srv>/logs
    

What this feature does NOT do

  • It does not write logs to the JSON output, daemon mode, or Prometheus exporter. Log streaming is a TUI-only convenience.
  • The runtime view's INFO-and-above firehose can be noisy under high load. Use / to filter (for example to error) if the list moves faster than you can read.