Backend Routing: evidence-cited defaults, never a cage

Twin: examples/sdk_optimizer_profile_matrix.py · emits agent-learning.optimizer-routing-table.v1 · offline, no credentials. A coding agent can complete this page from the frontmatter alone.

1. What you are testing

When you omit the optimizer in an optimization manifest, something picks a backend for you. The routing table makes that pick auditable: rows keyed by (target_kind, framework_profile) recommend a backend, and every recommendation must cite at least one same-run matrix cell whose axes match and whose winner equals the recommendation. The table is generated by the matrix example from its own cells, committed at examples/optimizer_routing_table.json, and byte-compared by the release gate — a routing change is a visible diff backed by cell evidence, the same discipline the docs machine index uses.

The evidence cited is the trajectory fitness profile of each backend run — improvement frequency, semantic locality, dedupe rate, regression count, iterations, evaluations — computed post-hoc from optimization history: trajectory shape, not endpoint score, as routing evidence. Live-lane-classed evidence is recorded but excluded from recommendation computation: a release-admissible row may cite local_gate or captured_fixture evidence only.

Archetype	Dharma (what it may change)	Constraint (what it must preserve)
Transformer (proposer)	candidate config: harness, memory, tooling	the task contract
Critic	scores + objections	evidence admissibility
Mediator	candidate retention/merge	lineage continuity
Steward (preserver)	rollback / veto	governance + the regression baseline

The engagement contract: an omitted optimizer consults the table by default (selected_by: "routing_table", citations recorded); an explicit optimizer= (SDK) or --backend <name> (CLI) always overrides, with the spurned recommendation kept visible (selected_by: "override"); a missing row falls back to the static default with selected_by: "cold_start", empty citations, and a warning — exit 0, never an error.

2. Run it

CLI — regenerate the table from the matrix cells (env value is a local placeholder), then run an optimization with an explicit override:

AGENT_LEARNING_SDK_OPTIMIZER_PROFILE_MATRIX_KEY=local-dev-key \
  python examples/sdk_optimizer_profile_matrix.py \
  artifacts/optimizer-profile-matrix.json

agent-learn optimize examples/optimization_manifest.json \
  --backend tpe --output artifacts/override-optimization.json

SDK — the default picker and the table builder:

from fi.alk import optimize

# Omitted optimizer: the committed routing table picks, with citations.
manifest = optimize.build_target_optimization_manifest(
    name="routed-target",
    base_config=base_config,
    target_candidates=candidates,
    evaluation_config=evaluation_config,
    target_metadata={"task_kind": "prompt", "framework_profile": "llamaindex"},
)
evidence = manifest["optimization"]["optimizer_routing_evidence"]
assert evidence["selected_by"] in {"routing_table", "cold_start"}

# Rebuild the table from optimization artifacts carrying trajectory profiles.
table = optimize.build_optimizer_routing_table(artifacts)
rendered = optimize.render_optimizer_routing_table_json(table)
assert optimize.routing_table_matches_committed(table)

3. What you built

python -c "import json; p=json.load(open('artifacts/optimizer-profile-matrix.json')); t=p['routing_table']; assert t['kind']=='agent-learning.optimizer-routing-table.v1', t['kind']; assert all(row['evidence'] for row in t['rows'] if row['recommended_backend']); print('ok')"

The table carries rows[] of {target_kind, framework_profile, recommended_backend, evidence[], live_lane_evidence[]} plus the admissible evidence-class list. A row missing for a key means "no evidence" — never "use the overall best": the schema has no global aggregate, and the release gate fails if one appears.

4. When it fails

Symptom	First-mile class	Doctor check
routing byte-compare fails in release-check	matrix cells changed without recommitting the table	missing_engine_modules
selected_by: "cold_start" with a warning	no row for this (target_kind, framework_profile) — add cells, not defaults	missing_engine_modules
recommendation without citations	evidence entries lost their cell_ref/backend axes	missing_engine_modules
Set AGENT_LEARNING_SDK_OPTIMIZER_PROFILE_MATRIX_KEY...	missing local placeholder env	api_key_configured

5. Prove it / keep it

The optimizer_profile_matrix_readiness gate asserts the routing evidence in its routing_errors: byte-identity with the committed table, per-row citations, live-lane exclusion, and the default/override/cold-start engagement contract. The cells behind every recommendation live in the Optimizer Profile Matrix; the portfolio-level backend evidence lives in Optimizer Portfolio.