Long context and prefix reuse

September 1, 2026 ยท View on GitHub

Recommended use
Start withRTX PRO 6000 Blackwell and a Qwen3.8 or Ornith 1.5 configuration sized to the workload
PrioritizeKV capacity, prefix-cache size, session count, and admission behavior
AvoidReserving the model's maximum context when clients never send it
Read nextServing: prompt caching, Performance

Measure the actual prompt and turn shape. A cache-friendly long-running conversation and a fan-out batch over unrelated documents have different useful defaults even at the same token count.