LLM Providers

September 16, 2026 ยท View on GitHub

< Docs index | Project README

Contents

Using Claude Code Wrapper (Max Subscription)

Instead of using API credits, you can use the Claude Code OpenAI Wrapper to use your Claude Max subscription instead.

Quick Start:

  1. Start the wrapper service:

    docker compose --profile wrapper up -d

  2. Authenticate with Claude (first time only):

    docker compose --profile wrapper run --rm claude-wrapper claude auth login

  3. Configure minuspod to use the wrapper by updating your .env:

    LLM_PROVIDER=openai-compatible
OPENAI_BASE_URL=http://claude-wrapper:8000/v1
OPENAI_API_KEY=not-needed

  4. Restart minuspod:

    docker compose up -d minuspod

Other OpenAI-Compatible Endpoints:

The openai-compatible provider can work with other endpoints by configuring OPENAI_BASE_URL and OPENAI_API_KEY accordingly. Select models under Settings > AI & Processing > AI Models, or use OPENAI_MODEL to seed unset model settings.

Example .env for OpenAI-compatible mode:

# LLM Configuration (OpenAI-compatible)
LLM_PROVIDER=openai-compatible
OPENAI_BASE_URL=http://claude-wrapper:8000/v1
OPENAI_API_KEY=not-needed

# Server Configuration
BASE_URL=http://localhost:8000

OPENAI_MODEL is optional. It seeds any unset detection, verification, and chapters model; later Settings choices take precedence.

Using Ollama (Local or Cloud)

Ollama is an alternative to the Anthropic API. MinusPod supports both flavors:

  • Local (http://host:11434): no auth, no API costs, nothing leaves the machine.
  • Cloud (https://ollama.com/api): same OpenAI-compatible endpoints, just with Authorization: Bearer <key> on every request. Free tier works for this pipeline. Grab a key at ollama.com/settings/keys.

Configuration is identical either way: pick Ollama in Settings > LLM Provider, set the Base URL, and (for Cloud) paste the key. The key is stored encrypted. Leave it blank for local.

A Test connection button sits under the Base URL field for the Ollama and OpenAI Compatible providers. It hits the same /models route MinusPod uses to discover models (with the same automatic /v1 suffix for Ollama), works without an API key, and tells you whether the failure is an unreachable host, a wrong path, or a rejected key. You can test a URL before saving it; a saved API key is only sent when the tested URL matches the saved one.

Anthropic and OpenRouter get the same button. Their endpoints are fixed, so the test answers a different question: can this container reach the provider at all, and if not ok, is the problem the network or the saved key.

Heads up about Ollama Cloud model selection

Ollama Cloud's /v1/models advertises the full Ollama library, including previews and local-only tags that Cloud won't actually route. The dropdown shows whatever the endpoint returns, so entries like gemma4:31b, kimi-k2:1t, and gpt-oss:120b can appear but 404 when called.

If an episode processes with zero ads and the logs show {"type":"error","error":{"type":"not_found_error"}}, the model isn't really on Cloud. The reliable list is at ollama.com/search?c=cloud. Cross-check the base name (before the :) before saving.

Setup

  1. Install and start Ollama on your host machine
  2. Pull a model (see recommendations below): ollama pull qwen3:14b
  3. Update your docker-compose.yml:
environment:
  - LLM_PROVIDER=ollama
  - OPENAI_BASE_URL=http://host.docker.internal:11434/v1
  - OPENAI_MODEL=qwen3:14b

Linux users: host.docker.internal doesn't resolve by default on Linux. Add extra_hosts: ["host.docker.internal:host-gateway"] to your Docker service definition.

The OPENAI_API_KEY variable is not required for Ollama. Token counts will still be tracked in the UI but cost will always show as $0.00, which is accurate since local inference is free.

Note: LLMs are non-deterministic. The same prompt against the same model can yield different ads on different runs. The picks below are where I'd start, not where I'd stop tuning.

Cloud LLMs (benchmark-tested)

These come from the offline LLM benchmark included in this repo. The benchmark runs each candidate model against a corpus of human-verified episodes and scores accuracy (F0.5 and F1 at IoU >= 0.5), JSON compliance, latency, and per-episode cost. Full per-model breakdown (precision, recall, boundary accuracy, calibration, latency tail, token efficiency, cross-model agreement) is in benchmarks/llm/results/report.md. Want to expand the corpus or test more models? See benchmarks/llm/CONTRIBUTING.md.

The report ranks by F0.5, which weights precision twice as heavily as recall. The pipeline cuts the segments it flags, so a false positive (cutting real content) is a worse failure than a missed ad. F1 is shown next to it.

Use caseModelF0.5F1Cost / episodeWhy
Best accuracy overallclaude-haiku-4-5 (Anthropic direct)0.9080.920$1.08Tops the table with perfect JSON compliance and a clean pass on both no-ad controls. p50 latency is 24.2s, which is fine for MinusPod's offline pipeline but rules out interactive use. The top tier also holds qwen/qwen3.5-plus-02-15, google/gemini-3.5-flash-lite, claude-sonnet-4-6, the Gemini Flash line, the Grok line, and mistralai/mistral-medium-3-5; they trade wins across episodes, so pick among them on speed and cost. Between the two Gemini Flash entries, 3.7 costs $0.64 per episode against 3.6's $1.44 while statistically tied, so there is no reason to pick 3.6.
Fast and reliablegoogle/gemini-3.5-flash-lite (via OpenRouter)0.8600.851$0.36p50 0.7s, perfect JSON compliance, clean on both no-ad controls, and third overall, inside the top tier at a tenth of the frontier price. The production pick when you want speed and low cost without giving up accuracy.
Best Anthropic-direct at low latencyclaude-sonnet-4-60.8590.866$3.24Fourth overall, p50 4.1s versus Haiku's 24.2s, perfect JSON compliance, clean on both no-ad controls. For Anthropic-key users who want faster turnaround than Haiku 4.5; both Opus 4.8 ($5.37/episode) and Opus 5 ($5.38/episode) score lower.
Cheapest viableqwen/qwen3.7-flash (via OpenRouter)0.7670.774$0.07Best F0.5 per dollar (10.58) among models with no reliability flags. JSON compliance 0.96 and clean on both no-ad controls. p50 10.1s. A few cheaper models rank higher on raw F0.5/$, but all of them either false-positive on the no-ad controls or return brittle JSON.
Free while it lastsstealth/ox-alpha* (via OpenRouter)0.8060.810$0.00*One tier below the leaders at no cost today, and clean on both no-ad controls. Two things to weigh against that: p50 latency is 66.6s, the slowest in its tier by roughly an order of magnitude, and it is verbose enough that 6.4% of calls hit the token ceiling, giving a 5.7% parse-failure rate and 0.94 JSON compliance. Worth trying if your pipeline is patient and you can absorb a few dropped windows.

* stealth/ox-alpha is a cloaked listing: an unannounced model published under a placeholder name, free during the preview. Treat the $0.00 as promotional, not structural. When it ships under its real name the slug disappears and pricing appears, so anything depending on it should have a fallback configured. Cloaked listings also generally allow provider-side prompt logging, which matters if your transcripts are sensitive.

Caveats:

  • Corpus size, trial counts, statistical tiering, and how F0.5 and F1 are scored are covered in benchmarks/llm/results/report.md, linked above.
  • The sweep ran against a frozen system prompt, benchmarks/llm/prompts/2026-08.txt, not the live one. Reproducing these numbers needs that file, via benchmark run --snapshot.
  • Provider-side content moderation can take episodes out of a model's reach entirely. One roster model was refused on 15.2% of its calls because the provider's filter blocked explicit transcripts, losing a quarter of the ads in the two affected episodes. If your library includes explicit shows, check benchmarks/llm/results/parse-and-moderation.md before picking a model.

Local Ollama Models (by VRAM tier)

Note on benchmarking: the offline benchmark in benchmarks/llm/ covers cloud-hosted models (OpenRouter, Anthropic direct). Local Ollama runs are not in that sweep. Adding an Ollama provider would let contributors compare local quants apples-to-apples against the cloud numbers. The recommendations below come from author testing, not a structured benchmark, so verify them on your own content.

Models are loaded sequentially, not concurrently; VRAM requirements are not additive between passes.

Pass 1: First Pass Detection

Hardest task. Contextual reasoning, host-read ads, new sponsors. Use your best model here.

VRAMModelQuantizationNotes
8GBqwen3:8bQ4_K_MEntry level. Handles standard sponsor reads well.
12GBqwen3:14bQ4_K_MBest quality-to-VRAM ratio. Recommended.
16GBqwen3:14bQ5_K_MHigher quality quant; use if you have headroom.
24GBqwen3.5:27bQ4_K_MStrong contextual reasoning. 256K context.
24GBqwen3.5:35bQ4_K_MBest quality under 40GB. 256K context.
40GB+qwen3.5:122bQ4_K_MAuthor's best local option for hard cases. Not yet measured in the cloud benchmark.
Verification Pass

Easier task. Looks for remnants in already-cut audio. Speed matters more than raw accuracy.

VRAMModelQuantizationNotes
8GBqwen3:4bQ8_0Fast, good JSON compliance. Verification prompt is simpler.
12GBqwen3:8bQ5_K_MStrong JSON compliance, faster than 14B.
16GBmistral-nemo:12bQ4_K_MExcellent JSON reliability, fast inference.
24GBqwen3:14bQ5_K_MOverkill for verification but uses available VRAM productively.
Chapters

Simplest task. Summarization only, no structured detection. Minimize VRAM usage and latency.

VRAMModelQuantizationNotes
Anyqwen3:4bQ4_K_MSufficient for summarization. Fast.
Anyphi4-miniQ4_K_MLean alternative, strong instruction following.
Anyllama3.2:3bQ4_K_MSmallest viable option if VRAM is tight.

Example split for 16GB VRAM: Pass 1 -> qwen3:14b Q5_K_M / Verification -> qwen3:8b Q5_K_M / Chapters -> qwen3:4b Q4_K_M

Avoid models under 7B for production use. JSON reliability drops sharply at smaller sizes, turning recoverable errors into silent detection failures. See JSON Reliability Risks.

Cloud vs. Local: What Changes

The LLM only sees host-read ads that blend into content, new sponsors not yet in the pattern database, and ambiguous mid-rolls without promo codes or URLs. Everything else (audio fingerprinting, text pattern matching, pre/post-roll heuristics, audio-signal enforcement) runs without an LLM and catches a substantial share of ads regardless of model.

Content typeCloud-vs-local impact
Standard sponsor reads with promo codes / vanity URLsMinimal: patterns and fingerprinting cover most of these without the LLM
Heavy host-read or conversational ad integrationsNoticeable: requires strong contextual reasoning
Network-inserted brand-tagline ads (no promo code, no URL)Moderate: these are usually under 30 seconds, and short ads are the weakest detection bucket for most models in the cloud benchmark, so don't expect local to outperform
New sponsors not in the pattern databaseModerate: depends heavily on model capability

qwen3:14b locally is fine for standard sponsor reads. The gap to cloud-frontier shows up on conversational ad reads that lack clear transitions. To measure the gap on your own content, capture the episode (see benchmarks/llm/) and compare predictions against your verified ground truth.

JSON Reliability Risks

MinusPod's ad detection pipeline requires models to return structured JSON. The Anthropic API enforces this reliably. With Ollama or any open-weights serving, enforcement is model-dependent and failures are more likely.

Failure modes:

  • Malformed JSON: missing brackets, trailing commas, unquoted keys. The parser tries direct parse, then markdown-fence extraction, then regex scan. Structurally broken JSON falls through all three.
  • Truncated output: models under memory pressure or processing long transcript windows can cut off mid-response, leaving valid-looking but incomplete JSON.
  • Preamble text: some models prefix their JSON with conversational text ("Sure, here are the ads I found:"). The parser usually strips this, but it adds fragility.

When a window fails to parse, those ads are silently missed. No UI error; the episode processes normally with gaps in detection coverage.

Cloud models vary widely on this. Benchmark JSON compliance ranges from 0.07 (openai/o4-mini, which buries JSON in reasoning chains) and 0.10 (qwen/qwen3-8b, which answers in prose) up to 1.00 (the Claude line, the Gemini 3.5 and 3.6 Flash models, Grok, Mistral Medium). See the JSON compliance chart in benchmarks/llm/results/report.md, and parse-and-moderation.md for what the failures actually look like.

Reducing the risk for local runs:

  • Use a model of at least 7B parameters
  • Prefer Qwen3 or Mistral families (consistently high JSON compliance in author testing)
  • Don't run other GPU workloads concurrently; memory pressure increases truncation risk
  • Watch the extraction_method field in processing logs

Healthy run signal: extraction_method reads json_array_direct for most calls. Fallback methods (markdown_code_block, regex_*) mean the model isn't returning clean JSON. Frequent fallback in production means you should upgrade the model.

Using OpenRouter

OpenRouter is a unified API that routes to 200+ models (Claude, GPT, Gemini, open-weights) with one API key. OpenRouter is supported as an LLM provider only: it does not support the /v1/audio/transcriptions endpoint required for Whisper transcription. For transcription without a GPU, use a remote Whisper backend such as Groq.

Setup

  1. Get an API key from openrouter.ai/keys
  2. Add these to your .env, then start with docker-compose.yml (GPU) or docker-compose.cpu.yml (no GPU):
LLM_PROVIDER=openrouter
OPENROUTER_API_KEY=sk-or-v1-your-key-here

Model Selection

No model ships by default: pick one in the Settings UI, or seed it with the OPENAI_MODEL env var; see its row in Environment Variables for how seeding works.

Any OpenRouter model ID works:

  • anthropic/claude-sonnet-4-5: Claude Sonnet via OpenRouter
  • openai/gpt-4o: GPT-4o via OpenRouter
  • google/gemini-2.5-flash-preview: Gemini Flash via OpenRouter
  • openrouter/free: router alias that picks a free model per request
  • openrouter/auto: router alias that picks the best model for the prompt

The openrouter/free and openrouter/auto aliases are not in OpenRouter's /api/v1/models list, so MinusPod adds them to the dropdown for you. Other unlisted model IDs can be typed straight into any model field (the "Type a model ID" link next to each dropdown), or seeded with OPENAI_MODEL.

All of these can be changed at runtime from the Settings UI. No container restart needed.

Per-Stage Providers

MinusPod supports up to two full provider configurations: the primary LLM Provider, and one optional secondary provider with its own type, base URL, and API key. Secondary is off by default (see Secondary provider). Each pipeline stage then picks one of the two, not a provider type directly. Settings > AI & Processing > AI Models shows a slot selector next to each model selector:

  • Ad Detection Provider - Default (Primary) or Secondary. Secondary only appears in the list once the secondary provider is enabled and configured.
  • Verification Provider - Same as detection (default), Default (Primary), or Secondary. Pointing it at the other slot is useful for cost control (a cheap pass-2 sanity check) or comparing two providers' output on the same episode.
  • Chapters Provider - Same as detection (default), Default (Primary), or Secondary, independent of the other two.

The Ad Reviewer section has its own slot selector, described in Ad Reviewer. Same as pass inherits both the provider and model of whichever pass is being reviewed. Primary or Secondary runs the reviewer on that slot instead, and enables the Review Model selector for its catalog.

Switching a stage's slot only changes that stage. Model discovery re-runs for the newly selected slot, so the model dropdown next to it shows that slot's own catalog. Every other stage's slot and model stay exactly as you left them. A stage still accepts a hand-typed model ID (the "Type a model ID" link) for models a slot's catalog does not list, the same as the single-provider case. If a stage is set to Secondary and the secondary provider is later disabled, that stage falls back to Primary until Secondary is turned back on.

Each slot needs its own API key and, where relevant, base URL configured under Settings > LLM Provider before a stage can use it; an unconfigured slot's model list comes back empty until credentials are saved.

The slots are routes, not a failover chain

Primary and Secondary are two independent accounts a stage can be pointed at. They are not a chain, and nothing at runtime moves a call from one to the other. When a call fails, it retries against the same slot and endpoint, then gives up; the episode never spills onto the other provider. Credentials can change if a key is rotated between attempts; see provider key rotation and active runs. A rate-limit pause behaves the same way: the episode waits in the queue for that account's reset instead of rerouting.

The one time a stage changes slot is at configuration time, not on failure. A stage set to Secondary while the secondary provider is disabled or has no type saved resolves to Primary, and logs a warning once. That is a fail-safe for an incomplete configuration, evaluated when the route is resolved, not a response to an error.

A run's routes are frozen when it starts. Each phase's provider, model, and endpoint are snapshotted at run start and reused for the whole run, so editing a provider or a stage's model mid-run does not re-route work already underway. The change applies to the next run. Credentials are the exception: they are read when the client is built, so a rotated key is picked up without a restart. See Rotating a provider key.

Staying under an account's limits

Each slot has its own optional requests-per-minute, requests-per-day, and tokens-per-minute caps, counted per account rather than per provider type, so two accounts on the same provider throttle independently. The per-day counter resets at UTC midnight regardless of where you or the provider are. They are off by default. See Manual request-rate limits.

LLM Pricing

MinusPod tracks token usage and cost for every LLM call. The Settings page and GET /api/v1/system/token-usage show per-model breakdowns.

Custom model pricing

Settings > AI & Processing > AI Models shows pricing fields for each configured model ID, including IDs you type. Enter input and output rates in USD per 1 million tokens. An operator override wins over catalog pricing and appears as pricingSource: "operator". Leave both fields blank to use the catalog. Save both as 0 to mark a model explicitly free; missing pricing is different.

Where pricing data comes from

Pricing is fetched automatically based on your configured provider:

ProviderSourceMethod
Anthropicpricepertoken.comHTML scrape
OpenRouterOpenRouter API (/api/v1/models)JSON API
OpenAI, Groq, Mistral, DeepSeek, xAI, Together, Fireworks, Perplexity, Googlepricepertoken.comHTML scrape
Ollama / localhostN/AAlways $0

Pricing refreshes once every 24 hours in the background. You can also force a refresh from the API:

curl -X POST http://your-server:8000/api/v1/system/model-pricing/refresh

Or view current pricing:

curl http://your-server:8000/api/v1/system/model-pricing

How model matching works

Different sources use different names for the same model. A normalization step strips provider prefixes, date suffixes, and punctuation so that claude-sonnet-4-5-20250929 (Anthropic API), anthropic/claude-sonnet-4-5 (OpenRouter), and Claude Sonnet 4.5 (pricepertoken.com display name) all resolve to the same pricing entry.

Offline / air-gapped installs

If the pricing fetch fails on startup and the database has no pricing data, MinusPod seeds a built-in table of Anthropic prices. A non-Anthropic model without a catalog row has unknown pricing and accrues $0 until a match or operator override exists. Existing cached pricing is kept when a fetch fails.

Pricing accuracy

Pricing data comes from third-party sources and may lag behind provider announcements. Check your provider's billing dashboard for authoritative cost figures. MinusPod's cost tracking is an estimate for convenience, not a billing system.

Reviewer Calibration Self-Test

The reviewer stage (the LLM pass that confirms, adjusts, or rejects candidate ad boundaries) can drift when you switch models or providers. tools.reviewer_calibration runs a small fixed corpus of 8 fictional transcripts (4 ad reads, 4 non-ads) through the same AdReviewer code path production uses, and reports how often the model's verdict agrees with the expected label plus how often its response carries the structured is_ad field the contradiction guard relies on.

Run it manually against your configured provider:

PYTHONPATH=src python -m tools.reviewer_calibration

It prints a pipe-delimited table (case id, expected label, verdict, agreement) followed by a summary line with the model, provider, agreement fraction, and structured fraction. The command exits 1 when agreement falls below 0.75, so it can gate a CI or release check.

Agreement below 0.75 means the reviewer is disagreeing with the expected keep/drop call on 2 or more of the 8 corpus cases. That is a signal to check the model's reasoning quality for this stage, not a hard failure, since the corpus is small and one model's phrasing conventions can differ from another's without being wrong. A low or dropping structured fraction on a model that used to report it consistently means the model has stopped emitting the is_ad field, which weakens the reviewer's contradiction guard.

The self-test also runs automatically in a background thread whenever the reviewer model setting (reviewModel) changes via the Settings API, and stores its result under the reviewer_calibration_last setting. A failed or slow calibration run never blocks the settings save. Set reviewer_calibration_on_change to false (or the REVIEWER_CALIBRATION_ON_CHANGE environment variable) to disable the auto-run. Each auto-run costs 8 LLM calls, one per corpus case. Calibration runs go through the production reviewer, so they emit reviewer telemetry log lines tagged slug=reviewer-calibration; exclude that slug from reviewer metric queries so corpus cases do not skew real verdict counts.

< Docs index | Project README