Test Suites

Rig's root crate uses integration test targets under tests/.

• tests/<provider>.rs are provider-specific test targets.
• tests/providers/<provider>/cassette/ contains provider tests backed by committed HTTP cassettes.
• tests/providers/<provider>/live/ contains provider tests that still require a real service.
• tests/integrations.rs is the vector-store and external-service integration target.
• tests/core.rs contains provider-agnostic core behavior tests.

Most provider tests are ignored live tests unless they have been migrated to cassettes.

Core Tests

Run provider-agnostic core tests with:

cargo test -p rig --test core

Run all default non-ignored tests for the root crate with:

cargo test -p rig

Run the same checks with all root crate features enabled:

cargo test -p rig --all-features

Cassette Provider Tests

Cassette tests replay committed HTTP interactions by default and do not require provider API keys. Cassette files live under tests/cassettes/<provider>/....

Replay the migrated provider suites with:

cargo test -p rig --all-features --test openai openai::cassette -- --nocapture --test-threads=1
cargo test -p rig --all-features --test anthropic anthropic::cassette -- --nocapture --test-threads=1
cargo test -p rig --all-features --test gemini gemini::cassette -- --nocapture --test-threads=1
cargo test -p rig --all-features --test chatgpt chatgpt::cassette -- --nocapture --test-threads=1
cargo test -p rig --all-features --test bedrock bedrock::cassette -- --nocapture --test-threads=1
cargo test -p rig --all-features --test doubleword doubleword::cassette -- --nocapture --test-threads=1

Bedrock cassette replay does not require AWS credentials. Bedrock record mode uses the AWS SDK credential provider chain and a direct SigV4-aware recorder, so it requires AWS credentials with Bedrock model access in us-east-1 and overwrites existing cassette files. The Bedrock recorder buffers streaming/event-stream responses and stores non-UTF-8 cassette bodies as base64; those opaque bodies are intended for replay fidelity, and safety checks also scan their decoded bytes for credential-shaped material.

Record mode requires the relevant provider credentials in the environment and overwrites existing cassette files:

RIG_PROVIDER_TEST_MODE=record \
cargo test -p rig --all-features --test openai openai::cassette -- --nocapture --test-threads=1

RIG_PROVIDER_TEST_MODE=record \
cargo test -p rig --all-features --test anthropic anthropic::cassette -- --nocapture --test-threads=1

RIG_PROVIDER_TEST_MODE=record \
cargo test -p rig --all-features --test gemini gemini::cassette -- --nocapture --test-threads=1

RIG_PROVIDER_TEST_MODE=record \
cargo test -p rig --all-features --test bedrock bedrock::cassette -- --nocapture --test-threads=1

RIG_PROVIDER_TEST_MODE=record \
cargo test -p rig --all-features --test doubleword doubleword::cassette -- --nocapture --test-threads=1

CHATGPT_ACCESS_TOKEN=... CHATGPT_ACCOUNT_ID=... RIG_PROVIDER_TEST_MODE=record \
cargo test -p rig --all-features --test chatgpt chatgpt::cassette -- --nocapture --test-threads=1

Run one cassette test by passing a test-name substring:

cargo test -p rig --all-features --test gemini \
  streaming_tools_smoke \
  -- --nocapture --test-threads=1

The test filter after --test <target> is a substring match. Use the full module path only when the shorter test name is ambiguous.

Cassette Safety

Record mode scrubs and safety-checks cassette contents before writing fixtures. The committed cassette safety tests enforce the same scrubbed form during normal test runs.

Review cassette diffs for:

• no API keys, bearer tokens, cookies, or provider account identifiers;
• expected request paths, methods, and bodies;
• expected provider responses for the scenario;
• no unrelated cassette churn.

Live Provider Tests

Live provider tests use real provider APIs, local model servers, or account credentials. They are ignored by default unless a test file says otherwise.

Run ignored tests for one provider target with:

cargo test -p rig --all-features --test openrouter -- --ignored --nocapture --test-threads=1

Run one ignored provider test with:

cargo test -p rig --all-features --test openai \
  responses_document_file_id_roundtrip_live \
  -- --ignored --nocapture --test-threads=1

Use the provider-specific environment variables named in the ignored test reason or provider module, such as OPENROUTER_API_KEY, MISTRAL_API_KEY, GROQ_API_KEY, XAI_API_KEY, HUGGINGFACE_API_KEY, or local services such as Ollama, llamafile, and llama.cpp.

Local Artifact Model Tests

rig-candle has an ignored native model-contract suite. It is not an HTTP cassette: it loads one pinned Qwen3 GGUF artifact and runs provider-neutral completion, buffered/raw-streaming parity, parallel and sequential tools, zero-argument and complex typed arguments, call/result history correlation, result serialization, invalid-call recovery, hook rewrite chaining, turn-local request patches, cancellation/max-turn diagnostics, extraction with usage, tool-choice, protocol hygiene, and synthetic structured-output scenarios through Rig's agent driver.

export RIG_CANDLE_TEST_MODEL_DIR="$PWD/crates/rig-candle/test-models/qwen3-4b-q4-k-m"
./crates/rig-candle/tests/download_qwen3.sh
cargo test --release -p rig-candle --test live_conformance \
  -- --ignored --nocapture --test-threads=1

The 2.33-GiB model is checksum-verified, cached in an ignored directory, and loaded once per test binary. The measured ARM64 release run completed in 164.41 seconds; allow at least fifteen minutes for slower CPU hosts and more than twice the checkpoint size during loading. Use serial execution to bound CPU and memory use. See crates/rig-candle/README.md for revisions, hashes, measured performance, and the boundary between model-contract and provider-transport tests.

Reusable scenarios and typed validators are exported from rig_core::test_utils. A provider suite should call a complete model-driving scenario when its cassette records the same prompt and tool definitions. When wire-specific prompts, schemas, request parameters, or metadata must remain local, the provider test should retain that transport setup and call the shared validator on its public Rig result. Do not move authentication, HTTP body, SSE, hosted-tool, remote-file, or provider-session assertions into the portable module.

Universal scenarios require only the public completion/agent contract. Optional capability scenarios—parallel model emission, structured reasoning, provider-assigned IDs, native constrained decoding, and hosted tools—must be selected explicitly. A provider that does not expose one optional capability must not weaken the universal assertions or silently mark the scenario passed.

Integration Tests

External-service integration tests are collected under the integrations target and are gated by feature flags.

Run all enabled non-ignored integration tests with:

cargo test -p rig --all-features --test integrations

Run one feature-gated integration group with:

cargo test -p rig --features qdrant --test integrations qdrant -- --nocapture
cargo test -p rig --features mongodb --test integrations mongodb -- --nocapture
cargo test -p rig --features sqlite --test integrations sqlite -- --nocapture

Some integration tests start Docker containers through testcontainers; Docker must be running. Other integrations are ignored because they need external credentials or pre-provisioned services. Run ignored integration tests explicitly:

cargo test -p rig --features bedrock --test integrations bedrock -- --ignored --nocapture --test-threads=1
cargo test -p rig --features vectorize --test integrations vectorize -- --ignored --nocapture --test-threads=1

Check each integration module for required environment variables. For example, Vectorize requires VECTORIZE_INDEX_NAME, and Bedrock tests require AWS credentials plus access to the configured Bedrock models.