Amplifier OpenAI Provider Module

May 8, 2026 · View on GitHub

GPT model integration for Amplifier via OpenAI's Responses API.

Prerequisites

Python 3.11+
UV - Fast Python package manager

Installing UV

# macOS/Linux/WSL
curl -LsSf https://astral.sh/uv/install.sh | sh

# Windows
powershell -c "irm https://astral.sh/uv/install.ps1 | iex"

Purpose

Provides access to OpenAI's GPT-5 and GPT-4 models as an LLM provider for Amplifier using the Responses API for enhanced capabilities.

Contract

Module Type: Provider Mount Point: providers Entry Point: amplifier_module_provider_openai:mount

Supported Models

gpt-5.5 - GPT-5 optimized for code (default)
gpt-5.5 - Latest GPT-5 model
gpt-5-mini - Smaller, faster GPT-5
gpt-5-nano - Smallest GPT-5 variant

Configuration

[[providers]]
module = "provider-openai"
name = "openai"
config = {
    base_url = null,                       # Optional custom endpoint (null = OpenAI default)
    default_model = "gpt-5.5",
    max_tokens = 4096,
    temperature = 0.7,
    reasoning = "low",                     # Reasoning effort: minimal|low|medium|high|xhigh
    reasoning_summary = "detailed",        # Reasoning verbosity: auto|concise|detailed
    truncation = null,                     # null omits the field; OpenAI returns an explicit
                                           # error on context overflow. Opt in to legacy
                                           # auto-drop with truncation = "auto" (busts cache).
    prompt_cache_key = "",                 # Stable cache-routing identifier; empty = unset
    prompt_cache_retention = "24h",        # "24h" | "in_memory" | null (use model default)
    enable_response_chaining = "auto",     # "auto" | true | false  (reasoning-model chaining)
    enable_state = false,
    debug = false,                         # Enable standard debug events
    raw_debug = false                      # Enable ultra-verbose raw API I/O logging
}

Note: safety_identifier is intentionally NOT a deployment config field. It is a per-end-user signal (abuse tracking) and must be set per-call via kwargs. See Prompt Caching below.

The provider exposes OpenAI's prompt-caching hint parameters (prompt_cache_key, prompt_cache_retention, safety_identifier) plus an enable_response_chaining toggle that activates the Responses API's previous_response_id mechanism for reasoning models.

TL;DR — what the defaults give you

prompt_cache_retention = "24h" — extended GPU-local KV storage on every supported model, instead of OpenAI's per-model "in_memory" default (5–10 min) for gpt-5.4 and below.
enable_response_chaining = "auto" — for reasoning-capable models, the provider sends store = true and previous_response_id on subsequent turns, and stops re-inserting encrypted reasoning blocks inline. Empirical smoke against gpt-5.5 measured 85% prefix cache hit on turn 2 with chaining on, vs 0% off.
truncation = null — the field is omitted from requests so the cached prefix is never silently rewritten on context overflow. Opt back into the legacy auto-drop behavior with truncation = "auto".

prompt_cache_key is empty by default; setting it is opt-in. safety_identifier is kwargs-only.

`prompt_cache_key` — cache-routing identifier

OpenAI shards Responses API traffic across machines by hashing the first ~256 input tokens. Without a stable key, requests with identical prefixes still hit the same shard most of the time, but as soon as anything in the prefix shifts (time-of-day stamp, shuffled tools, rewritten system prompt), routing diverges. A stable prompt_cache_key keeps a logical conversation pinned to one machine regardless of small prefix drift, and is the recommended cache signal as of OpenAI's July 2025 guidance. (The legacy user field still works on the API but is no longer the recommended cache signal.)

Granularity guidance:

Deployment shape	Recommended key
Single-user agent loop (typical Amplifier)	conversation/session ID, e.g. `"conv_abc123"`
Multi-tenant with shared system prompt	`f"{tenant_id}:{system_prompt_version}"`
Low-volume single-session	leave unset; prefix-hash routing is sufficient

Watch out for the ~15 RPM threshold per (prefix, key) pair. Past that, OpenAI spills overflow requests to fresh machines; high-volume conversations should prefer a key that distributes across tenants/sessions rather than one global constant.

`prompt_cache_retention` — TTL hint

Value	Meaning
`"24h"`	Extended GPU-local KV storage. Provider default for all supported models.
`"in_memory"`	5–10 min in-process cache. OpenAI's per-model default for gpt-5.4 and below.
`null`	Field omitted; OpenAI picks the per-model default.

The capability layer auto-drops values a model would reject:

gpt-5.5+ rejects "in_memory" — provider drops it with a [PROVIDER] Dropping prompt_cache_retention='in_memory' warning.
Any future model that rejects "24h" (capability flag supports_24h_retention = False) gets the field dropped the same way.

You do not need to special-case retention per model; the default "24h" is safe everywhere.

`enable_response_chaining` — reasoning-model chaining

Tri-state config (and per-call kwarg). Controls whether the provider uses the Responses API's previous_response_id mechanism, which is the high-leverage caching path for reasoning models:

Value	Behavior
`"auto"`	On iff `get_capabilities(model).supports_reasoning` is True. Default. Right answer for most.
`true`	Force on regardless of model. Useful for testing or non-reasoning models that still benefit.
`false`	Force off. Use for ZDR / regulated-industry deployments that cannot retain server-side state.

When chaining is active for a reasoning model, three things happen on each call:

store = true is set automatically (chaining requires it; this overrides enable_state for reasoning models).
previous_response_id = <id from last assistant.metadata> is sent on subsequent turns.
Encrypted reasoning items (include=["reasoning.encrypted_content"] and re-insertion in _convert_messages) are NOT sent — the server holds reasoning state under previous_response_id, and inlining encrypted blobs would bust the cache prefix.

For non-reasoning models, or enable_response_chaining = false, behavior is unchanged: stateless mode with explicit reasoning re-insertion (see Reasoning State Preservation).

On previous_response_id invalidation (HTTP 404 + response_not_found), the provider retries once without the field and emits a provider:response_chain_invalidated event.

`safety_identifier` — kwargs-only

The request-side counterpart to prompt_cache_key: an abuse-tracking signal that should carry a per-end-user value. Intentionally NOT exposed as a deployment ConfigField — surfacing it in deployment config invites operators to set one global value, which defeats its purpose. Set it via per-call kwargs only.

Behavioral change: `truncation` default

The truncation default flipped from "auto" to null (omit the field).

Before: silently dropped oldest messages on context overflow.
After: OpenAI returns an explicit context_length_exceeded error.

Reason: truncation = "auto" rewrites the cached prefix and is on OpenAI's own troubleshooting checklist as a top cause of low cache hit rates. To opt back into the legacy auto-drop:

config = { truncation = "auto" }

Recommended configurations

Single-user agent loop (typical Amplifier session): defaults are correct. For high-volume sessions, optionally pin to a session ID:

config = { prompt_cache_key = "session_${SESSION_ID}" }

Multi-tenant deployment: key per tenant + system-prompt version to shard load while preserving cache stickiness. Use safety_identifier per-call for abuse tracking:

config = { prompt_cache_key = "tenant_42:sysprompt_v7" }
# In application code, per request:
# await provider.complete(request, safety_identifier="end_user_abc")

ZDR / regulated industries: disable chaining so no server-side state is retained for reasoning models:

config = { enable_response_chaining = false, enable_state = false }

Observability

Cache hit rate surfaces as usage.cache_read_tokens on responses, and is emitted in llm:response events as cache_read_tokens. Note: OpenAI does NOT report a cache_creation_tokens metric (unlike Anthropic) — cache writes are implicit and not counted separately.

Chain invalidation is observable via the provider:response_chain_invalidated event.

Debug Configuration

Standard Debug (debug: true):

Emits llm:request:debug and llm:response:debug events
Contains request/response summaries with message counts, model info, usage stats
Moderate log volume, suitable for development

Raw Debug (debug: true, raw_debug: true):

Emits llm:request:raw and llm:response:raw events
Contains complete, unmodified request params and response objects
Extreme log volume, use only for deep provider integration debugging
Captures the exact data sent to/from OpenAI API before any processing

Example:

providers:
  - module: provider-openai
    config:
      debug: true # Enable debug events
      raw_debug: true # Enable raw API I/O capture
      default_model: gpt-5.5

Environment Variables

export OPENAI_API_KEY="your-api-key-here"

Usage

# In amplifier configuration
[provider]
name = "openai"
model = "gpt-5.5"

Features

Responses API Capabilities

Reasoning Control - Adjust reasoning effort (minimal, low, medium, high, xhigh)
Reasoning Summary Verbosity - Control detail level of reasoning output (auto, concise, detailed)
Extended Thinking Toggle - Enables high-effort reasoning with automatic token budgeting
Explicit Reasoning Preservation - Re-inserts reasoning items (with encrypted content) into conversation for robust multi-turn reasoning
Prompt Caching Hints - prompt_cache_key, prompt_cache_retention (default "24h"), and per-call safety_identifier wired into the request builder. See Prompt Caching.
Response Chaining for Reasoning Models - enable_response_chaining activates previous_response_id for reasoning models, materially improving prefix cache hit rate (default "auto").
Cache-Stable Truncation - truncation defaults to null (omitted) so the cached prefix is never silently rewritten on context overflow.
Stateful Conversations - Optional conversation persistence
Native Tools - Built-in web search, image generation, code interpreter
Structured Output - JSON schema-based output formatting
Function Calling - Custom tool use support
Token Counting - Usage tracking and management (including cache_read_tokens)

Reasoning Summary Levels

The reasoning_summary config controls the verbosity of reasoning blocks in the model's response:

auto (default if not specified) - Model decides appropriate detail level
concise - Brief reasoning summaries (faster, fewer tokens)
detailed - Verbose reasoning output similar to Anthropic's extended thinking blocks

Example comparison:

# Concise reasoning (brief summaries)
providers:
  - module: provider-openai
    config:
      reasoning: "medium"
      reasoning_summary: "concise"

# Detailed reasoning (verbose like Anthropic's thinking blocks)
providers:
  - module: provider-openai
    config:
      reasoning: "high"
      reasoning_summary: "detailed"

Note: Detailed reasoning consumes more output tokens but provides deeper insight into the model's thought process, useful for complex problem-solving and debugging.

Tool Calling

The provider detects OpenAI Responses API function_call / tool_call blocks automatically, decodes JSON arguments, and returns standard ToolCall objects to Amplifier. No extra configuration is required—tools declared in your config or profiles execute as soon as the model requests them.

Incomplete Response Auto-Continuation

The provider automatically handles incomplete responses from the OpenAI Responses API:

The Problem: OpenAI may return status: "incomplete" when generation is cut off due to:

max_output_tokens limit reached
Content filter triggered
Other API constraints

The Solution: The provider automatically continues generation using previous_response_id until the response is complete:

Transparent continuation - Makes follow-up calls automatically (up to 5 attempts)
Output accumulation - Merges reasoning items and messages from all continuations
Single response - Returns complete ChatResponse to orchestrator
Full observability - Emits provider:incomplete_continuation events for each continuation

Example flow:

# User request triggers large response
response = await provider.complete(request)

# Provider internally (if incomplete):
# 1. Initial call returns status="incomplete", reason="max_output_tokens"
# 2. Continuation 1: Uses previous_response_id, gets more output
# 3. Continuation 2: Uses previous_response_id, gets final output
# 4. Returns merged response with all content

# Orchestrator receives complete response, unaware of continuations

Configuration: Set maximum continuation attempts (default: 5):

# In _constants.py
MAX_CONTINUATION_ATTEMPTS = 5  # Prevents infinite loops

Observability: Monitor via events in session logs:

{
  "event": "provider:incomplete_continuation",
  "provider": "openai",
  "response_id": "resp_abc123",
  "reason": "max_output_tokens",
  "continuation_number": 1,
  "max_attempts": 5
}

Reasoning State Preservation

The provider preserves reasoning state across conversation steps for improved multi-turn performance:

The Problem: Reasoning models (o3, o4, gpt-5.5) produce internal reasoning traces (rs_* IDs) that improve subsequent responses by ~3-5% when preserved. This is especially critical when tool calls are involved.

Important Distinction:

Turn: A user prompt → (possibly multiple API calls) → final assistant response
Step: Each individual API call within a turn (tool call loops = multiple steps per turn)
Reasoning items must be preserved across STEPS, not just TURNS

The Solution: The provider uses explicit reasoning re-insertion for robust step-by-step reasoning:

Requests encrypted content - API call includes include=["reasoning.encrypted_content"]
Stores complete reasoning state - Both encrypted content and reasoning ID stored in ThinkingBlock.content field
Re-inserts reasoning items - Explicitly converts reasoning blocks back to OpenAI format in subsequent turns
Maintains metadata - Also tracks reasoning IDs in metadata for backward compatibility

How it works (tool call example showing step-by-step preservation):

# Step 1: User asks question requiring tool
response_1 = await provider.complete(request)
# response_1.output contains:
#   - reasoning item: rs_abc123 (with encrypted_content)
#   - tool_call: get_weather(latitude=48.8566, longitude=2.3522)
#
# Provider stores ThinkingBlock with:
#   - thinking: "reasoning summary text"
#   - content: [encrypted_content, "rs_abc123"]  # Full reasoning state
#   - metadata: {"openai:reasoning_items": ["rs_abc123"], ...}

# Orchestrator executes tool, adds result to context
# (Note: This is still within the SAME TURN, just a different STEP)

# Step 2: Provider called again with tool result (SAME TURN!)
response_2 = await provider.complete(request_with_tool_result)
# Provider reconstructs reasoning item from previous step:
# {
#   "type": "reasoning",
#   "id": "rs_abc123",
#   "encrypted_content": "...",  # From ThinkingBlock.content[0]
#   "summary": [{"type": "summary_text", "text": "..."}]
# }
# OpenAI receives: [user_msg, reasoning_item, tool_call, tool_result]
# Model uses preserved reasoning from step 1 to generate final answer

Key insight from OpenAI docs: "While this is another API call, we consider this as a single turn in the conversation." Reasoning must be preserved across steps (API calls) within the same turn, especially when tools are involved.

Benefits:

More robust - Explicit re-insertion doesn't rely on server-side state
Stateless compatible - Works with store: false configuration
Better multi-turn performance - ~5% improvement per OpenAI benchmarks
Critical for tool calling - Recommended by OpenAI for reasoning models with tools
Follows OpenAI docs - Implements "context += response.output" pattern

Automatic Context Management (Truncation)

The provider supports OpenAI's optional truncation parameter for automatic conversation history management. The default flipped from "auto" to null (omitted) as of PR #34, because truncation = "auto" rewrites the cached prefix on overflow and busts prompt caching.

Configuration:

providers:
  - module: provider-openai
    config:
      truncation: null    # Default. Field omitted; OpenAI errors on overflow.
      # OR
      truncation: "auto"  # Opt in: drop oldest messages on overflow (busts cache).

How "auto" works (when explicitly opted in):

OpenAI automatically removes oldest messages when context limit approached
FIFO (first-in, first-out) - most recent messages preserved
Transparent to application - no errors or warnings
Works with all conversation types (reasoning, tools, multi-turn)

Trade-offs:

null (default): explicit context_length_exceeded error on overflow, but the cached prefix is preserved across turns.
"auto": simplicity and never-hit-the-limit, at the cost of cache hit rate. Listed on OpenAI's troubleshooting checklist as a top cause of low cache utilization.

When to use "auto":

Legacy behavior compatibility, or workloads where simplicity outweighs cache efficiency. For most users, the new default (null) is correct; pair it with explicit context management upstream.

See also: Prompt Caching for the broader cache-stability rationale.

Metadata Keys

The provider populates ChatResponse.metadata with OpenAI-specific state:

Key	Type	Description
`openai:response_id`	`str`	Response ID for continuation and reasoning preservation
`openai:status`	`str`	Response status: `"completed"` or `"incomplete"`
`openai:incomplete_reason`	`str`	Reason if incomplete: `"max_output_tokens"` or `"content_filter"`
`openai:reasoning_items`	`list[str]`	Reasoning item IDs (rs_*) for state preservation
`openai:continuation_count`	`int`	Number of auto-continuations performed (if > 0)

Example metadata:

{
    "openai:response_id": "resp_05fb664e4d9dca6a016920b9b1153c819487f88da867114925",
    "openai:status": "completed",
    "openai:reasoning_items": ["rs_05fb664e4d9dca6a016920b9b1daac81949b7ea950bddef95a"],
    "openai:continuation_count": 2
}

Namespacing: All keys use openai: prefix to prevent collisions with other providers (per kernel philosophy).

Graceful Error Recovery

The provider implements graceful degradation for incomplete tool call sequences:

The Problem: If tool results are missing from conversation history (due to context compaction bugs, parsing errors, or state corruption), the OpenAI API rejects the entire request, breaking the user's session.

The Solution: The provider automatically detects missing tool results and injects synthetic results that:

Make the failure visible - LLM sees [SYSTEM ERROR: Tool result missing] message
Maintain conversation validity - API accepts the request, session continues
Enable recovery - LLM can acknowledge the error and ask user to retry
Provide observability - Emits provider:tool_sequence_repaired event with details

Example:

# Broken conversation history (missing tool result)
messages = [
    {"role": "assistant", "tool_calls": [{"id": "call_123", "function": {"name": "get_weather", ...}}]},
    # MISSING: {"role": "tool", "tool_call_id": "call_123", "content": "..."}
    {"role": "user", "content": "Thanks"}
]

# Provider injects synthetic result:
{
    "role": "tool",
    "tool_call_id": "call_123",
    "content": "[SYSTEM ERROR: Tool result missing from conversation history]\n\nTool: get_weather\n..."
}

# LLM responds: "I notice the weather tool failed. Let me try again..."
# Session continues instead of crashing

Observability: Repairs are logged as warnings and emit provider:tool_sequence_repaired events for monitoring.

Philosophy: This is graceful degradation following kernel philosophy - errors in other modules (context management) don't crash the provider or kill the user's session.

Dependencies

amplifier-core>=1.0.0
openai>=1.0.0

Contributing

Note

This project is not currently accepting external contributions, but we're actively working toward opening this up. We value community input and look forward to collaborating in the future. For now, feel free to fork and experiment!

Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit Contributor License Agreements.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.

Trademarks

This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft's Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party's policies.