Mastra with sbproxy

Last modified: 2026-08-19

A Mastra agent normally reaches providers directly: the model comes from the AI SDK provider layer and calls api.openai.com, and each MCP tool server is a separate connection with its own credentials. Point both sides at an sbproxy you run and every model call and every tool call crosses one gateway you control. That is where virtual keys scope models and attribute spend, budgets meter tokens and dollars, guardrails screen traffic, the usage ledger records what happened, and repeated completions can come back from cache. On the Mastra side the change is a base URL on the model and one server entry for tools.

Chat completions through the gateway

sbproxy serves an OpenAI-compatible endpoint at /v1/chat/completions, and a Mastra Agent takes its model straight from the AI SDK provider layer, so createOpenAI from @ai-sdk/openai works unchanged. Set baseURL to the gateway, pass your virtual key as the apiKey, and take the model from .chat():

// Validated with @mastra/core@1.50.1, @ai-sdk/openai@4.0.10, zod@3.25.76.
// Install: npm install @mastra/core @ai-sdk/openai zod
import { Agent } from "@mastra/core/agent";
import { createOpenAI } from "@ai-sdk/openai";

const openai = createOpenAI({
  baseURL: "http://127.0.0.1:8080/v1",
  apiKey: "sk-your-virtual-key",
});

const agent = new Agent({
  name: "gateway-agent",
  instructions: "You are a concise assistant.",
  model: openai.chat("gpt-4o-mini"),
});

const result = await agent.generate("In one sentence, what does an AI gateway do?");
console.log(result.text);

Save it as agent.mjs and run node agent.mjs. The .chat() call is deliberate: the bare form openai("gpt-4o-mini") builds a model for OpenAI's Responses API. The gateway serves /v1/responses for stateless requests, streaming included, but it refuses anything that leans on OpenAI-side state: previous_response_id, conversation, and store: true each return a 400 rather than silently running without the state they reference, and only function tools are forwarded (see the Responses API boundaries in the AI gateway guide). openai.chat("gpt-4o-mini") speaks /v1/chat/completions, the wire format sbproxy translates for every provider it fronts, and avoids those boundaries entirely. If you prefer a provider with no OpenAI-specific behavior, @ai-sdk/openai-compatible (validated at 3.0.7) works the same way: createOpenAICompatible({ name: "sbproxy", baseURL, apiKey }).chatModel("gpt-4o-mini").

The gateway needs an origin with an ai_proxy action and a credential for the virtual key. Save this as sb.yml and start the gateway with sbproxy sb.yml:

proxy:
  http_bind_port: 8080

origins:
  "127.0.0.1":
    action:
      type: ai_proxy
      require_governed_key: true
      providers:
        - name: openai
          api_key: ${OPENAI_API_KEY}
          default_model: gpt-4o-mini
          models:
            - gpt-4o-mini
    credentials:
      - name: mastra-app
        type: ai_provider
        provider: openai
        key: sk-your-virtual-key
        attrs:
          project: mastra
          tags: [mastra-app]
          budget:
            max_tokens: 1000000
            max_cost_usd: 25
        models:
          allow: [gpt-4o-mini]

Origin keys match the Host header and hostname matching strips the port, so "127.0.0.1" matches a client whose base URL is http://127.0.0.1:8080. When the gateway runs elsewhere, key the origin with the hostname your application uses. The real provider key comes from the environment through ${OPENAI_API_KEY} interpolation; never put a raw provider key in the file.

Be precise about what the virtual key does here. When a request arrives with Authorization: Bearer sk-your-virtual-key, the gateway matches it to the mastra-app credential, enforces the models.allow list (a request for a model outside the list is rejected with 403 before any upstream call), stamps the request with the credential's project and tags for attribution in metrics and the ledger, and swaps in the real ${OPENAI_API_KEY} before calling the provider. Your application never holds the provider key. The attrs.budget block is attribution metadata that surfaces as attribution labels on the sbproxy_ai_*_attributed_total metrics; enforced spend ceilings live in an action-level budget: block. action.require_governed_key: true is what makes any of this enforced rather than declarative: config compile now refuses an origin that declares credentials: without it, and with it set, a request presenting an unknown key or no key gets a 401 before any upstream call. The virtual key is still a static bearer secret with no rate limiting on guessing it, so add an authentication block to the origin when the gateway is reachable beyond localhost. ai-gateway.md covers all of this in depth.

Run it without a provider account

The repository ships this page's gateway config as a runnable example in examples/mastra/, with the openai provider pointed at a local OpenAI-shaped fixture instead of api.openai.com. The virtual key match, the models.allow gate, and the provider dispatch all run for real; only the model is canned (it answers fixture response and echoes the model name back). Boot it and send the curl equivalent of what agent.generate puts on the wire:

cd examples/mastra
docker compose up -d --wait
curl -sS http://127.0.0.1:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -H 'Authorization: Bearer sk-your-virtual-key' \
  -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"In one sentence, what does an AI gateway do?"}]}'

{
  "id": "chatcmpl-fixture",
  "object": "chat.completion",
  "created": 0,
  "model": "gpt-4o-mini",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "fixture response"
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 1,
    "completion_tokens": 1,
    "total_tokens": 2
  }
}

Ask for a model outside the credential's allow list and the gateway refuses before any upstream call:

curl -sS -i http://127.0.0.1:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -H 'Authorization: Bearer sk-your-virtual-key' \
  -d '{"model":"gpt-4o","messages":[{"role":"user","content":"Try the expensive model."}]}'

HTTP/1.1 403 Forbidden
content-type: application/json
content-length: 54
Date: Sun, 02 Aug 2026 03:43:54 GMT
Connection: keep-alive

{"error":"model 'gpt-4o' is not allowed for this key"}

The agent.mjs snippet above works against the same stack unchanged. docker compose down -v tears it down.

MCP tools through the gateway

sbproxy is also a gateway for the Model Context Protocol (MCP), the JSON-RPC protocol agents use to discover and call tools. The gateway aggregates any number of upstream MCP servers behind one endpoint: clients POST JSON-RPC requests such as tools/list and tools/call to the origin root, and the gateway federates the catalog, applies guardrails, and routes each call to the upstream that owns the tool.

A minimal mcp origin federating two upstream tool servers:

proxy:
  http_bind_port: 8080

origins:
  "127.0.0.1":
    action:
      type: mcp
      mode: gateway
      server_info:
        name: gateway-tools
        version: "1.0.0"
      federated_servers:
        - origin: orders.internal
          prefix: orders
        - origin: weather.internal
          prefix: weather

Bare hostnames under federated_servers are normalized to https://<host>/mcp; use a full URL for any other path. The prefix is the server label: by default tool names stay bare in the federated catalog and the label steps in only when two upstreams advertise the same name (the first-listed server keeps the bare name and the later server's tool becomes orders.get_order_status). Set namespace: always on an entry to prefix everything it exposes. An origin key carries one action, so when you want chat completions and MCP behind the same gateway process, give each its own origin keyed by hostname.

The two remaining fields are the gateway's own identity. mode: gateway is what puts the origin in federating mode, where the gateway owns the catalog and fans each tools/call out to the upstream that holds the tool; server_info is the name and version the gateway reports for itself during the initialize handshake, which is what MCP clients display when they list their connected servers.

Both federated_servers entries above are placeholders. orders.internal and weather.internal do not resolve, so booting that config as written gives you an empty tools/list: the gateway degrades per server, dropping an unreachable upstream with a log line instead of failing the whole catalog. Substitute your own tool servers, or run the federation example below against an upstream that answers.

Run the MCP half without writing a tool server

examples/mcp-federation/ is the runnable version of the config above, and it ships its own upstream so nothing external has to resolve. Its gh entry is a type: openapi federated server: the gateway derives MCP tools from an inline OpenAPI spec and dispatches each tools/call as an ordinary REST request, so there is no MCP server code anywhere in the example. It also sets namespace: always, which is why its tools arrive prefixed as gh.* rather than bare. Its db entry is left unresolvable on purpose, so you can watch the per-server degradation described above happen right next to an upstream that works.

It runs as two processes, the mock REST API and the gateway that federates it:

sbproxy serve -f examples/mcp-federation/upstream.yml &
sbproxy serve -f examples/mcp-federation/sb.yml

Every call below is an HTTP POST of a JSON-RPC envelope. The Accept header has to offer both application/json and text/event-stream, because the streamable HTTP transport picks between them per response, and MCPClient sends both for you.

This is the federated catalog, the same list mcp.listTools() returns:

{"description":"Search repositories by query.","inputSchema":{"properties":{"q":{"type":"string"}},"required":["q"],"type":"object"},"name":"gh.search_repos"}

One tool, not two, because the db upstream never answered. Calling the tool that did register dispatches a real HTTP request to the mock upstream and returns its response as MCP tool-result content:

[{"full_name":"soapbucket/sbproxy","name":"sbproxy","stars":4200},{"full_name":"soapbucket/docs","name":"docs","stars":12}]

To point the script below at this stack, set the server entry's url to http://127.0.0.1:8080/ with a Host: mcp.example.com header. Mastra namespaces by your server entry name on top of the gateway's own prefix, so the tool arrives as gateway_gh.search_repos; the suffix match in the snippet still finds it if you search for search_repos. The example's origin is keyed by hostname, so the Host header is what selects the MCP origin.

Mastra consumes the gateway through the MCPClient in @mastra/mcp (validated at 1.13.1), which speaks the streamable HTTP transport the gateway serves. One server entry pointed at the gateway is enough, because the gateway is already the aggregation point:

// Install: npm install @mastra/mcp (plus the chat packages above)
import { Agent } from "@mastra/core/agent";
import { createOpenAI } from "@ai-sdk/openai";
import { MCPClient } from "@mastra/mcp";

const mcp = new MCPClient({
  servers: {
    gateway: {
      url: new URL("http://127.0.0.1:8080/"),
    },
  },
});

const tools = await mcp.listTools();
console.log("tools:", Object.keys(tools));

const openai = createOpenAI({
  baseURL: "http://127.0.0.1:8080/v1",
  apiKey: "sk-your-virtual-key",
});

const agent = new Agent({
  name: "gateway-agent",
  instructions: "Use the available tools to answer questions.",
  model: openai.chat("gpt-4o-mini"),
  tools,
});

const reply = await agent.generate("What is the weather in Lisbon?");
console.log(reply.text);

// Tools are ordinary Mastra tools, so you can also call one directly.
// The gateway routes the tools/call to the upstream that owns it.
const weatherKey = Object.keys(tools).find((n) => n.endsWith("get_weather"));
console.log(await tools[weatherKey].execute({ city: "Lisbon" }));

await mcp.disconnect();

listTools() returns the federated catalog with each tool namespaced by your server entry name, so get_weather arrives as gateway_get_weather; that is why the lookup above matches on the suffix. Passing the result as tools: fixes the catalog when the agent is constructed. To inject tools per request instead, listToolsets() groups them by server and generate() accepts them per call:

const reply = await agent.generate("What is the weather in Lisbon?", {
  toolsets: await mcp.listToolsets(),
});

Older @mastra/mcp releases named these methods getTools() and getToolsets(); 1.13 renamed them to listTools() and listToolsets(). Guardrails such as tool_allowlist, per-upstream RBAC, and per-server timeouts from mcp.md apply to every call the client makes.

What the gateway gives you

With both flows on the gateway, you set token and dollar budgets in one place instead of per application, and a runaway agent hits a 403 instead of a surprise invoice. Guardrails screen prompts, completions, and tool calls at the choke point, so a policy change is a config edit rather than a code deploy. Every model call and tool call lands in the hash-chained usage ledger, a tamper-evident record of what each key spent and which tools each agent touched. Response caching serves repeated completions without an upstream call, which is free latency and free money on eval loops and retries. Details live in ai-gateway.md and mcp.md.