TrustGate

TrustGate is an open-source AI gateway that routes, secures, and observes traffic between applications, AI agents, and LLM providers.

A security-first LLM and AI Agent gateway built in Go.

Route, govern, and observe all LLM and MCP traffic through a single control point.

Documentation  |  Quick Start  |  Examples  |  Architecture  |  Community

Why TrustGate?

TrustGate is purpose-built for teams that need enterprise-grade governance over their LLM and agent traffic — not just routing and observability.

TrustGate differentiators:

1. Security-first architecture — API-key auth, per-consumer policies, plugin stages (rate limit, token rate limit, request size, semantic cache) that run before traffic hits providers.
2. MCP aggregation plane — A dedicated :8082 plane that aggregates upstream MCP servers, so AI agents connect to one gateway instead of many tools. See the MCP testing guide and OpenAPI → MCP limits.
3. Single static binary — No Python, no Node, no runtime dependencies. Deploy anywhere: Docker, Kubernetes, bare metal.

60-Second Quick Start

Option A: One-line install (recommended)

curl -fsSL https://raw.githubusercontent.com/NeuralTrust/TrustGate/main/scripts/install.sh | bash

This clones the repo, seeds .env, and starts the full stack. When Go is installed, it also builds the trustgate CLI.

Option B: Docker Compose

git clone https://github.com/NeuralTrust/TrustGate.git && cd TrustGate
cp .env.example .env
make up

Verify it's running

curl localhost:8080/healthz   # Admin plane
curl localhost:8081/healthz   # Proxy plane
curl localhost:8082/healthz   # MCP plane

Your first chat completion

Once running, make a request through the proxy (full setup in examples/curl-first-request/):

# Assumes you've created a gateway, registry, and consumer (see examples/)
curl -X POST "http://localhost:8081/my-app/v1/chat/completions" \
  -H "X-AG-Gateway-Slug: demo" \
  -H "X-AG-API-Key: $CONSUMER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"Hello!"}]}'

# 7. Embeddings (OpenAI, Azure OpenAI, Mistral, Vertex, Bedrock Titan, openai_compatible, Cohere)
curl -s -X POST "$PROXY/$CON_SLUG/v1/embeddings" \
  -H "X-AG-Gateway-Slug: $GW_SLUG" -H "X-AG-API-Key: $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"text-embedding-3-small","input":["Hello from TrustGate"]}'

OpenAI-shaped clients always call POST /{consumer}/v1/embeddings. OpenAI, Azure, Mistral, and custom openai_compatible registries forward that payload to the upstream embeddings URL. Vertex uses Gemini :embedContent / :batchEmbedContents, and Bedrock Titan embed uses InvokeModel with {inputText}. A Cohere registry accepts the same OpenAI-shaped request and adapts it to Cohere /v2/embed:

curl -s -X POST "$PROXY/$CON_SLUG/v1/embeddings" \
  -H "X-AG-Gateway-Slug: $GW_SLUG" -H "X-AG-API-Key: $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"embed-english-v3.0","input":["Hello from TrustGate"]}'

OpenAI-shaped clients call /{consumer}/v1/files for upload, list, retrieve, delete, and content download. OpenAI, Azure, OpenRouter, xAI, Mistral, and Anthropic registries that expose a Files API are forwarded as-is (Azure uses {endpoint}/openai/files?api-version=…; Anthropic uses https://api.anthropic.com/v1/files with x-api-key and anthropic-version). Providers without a Files store are filtered out of the pool. Retrieve, content, and delete then pin by file-id prefix: file_ stays on Anthropic; any other files-capable provider is treated as OpenAI-compatible (file-). A 404 from one store is retried on the remaining backends in that family so a file uploaded to OpenAI is still found if the next request would otherwise land on Mistral. List and upload still load-balance across files-capable registries:

# 8. Files (OpenAI, Azure OpenAI, OpenRouter, xAI, Mistral, Anthropic)
curl -s -X POST "$PROXY/$CON_SLUG/v1/files" \
  -H "X-AG-Gateway-Slug: $GW_SLUG" -H "X-AG-API-Key: $API_KEY" \
  -F purpose=assistants \
  -F file="@notes.txt"

# 9. Audio speech (TTS) — raw audio bytes
curl -s -X POST "$PROXY/$CON_SLUG/v1/audio/speech" \
  -H "X-AG-Gateway-Slug: $GW_SLUG" -H "X-AG-API-Key: $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"tts-1","input":"Hello from TrustGate","voice":"alloy"}' \
  --output speech.mp3

# 10. Audio transcriptions (STT)
curl -s -X POST "$PROXY/$CON_SLUG/v1/audio/transcriptions" \
  -H "X-AG-Gateway-Slug: $GW_SLUG" -H "X-AG-API-Key: $API_KEY" \
  -F model=whisper-1 \
  -F file="@speech.mp3"

# 11. Images (OpenAI, Azure OpenAI, openai_compatible, OpenRouter)
curl -s -X POST "$PROXY/$CON_SLUG/v1/images/generations" \
  -H "X-AG-Gateway-Slug: $GW_SLUG" -H "X-AG-API-Key: $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"dall-e-3","prompt":"A minimal TrustGate logo","n":1,"size":"1024x1024"}'
curl -s -X POST "$PROXY/$CON_SLUG/v1/images/edits" \
  -H "X-AG-Gateway-Slug: $GW_SLUG" -H "X-AG-API-Key: $API_KEY" \
  -F model=dall-e-2 -F prompt="make it blue" -F image=@logo.png
curl -s -X POST "$PROXY/$CON_SLUG/v1/images/variations" \
  -H "X-AG-Gateway-Slug: $GW_SLUG" -H "X-AG-API-Key: $API_KEY" \
  -F model=dall-e-2 -F image=@logo.png

# 12. Model discovery (OpenAI-compatible, gateway-owned)
curl -s "$PROXY/$CON_SLUG/v1/models" \
  -H "X-AG-Gateway-Slug: $GW_SLUG" -H "X-AG-API-Key: $API_KEY"
curl -s "$PROXY/$CON_SLUG/v1/models/gpt-4o-mini" \
  -H "X-AG-Gateway-Slug: $GW_SLUG" -H "X-AG-API-Key: $API_KEY"

OpenAI-shaped clients call POST /{consumer}/v1/audio/speech (TTS, raw audio bytes) and POST /{consumer}/v1/audio/transcriptions (STT, multipart file). OpenAI, Azure OpenAI, openai_compatible, OpenRouter, Groq, and Mistral registries that expose those APIs are forwarded as-is (Azure uses {endpoint}/openai/deployments/{model}/audio/{speech|transcriptions}?api-version=…). Mistral speech JSON {audio_data} is unwrapped to raw bytes so the gateway response stays OpenAI-shaped. /v1/audio/translations is not served yet. Providers without the matching audio capability are filtered out of the pool.

OpenAI-shaped clients call POST /{consumer}/v1/images/generations (JSON), plus multipart POST /{consumer}/v1/images/edits and POST /{consumer}/v1/images/variations. OpenAI, Azure, and openai_compatible registries forward the payload to the matching upstream images URL (Azure uses {endpoint}/openai/deployments/{model}/images/{generations|edits|variations}?api-version=…). OpenRouter registries map generations to POST /api/v1/images and keep edits/variations on /api/v1/images/edits and /api/v1/images/variations. Providers without an Images API are filtered out of the pool; pinning an incapable provider is a terminal 400.

GET /{consumer}/v1/models returns the union of native model ids the consumer can actually call (registries ∩ allowlists/policies ∩ provider capabilities). It is not an upstream /v1/models passthrough and not the full admin catalog.

Or use any OpenAI SDK — see examples/openai-sdk/.

Features

• High Performance — Built in Go on Fiber, tuned for low latency and high concurrency.
• Multi-Provider — OpenAI, Anthropic, Azure OpenAI, AWS Bedrock, Google Gemini, Vertex AI, Groq, Mistral, DeepSeek.
• Smart Routing — Round-robin, weighted, IP-hash strategies with health checks and fallback targets.
• Plugin System — Rate limiting, token rate limiting, request size guard, semantic cache, CORS.
• Semantic Cache — Embedding-based response caching for repeated prompts.
• Multi-Tenancy — Per-gateway consumers, API-key auth, scoped policies.
• Observability — Built-in metrics, OpenTelemetry (OTLP) export. See Telemetry Configuration.
• Independent Planes — Admin (:8080), Proxy (:8081), MCP (:8082) scale separately.

Architecture

TrustGate ships a single binary that boots one HTTP server per plane:

./trustgate              # proxy (default)
./trustgate admin        # admin
./trustgate mcp          # MCP server
./trustgate run          # admin + proxy together (single-node)

flowchart LR
    subgraph Clients["Clients & Agents"]
        APP["Apps / SDKs / Agents"]
    end

    subgraph AG["TrustGate"]
        direction TB
        ADMIN["Admin Plane :8080\nGateways · Registries · Consumers\nAuth · Policies · Catalog"]
        PROXY["Proxy Plane :8081\nRouting · Load Balancing\nPolicy Stages · Plugins"]
        MCP["MCP Plane :8082\nMCP targets & tools for agents"]
    end

    subgraph Plugins["Policy Plugins"]
        RL["Rate Limit"]
        TRL["Token Rate Limit"]
        RS["Request Size"]
        SC["Semantic Cache"]
        CORS["CORS"]
    end

    subgraph Providers["LLM Providers"]
        P1["OpenAI · Anthropic\nAzure · Bedrock"]
        P2["Gemini · Vertex\nGroq · Mistral"]
    end

    subgraph Infra["Infrastructure"]
        PG[("Postgres")]
        RD[("Redis")]
        KFK[["Kafka"]]
    end

    APP -->|API key| PROXY
    APP -->|MCP| MCP
    PROXY --> Plugins
    PROXY -->|load balance| Providers
    ADMIN -. config .-> PROXY
    ADMIN -. config .-> MCP
    ADMIN --- PG
    PROXY --- PG
    PROXY --- RD
    MCP --- PG
    PROXY -->|telemetry| KFK

MCP Plane for AI Agents

TrustGate's MCP plane (:8082) lets AI agents like Cursor and Claude connect to multiple MCP tool servers through a single gateway. Configure once, use everywhere.

// Cursor mcp.json example
{
  "mcpServers": {
    "trustgate": {
      "url": "http://localhost:8082/agent-client/mcp",
      "headers": {
        "X-AG-API-Key": "<consumer api key>"
      }
    }
  }
}

See examples/mcp-cursor/ for setup instructions, docs/mcp/testing-guide.md for the full guide, and docs/mcp/openapi.md for OpenAPI→MCP limits.

Providers

Plugins

Plugins run in ordered policy stages (sequential or parallel):

Configuration

All config is via environment variables. Copy .env.example to .env for development.

# Core ports
SERVER_ADMIN_PORT=8080
SERVER_PROXY_PORT=8081
SERVER_MCP_PORT=8082

# Infrastructure
DB_HOST=localhost
REDIS_HOST=localhost
KAFKA_BROKERS=localhost:9092

See .env.example for all options.

Observability

TrustGate emits request telemetry to OpenTelemetry collectors. Configure per-gateway OTLP exporters:

{
  "telemetry": {
    "exporters": [{
      "name": "otlp",
      "settings": {
        "endpoint": "collector:4317",
        "protocol": "grpc"
      }
    }]
  }
}

Full telemetry configuration, including default exporters and the OTLP contract, is documented in:

• docs/telemetry/otlp-metadata-contract.md
• docs/pricing.md — per-request cost.total_usd resolution (catalog, registry overrides, LLM Budget)
• config/telemetry.example.yaml

Local Development

# Boot infra in Docker, run planes locally (for debugging)
make compose-up
make run-admin      # terminal 1
make run-proxy      # terminal 2
make run-mcp        # terminal 3 (optional)

# Tests
make test           # unit tests
make test-race      # with race detector
make test-functional # against real server

See CONTRIBUTING.md for the full development guide.

make up   # admin :8080, proxy :8081 + Postgres/Redis/Kafka

ADMIN="http://localhost:8080"
PROXY="http://localhost:8081"
TOKEN="$ADMIN_TOKEN"   # see "Admin token" below

# 1. Create a gateway
GW=$(curl -s -X POST "$ADMIN/v1/gateways" \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d '{"name":"My Gateway","slug":"demo"}')
GW_ID=$(echo "$GW" | jq -r .id); GW_SLUG=$(echo "$GW" | jq -r .slug)

# 2. Register an upstream LLM provider
REG=$(curl -s -X POST "$ADMIN/v1/gateways/$GW_ID/registries" \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d '{"name":"openai-primary","provider":"openai",
       "auth":{"type":"api_key","api_key":{"api_key":"'"$OPENAI_API_KEY"'"}}}')
REG_ID=$(echo "$REG" | jq -r .id)

# 3. Create a consumer bound to that registry
CON=$(curl -s -X POST "$ADMIN/v1/gateways/$GW_ID/consumers" \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d '{"name":"my-app","registries":[{"id":"'"$REG_ID"'"}]}')
CON_ID=$(echo "$CON" | jq -r .id); CON_SLUG=$(echo "$CON" | jq -r .slug)

# 4. Mint a consumer API key
AUTH=$(curl -s -X POST "$ADMIN/v1/gateways/$GW_ID/auths" \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d '{"name":"my-app-key","type":"api_key"}')
AUTH_ID=$(echo "$AUTH" | jq -r .id); API_KEY=$(echo "$AUTH" | jq -r .api_key)

# 5. Attach the key to the consumer
curl -s -X POST "$ADMIN/v1/gateways/$GW_ID/consumers/$CON_ID/auths/$AUTH_ID" \
  -H "Authorization: Bearer $TOKEN"

# 6. Call the proxy
curl -s -X POST "$PROXY/$CON_SLUG/v1/chat/completions" \
  -H "X-AG-Gateway-Slug: $GW_SLUG" -H "X-AG-API-Key: $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"Hello!"}]}'

export SERVER_SECRET_KEY="$(grep ^SERVER_SECRET_KEY .env | cut -d= -f2-)"
export ADMIN_TOKEN=$(python3 - <<'PY'
import jwt, os, time
secret = os.environ["SERVER_SECRET_KEY"]
print(jwt.encode({"sub": "admin", "iat": int(time.time()), "exp": int(time.time()) + 3600}, secret, algorithm="HS256"))
PY
)

Repository Layout

cmd/trustgate/         # entry point (single binary: proxy | admin | mcp | run)
pkg/domain/            # domain entities and port interfaces
pkg/app/               # application services (use cases)
pkg/infra/providers/   # provider adapters (openai, anthropic, bedrock, …)
pkg/infra/plugins/     # policy plugins
pkg/server/            # Server interface + routers
examples/              # runnable examples for common use cases
docs/                  # API specs, telemetry docs, MCP guide

API Documentation

The Admin API ships Swagger 2.0 and OpenAPI 3 specs:

make swagger   # generate docs/swagger.{json,yaml}
make openapi   # convert to docs/openapi.json

FAQ

How is TrustGate different from LiteLLM and other AI gateways? TrustGate is the only AI gateway built by a security company — every design choice assumes the gateway is the substrate for security enforcement, not just an operational convenience. See the full comparison with LiteLLM, Kong, and others.

Is TrustGate free to use? Yes. The multi-protocol gateway engine (LLM, MCP, A2A routing, failover, retries, caching) and the control plane are open source under Apache 2.0. A team can run this in real production at no license cost. Paid tiers add SSO/RBAC, audit logging, long-term retention, and managed/hybrid deployment for organizations governing AI traffic across many teams.

What license is TrustGate released under? Apache 2.0.

What's the difference between TrustGate and TrustGuard? TrustGate is the gateway — routing, policy, and observability across LLM, MCP, and agent-to-agent traffic. TrustGuard is the security detection engine that attaches to a Route and inspects requests before they reach their target. TrustGate works standalone as an operational gateway; adding TrustGuard turns it into an enforcement point for runtime AI security.

Does it support OpenAI-compatible endpoints? Yes. Point any OpenAI SDK at the proxy with no client changes beyond the base URL and two headers — see Quick Start.

Does TrustGate govern MCP and agent-to-agent (A2A) traffic, or only LLM calls? All three. A model call, the MCP tool calls it triggers, and any A2A delegation resolve into one unified trace tree — not separate logs in separate systems.

Does TrustGate replace my existing API gateway (Kong, Apigee, AWS API Gateway)? No. TrustGate governs AI-specific traffic and is built to layer alongside an existing gateway, not replace it. General microservice traffic stays where it already is.

Is it production-ready? Yes. TrustGate holds ISO 27001 certification via NeuralTrust and is recognized in Gartner's 2025 Market Guide for AI Gateways.

Can I self-host it fully air-gapped? Yes. TrustGate deploys on-premises with no external calls required; all data stays within your infrastructure.

How TrustGate Compares

• The 10 Best AI Gateways for Enterprise AI Security in 2026
• vs. Kong
• vs. LiteLLM
• vs. Portkey
• vs. Apache APISIX
• vs. Solo.io agentgateway
• vs. HAProxy
• vs. MLflow
• vs. TrueFoundry
• vs. Google

Contributing

We welcome contributions! See CONTRIBUTING.md for guidelines.

Good first issues: Check .github/GOOD_FIRST_ISSUES.md for curated starter tasks.

Examples: Help us add more examples in examples/.

License

Apache License 2.0 — see LICENSE.

Community & Support

• Documentation
• Slack Community
• GitHub Issues
• Twitter
• Blog