Bedrock Claude AG-UI Server

October 20, 2025 · View on GitHub

A FastAPI server that implements the AG-UI protocol for Amazon Bedrock's Claude models. This server enables any AG-UI compatible client to interact with Claude models hosted on AWS Bedrock infrastructure.

Features

  • AG-UI Protocol Compatibility: Full implementation of the AG-UI protocol for seamless integration
  • Amazon Bedrock Integration: Direct integration with Bedrock's Claude models
  • Latest Claude Models: Full support for Claude 3.7+ and Claude 4.x (including Haiku 4.5) with cross-region inference profiles
  • Real-time Streaming: Server-Sent Events (SSE) for streaming responses
  • Tool Calling Support: Complete tool calling lifecycle with argument streaming
  • Backend Tools via MCP: Execute server-side tools through Model Context Protocol (see MCP Quick Start)
  • Context Support: Pass contextual information to enhance model responses (see Context Guide)
  • Comprehensive Error Handling: Structured error responses with detailed logging
  • Configurable Deployment: Flexible configuration via environment variables
  • CORS Support: Configurable CORS for cross-origin requests
  • Health Checks: Built-in health and readiness endpoints

Quick Start

Prerequisites

  1. Python 3.9 or higher
  2. AWS Account with Bedrock access
  3. AWS Credentials configured (see AWS Configuration)
  4. Bedrock Model Access: Ensure you have access to Claude models in your AWS region

Installation

# Clone the repository
git clone <repository-url>
cd bedrock-claude-ag-ui-server

# Install dependencies
poetry install

# Copy environment template
cp .env.example .env

# Edit .env with your configuration
nano .env

Using pip

# Clone the repository
git clone <repository-url>
cd bedrock-claude-ag-ui-server

# Create virtual environment
python -m venv .venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

# Copy environment template
cp .env.example .env

# Edit .env with your configuration
nano .env

AWS Credentials

The server supports multiple methods for AWS authentication:

  1. Environment Variables (recommended for production):

    export AWS_ACCESS_KEY_ID=your_access_key
export AWS_SECRET_ACCESS_KEY=your_secret_key
export AWS_REGION=us-east-1

  2. AWS Credentials File (~/.aws/credentials):

    [default]
aws_access_key_id = your_access_key
aws_secret_access_key = your_secret_key

  3. IAM Role (when running on AWS infrastructure like EC2, ECS, Lambda)

  4. AWS SSO (for development):

    aws sso login --profile your-profile
export AWS_PROFILE=your-profile

Verify Setup

Run the validation script to ensure everything is configured correctly:

poetry run python validate_setup.py

This will check:

  • AWS credentials are valid
  • Bedrock service is accessible
  • Claude model is available in your region
  • Configuration parameters are valid

Configuration

The server can be configured using environment variables or a .env file. Copy .env.example to .env and customize as needed:

cp .env.example .env

Configuration Options

AWS Configuration

  • AWS_REGION: AWS region for Bedrock API calls (default: us-east-1)
  • MODEL_ID: Bedrock Claude model identifier (default: anthropic.claude-3-5-sonnet-20241022-v2:0)
    • Supported models: Claude 4.x, 3.7+, 3.5, 3 Sonnet/Opus/Haiku
    • For Claude 3.7+ and 4.x: See Setup Guide
    • Examples: anthropic.claude-haiku-4-5-20251001-v1:0, anthropic.claude-3-7-sonnet-20250219-v1:0
  • USE_INFERENCE_PROFILE: Enable cross-region inference (default: auto-enabled for Claude 3.7+ and 4.x)
  • INFERENCE_PROFILE_ARN: Custom inference profile ARN (optional, auto-generated if empty)
  • AWS_ACCESS_KEY_ID: Optional AWS access key (leave empty to use default credentials)
  • AWS_SECRET_ACCESS_KEY: Optional AWS secret key (leave empty to use default credentials)

Server Configuration

  • PORT: Server port number (default: 8000, range: 1-65535)
  • HOST: Server host address (default: 0.0.0.0)
  • LOG_LEVEL: Logging level (default: INFO, options: DEBUG, INFO, WARNING, ERROR, CRITICAL)

Model Parameters

  • MAX_TOKENS: Maximum tokens for model response (default: 4096, range: 1-200000)
  • TEMPERATURE: Model temperature for response generation (default: 0.7, range: 0.0-1.0)
  • TOP_P: Top-p sampling parameter (default: 1.0, range: 0.0-1.0)
  • TOP_K: Top-k sampling parameter (default: 250, range: 0-500)

CORS Configuration

  • CORS_ORIGINS: Allowed CORS origins (default: ["*"])
    • Can be JSON array: ["http://localhost:3000", "https://example.com"]
    • Or comma-separated: http://localhost:3000, https://example.com
  • CORS_ALLOW_CREDENTIALS: Allow credentials in CORS requests (default: true)
  • CORS_ALLOW_METHODS: Allowed HTTP methods (default: ["*"])
  • CORS_ALLOW_HEADERS: Allowed headers (default: ["*"])

Request Configuration

  • REQUEST_TIMEOUT: Request timeout in seconds (default: 300)
  • MAX_CONCURRENT_REQUESTS: Maximum concurrent requests (default: 100)

Configuration Validation

The server validates configuration on startup, including:

  • AWS credentials and connectivity
  • Bedrock model availability in the specified region
  • Parameter ranges and formats

If validation fails, the server will not start and will display detailed error messages.

Usage

Starting the Server

Development Mode

# With auto-reload for development
poetry run uvicorn main:app --reload --port 8000

# Or using the CLI command
poetry run bedrock-claude-server --reload

Production Mode

# Using the CLI command
poetry run bedrock-claude-server

# Or directly with uvicorn
poetry run uvicorn main:app --host 0.0.0.0 --port 8000 --workers 4

Using Docker

# Build the image
docker build -t bedrock-claude-ag-ui-server .

# Run the container
docker run -p 8000:8000 \
  -e AWS_REGION=us-east-1 \
  -e AWS_ACCESS_KEY_ID=your_key \
  -e AWS_SECRET_ACCESS_KEY=your_secret \
  bedrock-claude-ag-ui-server

# Or using docker-compose
docker-compose up

Health Checks

The server provides health check endpoints:

# Basic health check
curl http://localhost:8000/health

# Readiness check (includes AWS connectivity)
curl http://localhost:8000/ready

API Documentation

Interactive Documentation

Once the server is running, visit:

Main Endpoint

POST / - Run agent with AG-UI protocol

Request Format

{
  "threadId": "thread-123",
  "runId": "run-456",
  "messages": [
    {
      "id": "msg-1",
      "role": "user",
      "content": "Hello, how are you?"
    }
  ],
  "tools": [],
  "context": [],
  "state": {},
  "forwardedProps": {}
}

Response Format

The server streams AG-UI events using Server-Sent Events (SSE):

data: {"type":"RUN_STARTED","threadId":"thread-123","runId":"run-456"}

data: {"type":"TEXT_MESSAGE_START","messageId":"msg-2","role":"assistant"}

data: {"type":"TEXT_MESSAGE_CONTENT","messageId":"msg-2","delta":"Hello! "}

data: {"type":"TEXT_MESSAGE_CONTENT","messageId":"msg-2","delta":"I'm doing well."}

data: {"type":"TEXT_MESSAGE_END","messageId":"msg-2"}

data: {"type":"RUN_FINISHED","threadId":"thread-123","runId":"run-456"}

Event Types

The server emits the following AG-UI event types:

  • RUN_STARTED: Signals the start of an agent run
  • RUN_FINISHED: Signals successful completion
  • RUN_ERROR: Signals an error occurred
  • TEXT_MESSAGE_START: Begins a text message
  • TEXT_MESSAGE_CONTENT: Streams text content chunks
  • TEXT_MESSAGE_END: Completes a text message
  • TOOL_CALL_START: Begins a tool call
  • TOOL_CALL_ARGS: Streams tool arguments
  • TOOL_CALL_END: Completes a tool call
  • TOOL_CALL_RESULT: Returns tool execution result

Client Integration Examples

Python Client

import requests
import json

def stream_agent_response(message: str):
    url = "http://localhost:8000/"
    
    payload = {
        "threadId": "thread-123",
        "runId": "run-456",
        "messages": [
            {
                "id": "msg-1",
                "role": "user",
                "content": message
            }
        ],
        "tools": [],
        "context": [],
        "state": {},
        "forwardedProps": {}
    }
    
    headers = {
        "Content-Type": "application/json",
        "Accept": "text/event-stream"
    }
    
    with requests.post(url, json=payload, headers=headers, stream=True) as response:
        for line in response.iter_lines():
            if line:
                line_str = line.decode('utf-8')
                if line_str.startswith('data: '):
                    event_data = json.loads(line_str[6:])
                    print(f"Event: {event_data['type']}")
                    
                    if event_data['type'] == 'TEXT_MESSAGE_CONTENT':
                        print(event_data['delta'], end='', flush=True)

# Usage
stream_agent_response("What is the capital of France?")

JavaScript/TypeScript Client

async function streamAgentResponse(message: string) {
  const response = await fetch('http://localhost:8000/', {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      'Accept': 'text/event-stream',
    },
    body: JSON.stringify({
      threadId: 'thread-123',
      runId: 'run-456',
      messages: [
        {
          id: 'msg-1',
          role: 'user',
          content: message,
        },
      ],
      tools: [],
      context: [],
      state: {},
      forwardedProps: {},
    }),
  });

  const reader = response.body?.getReader();
  const decoder = new TextDecoder();

  while (true) {
    const { done, value } = await reader!.read();
    if (done) break;

    const chunk = decoder.decode(value);
    const lines = chunk.split('\n');

    for (const line of lines) {
      if (line.startsWith('data: ')) {
        const event = JSON.parse(line.slice(6));
        console.log('Event:', event.type);

        if (event.type === 'TEXT_MESSAGE_CONTENT') {
          process.stdout.write(event.delta);
        }
      }
    }
  }
}

// Usage
streamAgentResponse('What is the capital of France?');

Using AG-UI Client Library

import { HttpAgent } from '@ag-ui/client';

const agent = new HttpAgent({
  url: 'http://localhost:8000/',
  agentId: 'bedrock-claude',
  threadId: 'thread-123',
});

// Add a message
agent.messages.push({
  id: 'msg-1',
  role: 'user',
  content: 'What is the capital of France?',
});

// Run the agent
const result = await agent.runAgent({
  runId: 'run-456',
  tools: [],
  context: [],
});

console.log('Response:', result);

Tool Calling Example

import requests
import json

def call_agent_with_tools():
    url = "http://localhost:8000/"
    
    # Define a weather tool
    weather_tool = {
        "name": "get_weather",
        "description": "Get current weather for a location",
        "parameters": {
            "type": "object",
            "properties": {
                "location": {
                    "type": "string",
                    "description": "City name"
                },
                "unit": {
                    "type": "string",
                    "enum": ["celsius", "fahrenheit"],
                    "description": "Temperature unit"
                }
            },
            "required": ["location"]
        }
    }
    
    payload = {
        "threadId": "thread-123",
        "runId": "run-456",
        "messages": [
            {
                "id": "msg-1",
                "role": "user",
                "content": "What's the weather in Paris?"
            }
        ],
        "tools": [weather_tool],
        "context": [],
        "state": {},
        "forwardedProps": {}
    }
    
    headers = {
        "Content-Type": "application/json",
        "Accept": "text/event-stream"
    }
    
    tool_calls = {}
    
    with requests.post(url, json=payload, headers=headers, stream=True) as response:
        for line in response.iter_lines():
            if line:
                line_str = line.decode('utf-8')
                if line_str.startswith('data: '):
                    event = json.loads(line_str[6:])
                    
                    if event['type'] == 'TOOL_CALL_START':
                        tool_calls[event['toolCallId']] = {
                            'name': event['toolCallName'],
                            'args': ''
                        }
                    
                    elif event['type'] == 'TOOL_CALL_ARGS':
                        tool_calls[event['toolCallId']]['args'] += event['delta']
                    
                    elif event['type'] == 'TOOL_CALL_END':
                        tool_id = event['toolCallId']
                        tool_name = tool_calls[tool_id]['name']
                        tool_args = json.loads(tool_calls[tool_id]['args'])
                        
                        print(f"Tool called: {tool_name}")
                        print(f"Arguments: {tool_args}")
                        
                        # Execute tool and send result back
                        # (In a real implementation, you would make another request
                        # with the tool result in the messages array)

# Usage
call_agent_with_tools()

Troubleshooting

Common Issues

1. AWS Credentials Not Found

Error: Unable to locate credentials

Solution:

  • Ensure AWS credentials are configured (see AWS Credentials)
  • Verify credentials with: aws sts get-caller-identity
  • Check environment variables: echo $AWS_ACCESS_KEY_ID

2. Bedrock Model Not Available

Error: Model not found in region

Solution:

  • Verify model access in AWS Console → Bedrock → Model access
  • Request access to Claude models if not already granted
  • Check the model ID matches available models in your region
  • Ensure you're using the correct AWS region

3. Connection Timeout

Error: Request timeout or Connection timeout

Solution:

  • Increase REQUEST_TIMEOUT in configuration
  • Check network connectivity to AWS Bedrock
  • Verify security group rules if running on AWS infrastructure
  • Check if you're behind a proxy that needs configuration

4. CORS Errors

Error: CORS policy: No 'Access-Control-Allow-Origin' header

Solution:

  • Add your client origin to CORS_ORIGINS in .env
  • Example: CORS_ORIGINS=["http://localhost:3000", "https://yourdomain.com"]
  • For development, you can use CORS_ORIGINS=["*"] (not recommended for production)

5. Invalid Configuration

Error: Configuration validation failed

Solution:

  • Run poetry run python validate_setup.py to see detailed errors
  • Check parameter ranges (e.g., TEMPERATURE must be 0.0-1.0)
  • Ensure required fields are set (AWS_REGION, MODEL_ID)
  • Verify JSON format for array values like CORS_ORIGINS

6. Rate Limiting

Error: ThrottlingException or Rate exceeded

Solution:

  • Implement exponential backoff in your client
  • Reduce MAX_CONCURRENT_REQUESTS in configuration
  • Request quota increase in AWS Service Quotas console
  • Consider using multiple AWS accounts for higher throughput

7. Streaming Issues

Problem: Events not streaming properly

Solution:

  • Ensure client accepts text/event-stream content type
  • Check for buffering in reverse proxies (nginx, CloudFlare)
  • Verify client properly handles SSE format
  • Test with curl: curl -N -H "Accept: text/event-stream" http://localhost:8000/

8. Tool Calling Not Working

Problem: Tools not being called or arguments malformed

Solution:

  • Verify tool schema follows JSON Schema format
  • Check tool descriptions are clear and specific
  • Ensure required parameters are marked correctly
  • Test tool definitions with simple examples first

Debug Mode

Enable debug logging for detailed troubleshooting:

# Set log level to DEBUG
export LOG_LEVEL=DEBUG

# Run server
poetry run bedrock-claude-server

Checking Logs

# View logs in real-time
tail -f logs/bedrock-claude-server.log

# Search for errors
grep ERROR logs/bedrock-claude-server.log

# View specific request
grep "run-456" logs/bedrock-claude-server.log

Testing Connectivity

# Test AWS connectivity
aws bedrock list-foundation-models --region us-east-1

# Test specific model
aws bedrock get-foundation-model \
  --model-identifier anthropic.claude-3-5-sonnet-20241022-v2:0 \
  --region us-east-1

# Test server health
curl http://localhost:8000/health

# Test server readiness (includes AWS check)
curl http://localhost:8000/ready

Getting Help

If you encounter issues not covered here:

  1. Check the GitHub Issues
  2. Review AWS Bedrock documentation
  3. Check AG-UI protocol documentation
  4. Enable debug logging and examine the output
  5. Run the validation script: poetry run python validate_setup.py

Performance Tuning

Production Recommendations

# Use multiple workers
poetry run uvicorn main:app --workers 4 --host 0.0.0.0 --port 8000

# Adjust concurrent requests
export MAX_CONCURRENT_REQUESTS=50

# Optimize timeout for your use case
export REQUEST_TIMEOUT=120

# Use appropriate model parameters
export TEMPERATURE=0.7
export MAX_TOKENS=4096

Monitoring

Monitor these metrics for optimal performance:

  • Request latency
  • Token usage
  • Error rates
  • Concurrent connections
  • AWS API throttling events

Security Best Practices

  1. Never commit credentials to version control
  2. Use IAM roles when running on AWS infrastructure
  3. Restrict CORS origins in production
  4. Enable HTTPS in production deployments
  5. Rotate AWS credentials regularly
  6. Use AWS Secrets Manager for credential management
  7. Implement rate limiting at the application or infrastructure level
  8. Monitor for unusual activity in CloudWatch

Requirements

  • Python 3.9+
  • AWS credentials configured
  • Access to Amazon Bedrock Claude models
  • Network access to AWS Bedrock API endpoints

License

MIT License