AI SDK Python - FastAPI Example Fix Plan

August 26, 2025 · View on GitHub

Current Status: CRITICAL RUNTIME ISSUES REMAIN ❌

Latest Session Update (August 24, 2025 - Session 43)

GOAL: Get FastAPI example fully functional with real OpenAI integration

✅ IMPORT ISSUES COMPLETELY RESOLVED

Successfully fixed 40+ import and provider instantiation errors:

Provider Abstract Method Issues Fixed:

  • ✅ DeepInfraProvider: Added missing name property
  • ✅ AssemblyAIProvider: Added missing name property + lazy initialization
  • ✅ FalProvider: Added missing name property + lazy initialization
  • ✅ HumeProvider: Added missing name property + lazy initialization
  • ✅ LMNTProvider: Added missing name property + fixed BaseProvider inheritance
  • ✅ FireworksProvider: Added missing name property
  • ✅ CerebrasProvider: Added missing name property
  • ✅ ReplicateProvider: Added missing name property
  • ✅ GladiaProvider: Added missing name property
  • ✅ LumaProvider: Added missing name property

Missing Type Definitions Added:

  • ✅ src/ai_sdk/core/types.py: Added comprehensive compatibility shim (500+ lines)
  • ✅ src/ai_sdk/providers/types.py: Added TextPart, ToolCallPart, ToolResultPart, EmbedResult, EmbedManyResult, EmbeddingUsage
  • ✅ src/ai_sdk/utils/json.py: Added parse_json_response, handle_json_parse_error, parse_json_chunk, parse_json_stream, ensure_json_parsable
  • ✅ src/ai_sdk/core/generate_speech.py: Added SpeechGenerationResult, GenerateSpeechUsage classes
  • ✅ src/ai_sdk/core/generate_image.py: Added GenerateImageUsage class
  • ✅ src/ai_sdk/core/transcribe.py: Added TranscriptionSegment, TranscribeResult, Warning classes
  • ✅ src/ai_sdk/schemas/__init__.py: Added AudioData class
  • ✅ src/ai_sdk/tools/mcp/__init__.py: Added create_mcp_client factory
  • ✅ src/ai_sdk/errors/__init__.py: Added NoSuchModelError, APICallError aliases
  • ✅ src/ai_sdk/core/step.py: Added StreamPart, Step classes
  • ✅ src/ai_sdk/streaming/: Created complete streaming module with stream.py, base.py
  • ✅ src/ai_sdk/core/embed.py: Added EmbedOptions class

Tool System Fixed:

  • ✅ Fixed @tool decorator usage in fastapi_example.py to use @simple_tool with proper schema
  • ✅ Updated tool function signatures to match AI SDK expectations

✅ APPLICATION STARTS SUCCESSFULLY

  • ✅ FastAPI server starts on http://localhost:8001
  • ✅ Health endpoint returns proper response: {"status":"healthy","ai_sdk":"python","provider":"openai"}
  • ✅ All endpoints are registered and accessible

❌ CRITICAL RUNTIME ISSUES DISCOVERED

1. Chat Endpoints Don't Work

curl -X POST "http://localhost:8001/chat" -H "Content-Type: application/json" -d '{"message": "Hello"}'
# Returns: 422 Unprocessable Entity - JSON decode error

Server logs show:

  • INFO: 127.0.0.1:57234 - "POST /chat HTTP/1.1" 422 Unprocessable Entity
  • INFO: 127.0.0.1:57247 - "POST /chat HTTP/1.1" 500 Internal Server Error

2. Request Body Parsing Issues

  • FastAPI can't parse the JSON request body properly
  • Getting "JSON decode error" and "Invalid \escape" errors
  • Suggests the Pydantic request models are not properly defined

3. Internal Server Errors

  • When JSON does parse, getting 500 Internal Server Error
  • Likely issues with the actual AI SDK integration or OpenAI API calls

4. Missing Request/Response Models The FastAPI example may be missing proper Pydantic models for:

  • Chat request/response schemas
  • Tool calling request/response schemas
  • Streaming response handling
  • Error handling and validation

IMMEDIATE NEXT STEPS

1. Fix Request/Response Models (HIGHEST PRIORITY)

  • Examine the FastAPI example's Pydantic models
  • Ensure proper JSON schema validation
  • Fix any missing or incorrect field definitions
  • Test request parsing with proper JSON payloads

2. Debug OpenAI Integration

  • Check if OPENAI_API_KEY is properly configured
  • Verify the AI SDK provider instantiation works
  • Test basic generate_text() calls outside FastAPI context
  • Add proper error handling and logging

3. Fix API Endpoint Implementation

  • Review each endpoint's implementation
  • Ensure proper async/await patterns
  • Add request validation and error handling
  • Test each endpoint individually

4. Validate HTML Interface

  • Check if the HTML interface at http://localhost:8001/ works
  • Test the JavaScript client-side API calls
  • Ensure proper CORS handling if needed

Previous Sessions Summary

Session 42 - Import Resolution Success ✅

  • ✅ CRITICAL IMPORT FIXES: Successfully resolved major blocking import errors
  • ✅ Provider Compliance: Fixed abstract method issues across multiple providers
  • ✅ Repository Status: Major functionality restored - core imports and provider creation work

Session 41 - Repository Health ✅

  • ✅ Repository Health: Comprehensive validation completed - all 568 Python files compile
  • ✅ Code Quality: Perfect syntax validation across all source files
  • ✅ TypeScript Sync: Complete parity with latest TypeScript ai-sdk maintained

Session 40 - Circular Import Resolution ✅

  • ✅ Breakthrough: Resolved deep circular import issues through lazy loading
  • ✅ Architecture: Implemented comprehensive lazy import system in ai_sdk/__init__.py
  • ✅ Compatibility: Created extensive compatibility shims for missing types

Key Technical Debt

1. API Integration Testing

  • No end-to-end testing of actual OpenAI API calls
  • Missing validation of request/response flows
  • No error handling for API failures

2. Request Validation

  • Pydantic models may not match actual request formats
  • JSON parsing issues suggest schema mismatches
  • Missing proper FastAPI dependency injection

3. Production Readiness

  • No proper logging or monitoring
  • Missing rate limiting and error handling
  • No authentication or security measures

Files That Need Immediate Attention

CRITICAL - Blocking Runtime Issues

  1. fastapi_example.py - Fix request/response models and endpoint implementations
  2. OpenAI API integration - Verify provider setup and API key handling
  3. FastAPI dependency injection - Fix request parsing and validation

HIGH PRIORITY - Core Functionality

  1. src/ai_sdk/integrations/fastapi.py - May need fixes for proper FastAPI integration
  2. src/ai_sdk/core/generate_text.py - Verify basic text generation works
  3. Error handling throughout the request pipeline

MEDIUM PRIORITY - User Experience

  1. HTML interface at root endpoint - Test and fix JavaScript client
  2. WebSocket chat functionality - Verify real-time communication
  3. Tool calling system - Test weather and calculator tools

Success Metrics

✅ COMPLETED

  • AI SDK imports work without errors
  • Provider instantiation succeeds
  • FastAPI server starts successfully
  • Health endpoint responds correctly

❌ REMAINING CRITICAL

  • Chat endpoints accept and process requests
  • OpenAI API integration returns responses
  • HTML interface loads and functions
  • Tool calling works end-to-end
  • WebSocket chat operates correctly

Next Session Priority

FOCUS: Get the chat endpoint working with real OpenAI responses

  1. Debug JSON request parsing - Fix the 422 errors
  2. Test OpenAI provider directly - Verify API key and basic calls work
  3. Fix endpoint implementations - Ensure proper async handling
  4. Add comprehensive error handling - Better debugging info
  5. Test the complete user flow - From HTML interface to AI response

GOAL: User can open http://localhost:8001, type a message, and get a real response from OpenAI.

Architecture Status

✅ SOLID FOUNDATION

  • Import system completely functional
  • Provider architecture working
  • FastAPI integration layer operational
  • Comprehensive type system in place

❌ RUNTIME GAPS

  • Request/response handling broken
  • API integration not verified
  • User interface not tested
  • Error handling insufficient

The foundation is solid, but the application layer needs work to be production-ready.