Orpheus TTS FastAPI Server (Async)

November 15, 2025 · View on GitHub

A high-performance, production-ready FastAPI-based server that provides OpenAI-compatible Text-to-Speech (TTS) endpoints using the Orpheus TTS model with advanced error detection and async parallel processing. This project uses the original orpheus-speech python package with vLLM backend, loading the model in bfloat16 by default (with float16/float32 options). Using higher precision formats requires more VRAM but eliminates audio quality issues and artifacts commonly found in quantized models or alternative inference engines.

The server features sophisticated multi-stage error detection, adaptive retry logic, and comprehensive debugging tools to ensure reliable, high-quality audio generation. Supports async parallel chunk processing with intelligent text chunking that preserves dialogue and narrative flow.

🚀 Features

OpenAI-compatible API: Drop-in replacement for OpenAI's TTS API
Async Parallel Processing: Process multiple text chunks simultaneously for faster generation
Direct vLLM Integration: Uses vLLM's AsyncLLMEngine for optimal performance
Intelligent Text Chunking: Preserves dialogue, attribution, and narrative flow
Complete Audio Files: Returns complete WAV files optimized for quality
Advanced Error Detection: Multi-stage analysis prevents audio artifacts and quality issues
Adaptive Retry Logic: Up to 5 automatic retries with parameter adjustment
Token Repetition Detection: Prevents infinite audio loops with pattern analysis
Audio Quality Analysis: Detects silence, repetition, stretching, and monotonic audio
Duration Outlier Detection: Identifies abnormally long audio generation
Debug & Success Logging: Optional comprehensive logging for troubleshooting and tuning
SNAC Audio Decoding: High-quality audio reconstruction from tokens

🔧 Architecture

Modular Design

The server follows a modular architecture with specialized components:

fastapi_app.py: Main FastAPI application with request handling, CORS, and timeout middleware
audio_generator.py: Advanced audio generation with multi-stage error detection and retry logic
audio_decoder.py: SNAC-based audio decoding with custom exception handling
text_processor.py: Intelligent text chunking that preserves dialogue and narrative flow
audio_analysis.py: Comprehensive audio quality analysis using spectrograms and cross-correlation

Async Processing Pipeline

┌─────────────────────────────────────────────────┐
│ FastAPI Request Handler (Async)                 │
├─────────────────────────────────────────────────┤
│ Text Processing & Chunking (Intelligent)        │
├─────────────────────────────────────────────────┤
│ Parallel Token Generation (vLLM AsyncEngine)    │
├─────────────────────────────────────────────────┤
│ Multi-Stage Error Detection:                    │
│ • Token Repetition Detection                    │
│ • Token Count Ratio Analysis                    │
│ • Audio Duration Outlier Detection              │
│ • Audio Quality Analysis (Silence/Stretch/Mono) │
├─────────────────────────────────────────────────┤
│ Adaptive Retry Logic (5 attempts max)           │
├─────────────────────────────────────────────────┤
│ SNAC Audio Decoding                             │
├─────────────────────────────────────────────────┤
│ Audio File Generation (Async I/O)               │
├─────────────────────────────────────────────────┤
│ Debug/Success Logging (Optional)                │
└─────────────────────────────────────────────────┘

Key Improvements Over Sync Version

4x faster for long texts (parallel chunk processing)
Non-blocking operations throughout the pipeline
Better resource utilization with optimized thread pools
GPU memory monitoring and automatic optimization
Advanced error detection with multi-stage analysis to prevent audio artifacts
Intelligent retry logic with adaptive parameter adjustment
Comprehensive debugging with audio quality analysis and metadata logging

📋 API Endpoints

Core Endpoints

Endpoint	Method	Description
`/v1/audio/speech`	POST	Generate speech from text (OpenAI-compatible)
`/health`	GET	Health check and model status
`/`	GET	API information and available endpoints

Interactive Documentation

Swagger UI: http://localhost:8880/docs

🛠️ Installation

Prerequisites

CUDA-capable GPU (minimum 16GB VRAM recommended)
Sufficient disk space for model downloads

Install Dependencies

Install uv

curl -LsSf https://astral.sh/uv/install.sh | sh

Create a virtual environment with Python 3.12:

uv venv --python 3.12

Activate the virtual environment:

source .venv/bin/activate

Install dependencies

# Install the required packages
uv pip install -r requirements.txt

Get access to the Orpheus model from here
Login to Huggingface using the CLI after gaining access to the repo

huggingface-cli login

🔧 Configuration

Environment Variables

The server supports extensive configuration through environment variables. Copy .env.sample to .env and modify as needed:

cp .env.sample .env
# Edit .env with your preferred settings

🎯 Usage

Starting the Server

uvicorn fastapi_app:app --host 0.0.0.0 --port 8880

The server will start on http://localhost:8880 by default.

Making Requests

Using cURL

# Generate speech
curl -X POST "http://localhost:8880/v1/audio/speech" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "orpheus",
    "input": "Hello, this is a test of the Orpheus TTS API!",
    "voice": "tara",
    "response_format": "wav",
    "speed": 1.0
  }' \
  --output speech.wav

Using Python with requests

import requests

response = requests.post(
    "http://localhost:8880/v1/audio/speech",
    json={
        "model": "orpheus",
        "input": "Hello, this is a test!",
        "voice": "tara",
        "response_format": "wav",
        "speed": 1.0
    }
)

with open("output.wav", "wb") as f:
    f.write(response.content)

Using OpenAI Client Library

from openai import OpenAI

# Configure client to use local server
client = OpenAI(
    api_key="dummy-key",  # Not validated by local server
    base_url="http://localhost:8880"
)

response = client.audio.speech.create(
    model="orpheus",
    voice="tara",
    input="Hello, this is a test!"
)

with open("speech.wav", "wb") as f:
    f.write(response.content)

🎤 Voice Options

The API supports native Orpheus voice names:

Available Voices

tara (default) - neutral/balanced female
leah - warm female voice
jess - expressive female
leo - deep male voice
dan - male voice
mia - young female voice
zac - male voice
zoe - female voice

🔧 API Reference

POST /v1/audio/speech

Generate speech from text with automatic parallel processing and advanced error detection for long texts.

Request Body:

{
  "model": "orpheus",         // Model to use
  "input": "Text to speak",   // Text to synthesize (auto-chunked if long)
  "voice": "tara",           // Voice name (see voice options above)
  "response_format": "wav",   // Audio format (currently only "wav")
  "speed": 1.0,              // Speech speed (0.25 to 4.0)

  // Optional: Override environment defaults for sampling
  "temperature": 0.2,         // Sampling temperature (0.0-2.0, optional)
  "top_p": 0.9,              // Top-p nucleus sampling (0.0-1.0, optional)
  "repetition_penalty": 1.1,  // Repetition penalty (0.5-2.0, optional)
  "max_tokens": 4096         // Maximum tokens to generate (100-8192, optional)
}

Features:

Automatic Text Chunking: Long texts (>500 chars) are intelligently split preserving dialogue and narrative flow
Parallel Processing: Multiple chunks processed simultaneously with individual retry logic
Multi-Stage Error Detection: Advanced analysis prevents audio artifacts:
- Token repetition detection (prevents infinite loops)
- Token count ratio analysis (catches outlier generation)
- Audio duration outlier detection (identifies abnormally slow generation)
- Audio quality analysis (detects silence, repetition, stretching, monotonic audio)
Adaptive Retry Logic: Up to 5 automatic retries with parameter adjustment for failed chunks
Seamless Audio: Token-level combination creates single WAV file with natural flow
Debug Logging: Optional comprehensive logging of generation process and error analysis
Configurable Parameters: Override defaults per request with environment fallbacks

Response:

Content-Type: audio/wav
Body: Binary audio data

Error Handling: The API implements sophisticated error detection and recovery:

Token-Level Errors: Repetition patterns, count outliers, invalid ranges
Audio-Level Errors: Duration outliers, quality issues (silence, repetition, stretching)
Adaptive Retries: Automatic parameter adjustment (temperature, repetition_penalty) on retry attempts
Graceful Degradation: Failed chunks are logged and skipped rather than failing entire request
Debug Output: Failed generations saved to debug_audio_errors/ for analysis

GET /health

Health check endpoint with detailed system information.