English | 简体中文 | 繁體中文 | Русский

WhisperLive Real-Time Speech-to-Text on Docker

Part of the Self-Hosted AI Stack — deploy a complete self-hosted AI stack with a single command.

Docker image to run a WhisperLive real-time speech-to-text server, powered by faster-whisper. Provides WebSocket streaming for live audio transcription and an OpenAI-compatible REST API for file transcription. Based on Debian (python:3.12-slim). Designed to be simple, private, and self-hosted.

Features:

• Real-time WebSocket streaming — transcribe live microphone audio or audio streams with near-instant results
• OpenAI-compatible REST API — POST /v1/audio/transcriptions for file transcription; any app using the OpenAI Whisper API switches with a one-line change
• Supports all Whisper models: tiny, base, small, medium, large-v3, large-v3-turbo and more
• Voice Activity Detection (VAD) — automatically skips silence for faster, cleaner transcription
• Model management via a helper script (whisper_live_manage)
• Audio stays on your server — no data sent to third parties
• NVIDIA GPU (CUDA) acceleration for faster inference (:cuda image tag)
• Offline/air-gapped mode — run without internet access using pre-cached models (WHISPERLIVE_LOCAL_ONLY)
• Automatically built and published via GitHub Actions
• Persistent model cache via a Docker volume
• Multi-arch: linux/amd64, linux/arm64

Also available:

• AI stack: Self-Hosted AI Stack
• Try it online: Open in Colab — no Docker or installation required
• Related AI services: Whisper (batch STT), Kokoro (TTS), Embeddings, LiteLLM, Ollama (LLM), Docling, MCP Gateway

Community

• 📬 Subscribe for project updates (1–2 emails/month) — get free AI and VPN deployment guides (PDF)
• 💬 Join the r/selfhostedstack community for discussions and showcases
• ⭐ Star the repository if you find it useful — it helps others discover it

Other self-hosted projects: Setup IPsec VPN, IPsec VPN on Docker, WireGuard, OpenVPN, Headscale.

When to use WhisperLive vs. Whisper

Quick start

Use this command to set up a WhisperLive server:

docker run \
    --name whisper-live \
    --restart=always \
    -v whisper-live-data:/var/lib/whisper-live \
    -p 9090:9090 \
    -p 8000:8000 \
    -d hwdsl2/whisper-live-server

docker run \
    --name whisper-live \
    --restart=always \
    --gpus=all \
    -v whisper-live-data:/var/lib/whisper-live \
    -p 9090:9090 \
    -p 8000:8000 \
    -d hwdsl2/whisper-live-server:cuda

Note: For internet-facing deployments, using a reverse proxy to add HTTPS is strongly recommended. In that case, also replace -p 9090:9090 -p 8000:8000 with -p 127.0.0.1:9090:9090 -p 127.0.0.1:8000:8000 in the docker run command above, to prevent direct access to the unencrypted ports.

The Whisper base model (~145 MB) is downloaded and cached on first client connection. Check the logs to confirm the server is ready:

docker logs whisper-live

Once you see "WhisperLive real-time transcription server is ready":

Connect a real-time WebSocket client:

ws://your_server_ip:9090

Or transcribe a file via the REST API:

curl http://your_server_ip:8000/v1/audio/transcriptions \
    -F file=@audio.mp3 \
    -F model=whisper-1

Response:

{"text": "Your transcribed text appears here."}

Tip: Need a sample audio file to test the REST API? Download this English speech sample (WAV, MIT License) from the Azure Samples repository:

curl -L -o sample_speech.wav \
    "https://github.com/Azure-Samples/cognitive-services-speech-sdk/raw/master/sampledata/audiofiles/katiesteve.wav"

curl http://your_server_ip:8000/v1/audio/transcriptions \
    -F file=@sample_speech.wav \
    -F model=whisper-1

Requirements

• A Linux server (local or cloud) with Docker installed
• Supported architectures: amd64 (x86_64), arm64 (e.g. Raspberry Pi 4/5, AWS Graviton)
• Minimum RAM: ~700 MB free for the default base model (see model table)
• Internet access for the initial model download (the model is cached locally afterwards). Not required if using WHISPERLIVE_LOCAL_ONLY=true with pre-cached models.

For GPU acceleration (:cuda image):

• NVIDIA GPU with CUDA support (Compute Capability 6.0+)
• NVIDIA driver 575.57.08+ (Linux) or 576.57+ (Windows) installed on the host
• NVIDIA Container Toolkit installed
• The :cuda image supports linux/amd64 only

For internet-facing deployments, see Using a reverse proxy to add HTTPS.

Download

Get the trusted build from the Docker Hub registry:

docker pull hwdsl2/whisper-live-server

For NVIDIA GPU acceleration, pull the :cuda tag instead:

docker pull hwdsl2/whisper-live-server:cuda

Alternatively, you may download from Quay.io:

docker pull quay.io/hwdsl2/whisper-live-server
docker image tag quay.io/hwdsl2/whisper-live-server hwdsl2/whisper-live-server

Supported platforms: linux/amd64 and linux/arm64. The :cuda tag supports linux/amd64 only.

Environment variables

All variables are optional. Fresh installs with a mounted /var/lib/whisper-live volume auto-generate an API key. Existing installs without a key remain open for backward compatibility.

This Docker image uses the following variables, that can be declared in an env file (see example):

Note: In your env file, you may enclose values in single quotes, e.g. VAR='value'. Do not add spaces around =. If you change WHISPERLIVE_PORT or WHISPERLIVE_REST_PORT, update the -p flags in the docker run command accordingly.

Example using an env file:

cp whisper-live.env.example whisper-live.env
# Edit whisper-live.env with your settings, then:
docker run \
    --name whisper-live \
    --restart=always \
    -v whisper-live-data:/var/lib/whisper-live \
    -v ./whisper-live.env:/whisper-live.env:ro \
    -p 9090:9090 \
    -p 8000:8000 \
    -d hwdsl2/whisper-live-server

The env file is bind-mounted into the container, so changes are picked up on every restart without recreating the container.

docker run \
    --name whisper-live \
    --restart=always \
    -v whisper-live-data:/var/lib/whisper-live \
    -p 9090:9090 \
    -p 8000:8000 \
    --env-file=whisper-live.env \
    -d hwdsl2/whisper-live-server

Using docker-compose

cp whisper-live.env.example whisper-live.env
# Edit whisper-live.env as needed, then:
docker compose up -d
docker logs whisper-live

Example docker-compose.yml (already included):

services:
  whisper-live:
    image: hwdsl2/whisper-live-server
    container_name: whisper-live
    restart: always
    ports:
      - "9090:9090/tcp"  # WebSocket — for a host-based reverse proxy, change to "127.0.0.1:9090:9090/tcp"
      - "8000:8000/tcp"  # REST API  — for a host-based reverse proxy, change to "127.0.0.1:8000:8000/tcp"
    volumes:
      - whisper-live-data:/var/lib/whisper-live
      - ./whisper-live.env:/whisper-live.env:ro

volumes:
  whisper-live-data:
    name: whisper-live-data

Note: For internet-facing deployments, using a reverse proxy to add HTTPS is strongly recommended. In that case, also change "9090:9090/tcp" and "8000:8000/tcp" to their 127.0.0.1: equivalents in docker-compose.yml.

cp whisper-live.env.example whisper-live.env
# Edit whisper-live.env as needed, then:
docker compose -f docker-compose.cuda.yml up -d
docker logs whisper-live

services:
  whisper-live:
    image: hwdsl2/whisper-live-server:cuda
    container_name: whisper-live
    restart: always
    ports:
      - "9090:9090/tcp"  # WebSocket — for a host-based reverse proxy, change to "127.0.0.1:9090:9090/tcp"
      - "8000:8000/tcp"  # REST API  — for a host-based reverse proxy, change to "127.0.0.1:8000:8000/tcp"
    volumes:
      - whisper-live-data:/var/lib/whisper-live
      - ./whisper-live.env:/whisper-live.env:ro
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]

volumes:
  whisper-live-data:
    name: whisper-live-data

WebSocket streaming

The WebSocket endpoint at port 9090 supports real-time transcription of live audio streams. Clients send raw PCM audio chunks and receive transcription segments as they are decoded.

Protocol

On connection, send a JSON configuration message first:

{
  "uid": "unique-client-id",
  "language": "en",
  "model": "base",
  "use_vad": true
}

Then stream raw 16-bit PCM audio at 16 kHz sample rate as binary WebSocket frames. The server returns JSON transcription events:

{"uid": "unique-client-id", "segments": [{"text": "Hello, how are you?", "start": 0.0, "end": 2.4, "completed": true}]}

Python client example

from whisper_live.client import TranscriptionClient

client = TranscriptionClient(
    "your_server_ip",
    9090,
    lang="en",
    translate=False,
    model="base",
    use_vad=True,
)

# Transcribe from a file
client("audio.mp3")

# Or transcribe from microphone
# client()

Install the client library:

pip install whisper-live

Browser client example

const ws = new WebSocket("ws://your_server_ip:9090");

ws.onopen = () => {
  // Send configuration
  ws.send(JSON.stringify({
    uid: "browser-client-1",
    language: "en",
    model: "base",
    use_vad: true,
  }));
};

ws.onmessage = (event) => {
  const data = JSON.parse(event.data);
  if (data.segments) {
    data.segments.forEach(seg => console.log(seg.text));
  }
};

// Send audio chunks as ArrayBuffer (16-bit PCM, 16 kHz)
// ws.send(audioBuffer);

REST API reference

The REST API at port 8000 is compatible with OpenAI's audio transcription endpoint. Any application already calling https://api.openai.com/v1/audio/transcriptions can switch to self-hosted by setting:

OPENAI_BASE_URL=http://your_server_ip:8000

Transcribe audio

POST /v1/audio/transcriptions
Content-Type: multipart/form-data

Parameters:

Example:

curl http://your_server_ip:8000/v1/audio/transcriptions \
    -F file=@meeting.m4a \
    -F model=whisper-1 \
    -F language=en

Response:

{"text": "Your transcribed text appears here."}

Interactive API docs

An interactive Swagger UI is available at:

http://your_server_ip:8000/docs

Persistent data

All server data is stored in the Docker volume (/var/lib/whisper-live inside the container):

/var/lib/whisper-live/
├── models--Systran--faster-whisper-*/   # Cached Whisper model files (downloaded from HuggingFace)
├── .ws_port              # Active WebSocket port (used by whisper_live_manage)
├── .rest_port            # Active REST API port (used by whisper_live_manage)
├── .model                # Active model name (used by whisper_live_manage)
└── .server_addr          # Cached server IP (used by whisper_live_manage)

Back up the Docker volume to preserve downloaded models. Models are large (145 MB – 3 GB) and can take several minutes to download on first client connection; preserving the volume avoids re-downloading on container recreation.

Tip: The /var/lib/whisper-live volume uses the same HuggingFace cache layout as docker-whisper's /var/lib/whisper volume. If you have already downloaded a model with docker-whisper, you can bind-mount the same volume directory to avoid re-downloading.

Managing the server

Use whisper_live_manage inside the running container to inspect and manage the server.

Show server info:

docker exec whisper-live whisper_live_manage --showinfo

List available models:

docker exec whisper-live whisper_live_manage --listmodels

Pre-download a model:

docker exec whisper-live whisper_live_manage --downloadmodel large-v3-turbo

Switching models

To change the active model:

1. (Optional but recommended) Pre-download the new model while the server is running:

   docker exec whisper-live whisper_live_manage --downloadmodel large-v3-turbo
   

2. Update WHISPERLIVE_MODEL in your whisper-live.env file (or add -e WHISPERLIVE_MODEL=large-v3-turbo to your docker run command).

3. Restart the container:

   docker restart whisper-live
   

Available models:

Tip: large-v3-turbo offers accuracy close to large-v3 at roughly half the resource cost. It is the recommended upgrade path from base for most production deployments.

RAM figures are approximate and reflect INT8 quantization (default). Models are cached in the /var/lib/whisper-live Docker volume and only downloaded once.

Securing your server

If your WhisperLive server is reachable from the public internet — even briefly — apply at minimum these protections. WhisperLive is CPU/GPU-intensive, so an unprotected endpoint can be abused to burn your compute resources.

1. Use an API key. Fresh installs with a mounted /var/lib/whisper-live volume auto-generate an API key. Display it with docker exec whisper-live whisper_live_manage --showkey, or use docker exec whisper-live whisper_live_manage --getkey in scripts. Existing installs without a key remain open for backward compatibility; set WHISPERLIVE_API_KEY in your env file to enable authentication manually. REST clients must send Authorization: Bearer <key>; WebSocket clients can send the same header or add ?token=<key> to the URL.

2. Bind to localhost when fronted by a reverse proxy. Replace -p 9090:9090 -p 8000:8000 with -p 127.0.0.1:9090:9090 -p 127.0.0.1:8000:8000 (or change both port mappings to their 127.0.0.1: equivalents in docker-compose.yml) so the unencrypted ports are not reachable directly from outside the host.

3. Limit upload size at the proxy. Audio files can be large; configure your reverse proxy to reject oversized uploads (e.g. nginx client_max_body_size 100M;). This bounds the disk and memory footprint of a single request.

4. Mind the log level. WHISPERLIVE_LOG_LEVEL=DEBUG may write transcript text to logs. Keep it at INFO or higher on shared systems.

5. Restrict WebSocket origins at the proxy. WebSocket connections from untrusted browser origins can be blocked at the reverse proxy. For nginx, check $http_origin before upgrading the connection; for Caddy, use the header directive to validate the Origin header.

6. Consider rate limiting. Place a rate-limit (e.g. nginx limit_req_zone, Caddy rate_limit) in front of the server to cap concurrent transcriptions per client IP.

Using a reverse proxy

For internet-facing deployments, place a reverse proxy in front of the server to handle HTTPS and WSS (secure WebSocket) termination.

Use one of the following addresses to reach the container from your reverse proxy:

• whisper-live:9090 / whisper-live:8000 — if your reverse proxy runs as a container in the same Docker network.
• 127.0.0.1:9090 / 127.0.0.1:8000 — if your reverse proxy runs on the host and the ports are published.

Example with Caddy (Docker image) (automatic TLS, WebSocket proxying in the same Docker network):

Caddyfile:

whisper-live.example.com {
  # WebSocket streaming (wss://)
  handle /ws* {
    reverse_proxy whisper-live:9090
  }
  # REST API (https://)
  reverse_proxy whisper-live:8000
}

Example with nginx (reverse proxy on the host):

server {
    listen 443 ssl;
    server_name whisper-live.example.com;

    ssl_certificate     /path/to/cert.pem;
    ssl_certificate_key /path/to/key.pem;

    # REST API
    location /v1/ {
        proxy_pass         http://127.0.0.1:8000;
        proxy_set_header   Host $host;
        proxy_set_header   X-Real-IP $remote_addr;
        proxy_set_header   X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header   X-Forwarded-Proto $scheme;
        proxy_read_timeout 300s;
    }

    # WebSocket streaming
    location / {
        proxy_pass         http://127.0.0.1:9090;
        proxy_http_version 1.1;
        proxy_set_header   Upgrade $http_upgrade;
        proxy_set_header   Connection "upgrade";
        proxy_set_header   Host $host;
        proxy_read_timeout 600s;
    }
}

Important: WebSocket proxying requires proxy_http_version 1.1 and the Upgrade/Connection headers. Without these, real-time streaming will not work through nginx.

whisper-live.example.com {
  handle /v1/* {
    basicauth {
      user \$2a\$14$<bcrypt-hash-of-password>
    }
    reverse_proxy whisper-live:8000
  }
  handle /ws* {
    reverse_proxy whisper-live:9090
  }
  reverse_proxy whisper-live:8000
}

location /v1/ {
    auth_basic           "WhisperLive";
    auth_basic_user_file /etc/nginx/.htpasswd;
    proxy_pass           http://127.0.0.1:8000;
    proxy_set_header     Host $host;
    proxy_read_timeout   300s;
}

Update Docker image

To update the Docker image and container, first download the latest version:

docker pull hwdsl2/whisper-live-server

If the Docker image is already up to date, you should see:

Status: Image is up to date for hwdsl2/whisper-live-server:latest

Otherwise, it will download the latest version. Remove and re-create the container:

docker rm -f whisper-live
# Then re-run the docker run command from Quick start with the same volumes and ports.

Your downloaded models are preserved in the whisper-live-data volume.

Using with other AI services

WhisperLive can be used as the real-time speech-to-text service in a broader self-hosted AI setup.

For full and lightweight Docker Compose stacks, manual docker run examples, and voice/RAG/MCP pipeline examples with Kokoro, Embeddings, LiteLLM, Ollama, Docling, and MCP Gateway, see Self-Hosted AI Stack.

Technical details

• Base image: python:3.12-slim (Debian)
• Runtime: Python 3 (virtual environment at /opt/venv)
• STT engine: WhisperLive + faster-whisper with CTranslate2 (INT8 on CPU, FP16 on CUDA)
• VAD: Silero VAD via PyTorch (CPU or CUDA, auto-detected)
• WebSocket server: Python websockets library
• REST API framework: FastAPI + Uvicorn
• Data directory: /var/lib/whisper-live (Docker volume)
• Model storage: HuggingFace Hub format inside the volume — downloaded once, reused on restarts

License

Note: The software components inside the pre-built image (such as WhisperLive, faster-whisper, PyTorch, and their dependencies) are under the respective licenses chosen by their respective copyright holders. As for any pre-built image usage, it is the image user's responsibility to ensure that any use of this image complies with any relevant licenses for all software contained within.

Copyright (C) 2026 Lin Song
This work is licensed under the MIT License.

WhisperLive is Copyright (C) Vineet Suryan, Collabora Ltd., and is distributed under the MIT License.

faster-whisper is Copyright (C) SYSTRAN, and is distributed under the MIT License.

This project is an independent Docker setup and is not affiliated with, endorsed by, or sponsored by OpenAI, Collabora, or SYSTRAN.