FunASR Deployment Matrix

Use this page to choose the shortest deployment path for a product, demo, benchmark, or internal workflow. Start with the smallest surface that satisfies the job, then move to heavier runtimes only when throughput, latency, or integration requirements demand it.

Quick decision table

Path	Best for	Start here	Operational notes
Colab notebook	Browser smoke tests, first evaluation, shareable demos	Colab quickstart	No local setup; first run downloads model files, GPU runtime is faster.
Python API	Notebooks, offline jobs, first model evaluation	README quick start	Lowest ceremony; caller owns batching, retries, and files.
OpenAI-compatible API	Private speech API, agents, Dify/LangChain/AutoGen-style clients	OpenAI API example	Easiest integration for apps that already support OpenAI audio APIs.
Docker Compose API	Reproducible local smoke test or small internal service	OpenAI API Docker docs	CPU by default; adapt the image before using CUDA in containers.
Kubernetes API	Internal speech API for cluster services	Kubernetes template	Starts as private ClusterIP; add auth, TLS, network policy, and GPU scheduling before broader exposure.
Runtime WebSocket service	Live captions, meetings, call-center streams	Runtime service docs	Use when partial results, endpointing, or long-lived audio streams matter.
vLLM acceleration	Higher-throughput LLM-based ASR with Fun-ASR-Nano	vLLM guide	Use for LLM decoder throughput; does not apply to non-autoregressive Paraformer.
MCP server	Claude/Cursor/desktop agent speech tools	MCP example	Good when the ASR result should be exposed as a local tool.
Subtitle generator	SRT/VTT from long audio or video	Subtitle example	Use verbose segments and speaker labels when readability matters.
Batch ASR script	Archives, meetings, datasets, repeated offline runs	Batch example	Add queueing, manifests, and retry logs for production use.
Triton runtime	Specialized high-performance serving	Triton runtime docs	Heavier setup; choose when your team already operates Triton/GPU serving.

Common choices

I want to try FunASR in five minutes

Use the Colab quickstart when you want a browser-only smoke test, or use the Python API from the README for local work. It is the shortest route for validating installation, model download, device selection, and basic output shape. If you are unsure which model to start with, use the model selection guide.

I want a local replacement for cloud transcription

Use the OpenAI-compatible API. It exposes /v1/audio/transcriptions, /v1/models, /health, and Swagger docs. Start with sensevoice, run examples/openai_api/smoke_test.sh or examples/openai_api/smoke_test.py, then connect existing SDK or HTTP clients using client recipes and JavaScript/TypeScript recipes. For browser upload or microphone demos, use the Gradio browser demo. For Dify, n8n, HTTP nodes, or webhook workers, follow the workflow recipes. For API gateways, developer portals, and schema-driven imports, use the OpenAPI spec. Before sharing the service, review the security and gateway guide.

I want a repeatable container demo

Use examples/openai_api/docker-compose.yml for a CPU-mode smoke test:

cd examples/openai_api
cp .env.example .env
docker compose up --build

Keep CPU mode until you have a CUDA-capable PyTorch/FunASR image. After that, set FUNASR_DEVICE=cuda and verify with the same smoke test. Use python examples/openai_api/smoke_test.py --base-url http://localhost:8000 on systems without bash/curl.

I want an internal Kubernetes service

Use the Kubernetes template for a private ClusterIP OpenAI-compatible API with persistent model cache, /health probes, and a port-forward smoke-test path. Keep the CPU default until you have a CUDA-capable image and cluster GPU scheduling in place.

I need streaming or live captioning

Use the runtime WebSocket service. Validate chunk size, VAD, endpointing, punctuation, speaker diarization, reconnect behavior, and client backpressure with real audio before production rollout.

I need more LLM-based ASR throughput

Use the vLLM path for Fun-ASR-Nano. Benchmark with your own audio distribution and watch GPU memory, tensor parallel size, first-token latency, and warmup time.

Readiness checklist

• Pick a model alias and pin it in deployment notes.
• Record FunASR version, model version, device, CUDA/PyTorch version, Docker image tag, and command line.
• Run a short public smoke sample and at least one realistic private sample.
• Log audio duration, model, device, latency, response format, and error type for every request.
• Add upload-size limits, authentication, TLS, and rate limits before exposing an API outside a trusted network; use the security and gateway guide to plan the boundary.
• For streaming, test silence, noise, overlapping speakers, long sessions, reconnects, and slow clients.
• For benchmark claims, include input duration, hardware, batch size, model, runtime path, and whether model download/warmup time is excluded.

When to open an issue

Use Deployment Help for runtime, Docker, vLLM, Triton, Android, browser, or agent integration problems. Include your deployment path, exact command/config, logs, model, device, and audio characteristics.