installation.md

Flash-TTS Installation Guide

This document provides a detailed walkthrough for installing and deploying the Flash-TTS inference engine, including environment requirements, model weight downloads, and dependency installation steps.

---

Environment Requirements

• Python: Version 3.10 or above

• Operating System: Linux x86_64, macOS, or Windows (WSL2 is recommended)

• Required Dependencies:

  • fastapi
  • At least one inference backend: vllm, sglang, llama-cpp-python, mlx-lm, or tensorrt-llm

---

Model Weight Downloads

Model	Hugging Face	ModelScope	GGUF
Spark-TTS	SparkAudio/Spark-TTS-0.5B	SparkAudio/Spark-TTS-0.5B	SparkTTS-LLM-GGUF
Orpheus-TTS	canopylabs/orpheus-3b-0.1-ft & hubertsiuzdak/snac_24khz	canopylabs/orpheus-3b-0.1-ft	orpheus-gguf
Orpheus-TTS (Multilingual)	orpheus-multilingual-research-release & hubertsiuzdak/snac_24khz	-	-
MegaTTS3	ByteDance/MegaTTS3	-	-

---

Dependency Installation

1. Install PyTorch

Visit the official PyTorch website to get the installation command suitable for your system and CUDA version. For example, for CUDA 12.4:

pip install torch==2.6.0 torchaudio==2.6.0 --index-url https://download.pytorch.org/whl/cu124

2. Install Flash-TTS

• Install via pip:

pip install flashtts

• Or install from source:

git clone https://github.com/HuiResearch/FlashTTS.git
cd FlashTTS
pip install .

Windows User Notice: If you encounter compilation errors when installing WeTextProcessing, you can install dependencies via Conda first:

conda install -c conda-forge pynini==2.1.6
pip install WeTextProcessing==1.0.4.1

---

Install Inference Backends (choose as needed)

vLLM (Recommended)

• Version ≥ 0.7.2 is required. For CUDA 12.4:

pip install vllm

• For other versions, refer to the vLLM official documentation

---

llama-cpp-python

pip install llama-cpp-python

• If using GGUF format weights, place the model.gguf file under the checkpoints/<model>/LLM/ directory.
• To convert weights, use the following commands:

git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
python convert_hf_to_gguf.py Spark-TTS-0.5B/LLM --outfile Spark-TTS-0.5B/LLM/model.gguf

---

sglang

pip install sglang

• For more information, refer to the sglang installation guide

---

mlx-lm (Apple Silicon Only)

pip install mlx-lm

• More info: mlx-lm GitHub project

---

TensorRT-LLM

Example for CUDA 12.4:

pip install tensorrt-llm --extra-index-url https://pypi.nvidia.com --extra-index-url https://download.pytorch.org/whl/cu124

Notes:

• TensorRT-LLM on Windows currently supports only Python 3.10.
• Latest supported version for Windows: 0.16.0 (as of 2025-05-05)
• Details: NVIDIA PyPI Repository

Verify installation:

python -c "import tensorrt_llm; print(tensorrt_llm.__version__)"

Convert LLM Weights to TensorRT Engine

1. Refer to the official model conversion docs: https://github.com/NVIDIA/TensorRT-LLM/tree/main/examples/models/core/

2. Choose the appropriate model type (e.g., Spark-TTS uses qwen).

3. After conversion, rename the output engine folder to tensorrt-engine and move it to the model directory, for example:

Spark-TTS-0.5B/LLM/tensorrt-engine

Flash-TTS can now load and infer from the converted model.

---

Backend Support Matrix

Inference Backend	Linux ✅	Windows ✅	macOS ✅	Notes
vllm	✅	❌	❌	Linux-only, requires CUDA
sglang	✅	❌	❌	Linux-only, supports most GPUs
tensorrt-llm	✅	⚠️	❌	Windows supports Python 3.10 only, version ≤ 0.16.0
llama-cpp	✅	✅	✅	GGUF format supported, cross-platform
mlx-lm	❌	❌	✅	macOS only (Apple Silicon)
torch	✅	✅	✅	Core dependency, supported on all platforms

⚠️ Notes:

• On Windows, WSL2 is recommended for full Linux feature support.
• On macOS, mlx-lm is not available for non-Apple Silicon chips.