Supported models

FreeToken loads HF safetensors checkpoints directly (plus native GGUF for Gemma-4). The checkpoints below are known-good — the prebuilt kernels are tuned for them; other checkpoints of the same architectures work too.

Model	HF checkpoints
DeepSeek-V4	deepseek-ai/DeepSeek-V4-Flash-0731
GLM-5.3-Flash	RedHatAI/GLM-5.3-Flash-NVFP4
GLM-5.2	nvidia/GLM-5.2-NVFP4
GLM-4.7	nvidia/GLM-4.7-NVFP4
Qwen3.8-Flash-Next	Qwen/Qwen3.8-Flash-Next-FP8, RadixArk/Qwen3.8-Flash-Next-NVFP4
Qwen3.6 / Qwen3.5 MoE	Qwen/Qwen3.6-35B-A3B (-FP8), nvidia/Qwen3.6-35B-A3B-NVFP4, Qwen/Qwen3.5-35B-A3B (-FP8)
Qwen3.8 / Qwen3.6 dense	Qwen/Qwen3.8-27B (-FP8), RadixArk/Qwen3.8-27B-NVFP4, Qwen/Qwen3.6-27B (-FP8), nvidia/Qwen3.6-27B-NVFP4
Qwen3-MoE	Qwen/Qwen3-30B-A3B
gpt-oss	openai/gpt-oss-120b, openai/gpt-oss-20b
Gemma-4	google/gemma-4-26B-A4B-it, nvidia/Gemma-4-26B-A4B-NVFP4, google/gemma-4-12B-it, nvidia/Gemma-4-31B-IT-NVFP4 ..
MiniMax-M2.5	nvidia/MiniMax-M2.5-NVFP4
Muse-Glimmer	meta-models/Muse-Glimmer-30B, RedHatAI/Muse-Glimmer-30B-NVFP4

MoE backends

ft serve --moe-backend {auto,fused,offload,cpu,hybrid}:

• fused — experts resident on GPU (needs the VRAM); never auto-selected.
• offload — experts live in host RAM, an LRU cache of expert slots on GPU; misses stream over PCIe.
• cpu — misses are computed on the CPU instead of fetched.
• hybrid — per step, fetches some misses over PCIe and computes the rest on CPU, overlapped. Run ft bench bw once per machine to calibrate the split.
• auto — dense models always resolve to fused; MoE models resolve to offload, upgraded to hybrid when a cached ft bench bw profile recommends it.

Notes

• ft checkpoint conversion is optional — it pre-converts a checkpoint into FreeToken's fast-load format, and ft serve --model auto-detects the result.
• DeepSeek-V4 checkpoints must keep the inference/config.json subdir — the authoritative model args are read from there.
• Qwen3.8-Flash-Next keeps a 47.7 GiB PLE n-gram table pinned in host RAM.
• Multimodal checkpoints are served text-only.