π³ MiniCPM-V & o Cookbook
May 8, 2026 Β· View on GitHub
π Main Repository | π Full Documentation
Cook up amazing multimodal AI applications effortlessly with MiniCPM-o, bringing vision, speech, and live-streaming capabilities right to your fingertips! For version-specific deployment instructions, see the files in the deployment directory.
β¨ What Makes Our Recipes Special?
Easy Usage Documentation
Our comprehensive documentation website presents every recipe in a clear, well-organized manner. All features are displayed at a glance, making it easy for you to quickly find exactly what you need.
Broad User Spectrum
We support a wide range of users, from individuals to enterprises and researchers.
- Individuals: Enjoy effortless inference using Ollama and Llama.cpp with minimal setup.
- Enterprises: Achieve high-throughput, scalable performance with vLLM and SGLang.
- Researchers: Leverage advanced frameworks including Transformers , LLaMA-Factory, SWIFT, and Align-anything to enable flexible model development and cutting-edge experimentation.
Versatile Deployment Scenarios
Our ecosystem delivers optimal solution for a variety of hardware environments and deployment demands.
- Web demo: Launch interactive multimodal AI web demo with FastAPI.
- Quantized deployment: Maximize efficiency and minimize resource consumption using GGUF, BNB, and AWQ.
- Edge devices: Bring powerful AI experiences to iPhone and iPad, supporting offline and privacy-sensitive applications.
βοΈ Live Demonstrations
Explore real-world examples of MiniCPM-V deployed on edge devices using our curated recipes. These demos highlight the modelβs high efficiency and robust performance in practical scenarios.
- Run locally on iPhone with iOS demo.
Β Β Β Β
- Run locally on iPad with iOS demo, observing the process of drawing a rabbit.
π₯ Inference Recipes
Ready-to-run examples
| Recipe | Description |
|---|---|
| Vision Capabilities (MiniCPM-V 4.6) | |
| πΌοΈ Single-image QA | Question answering on a single image |
| 𧩠Multi-image QA | Question answering with multiple images |
| π¬ Video QA | Video-based question answering |
| π Document Parser | Parse and extract content from PDFs and webpages |
| π Text Recognition | Reliable OCR for photos and screenshots |
| π― Grounding | Visual grounding and object localization in images |
| Audio Capabilities (MiniCPM-o) | |
| π€ Speech-to-Text | Multilingual speech recognition |
| π£οΈ Text-to-Speech | Instruction-following speech synthesis |
| π Voice Cloning | Realistic voice cloning and role-play |
ποΈ Fine-tuning Recipes
Customize your model with your own ingredients
Data preparation
Follow the guidance to set up your training datasets.
Training
We provide training methods serving different needs as following:
| Framework | Description |
|---|---|
| Transformers | Most flexible for customization |
| LLaMA-Factory | Modular fine-tuning toolkit |
| SWIFT | Lightweight and fast parameter-efficient tuning |
| Align-anything | Visual instruction alignment for multimodal models |
π¦ Serving Recipes
Deploy your model efficiently
| Method | Description |
|---|---|
| vLLM | High-throughput GPU inference |
| SGLang | High-throughput GPU inference |
| Llama.cpp | Fast CPU inference on PC, iPhone and iPad |
| Ollama | User-friendly setup |
| OpenWebUI | Interactive Web demo with Open WebUI |
| Gradio | Interactive Web demo with Gradio |
| FastAPI | Interactive Omni Streaming demo with FastAPI |
| iOS | Interactive iOS demo with llama.cpp |
π₯ Quantization Recipes
Compress your model to improve efficiency
| Format | Key Feature |
|---|---|
| GGUF | Simplest and most portable format |
| BNB | Simple and easy-to-use quantization method |
| AWQ | High-performance quantization for efficient inference |
| GPTQ | Weight-only INT4 with vLLM-compatible packaging (v4.5 only β Qwen3 backbone) |
Framework Support Matrix
The latest release is MiniCPM-V 4.6 (Instruct + Thinking). The matrix below tracks v4.6 first, with v4.5 / o4.5 rows kept for reference.
| Model | Category | Framework | Cookbook Link | Upstream PR | Supported since (branch) | Supported since (release) |
|---|---|---|---|---|---|---|
| MiniCPM-V 4.6 (latest) | Inference | Transformers | Transformers Doc | huggingface/transformers (merged) | main | v5.7.0 |
| Edge (On-device) | Llama.cpp | Llama.cpp Doc | #22529 (2026-05-06) | master (2026-05-06) | b9049 | |
| Ollama | Ollama Doc | Merging | Merging | Waiting for official release | ||
| Serving (Cloud) | vLLM | vLLM Doc | #41254 (2026-04-29) | Merging | Waiting for official release | |
| SGLang | SGLang Doc | Merging | Merging | Waiting for official release | ||
| MiniCPM-o 4.5 | Edge (On-device) | Llama.cpp | Llama.cpp Doc | #19211(2026-01-30) | master(2026-01-30) | b7895 |
| Ollama | Ollama Doc | #12078(2025-08-26) | Merging | Waiting for official release | ||
| Serving(Cloud) | vLLM | vLLM Doc | #33431(2026-01-30) | Merging | Waiting for official release | |
| SGLang | SGLang Doc | #9610(2025-08-26) | Merging | Waiting for official release | ||
| Cross-version | Finetuning | LLaMA-Factory | LLaMA-Factory Doc | #9022 (2025-08-26) | main (2025-08-26) | Waiting for official release |
| Quantization | GGUF | GGUF Doc | β | β | β | |
| BNB | BNB Doc | β | β | β | ||
| AWQ | AWQ Doc | tc-mb/AutoAWQ | β | β | ||
| GPTQ | GPTQ Doc | openbmb/MiniCPM-V-4_5-GPTQ | β | β | ||
| Demos | Gradio Demo | Gradio Demo Doc | β | β | β |
If you'd like us to prioritize support for another open-source framework, please let us know via this short form.
Awesome Works using MiniCPM-V & o
- text-extract-api: Document extraction API using OCRs and Ollama supported models GitHub Repo stars
- comfyui_LLM_party: Build LLM workflows and integrate into existing image workflows GitHub Repo stars
- Ollama-OCR: OCR package uses vlms through Ollama to extract text from images and PDF GitHub Repo stars
- comfyui-mixlab-nodes: ComfyUI node suite supports Workflow-to-APPγGPT&3D and more GitHub Repo stars
- OpenAvatarChat: Interactive digital human conversation implementation on single PC GitHub Repo stars
- pensieve: A privacy-focused passive recording project by recording screen content GitHub Repo stars
- paperless-gpt: Use LLMs to handle paperless-ngx, AI-powered titles, tags and OCR GitHub Repo stars
- Neuro: A recreation of Neuro-Sama, but running on local models on consumer hardware GitHub Repo stars
π₯ Community
Contributing
We love new recipes! Please share your creative dishes:
- Fork the repository
- Create your recipe
- Submit a pull request
Issues & Support
- Found a bug? Open an issue
- Need help? Join our Discord
Institutions
This cookbook is developed by OpenBMB and OpenSQZ.
π License
This cookbook is served under the Apache-2.0 License - cook freely, share generously! π³
Citation
If you find our model/code/paper helpful, please consider citing our papers π and staring us βοΈοΌ
@misc{yu2025minicpmv45cookingefficient,
title={MiniCPM-V 4.5: Cooking Efficient MLLMs via Architecture, Data, and Training Recipe},
author={Tianyu Yu and Zefan Wang and Chongyi Wang and Fuwei Huang and Wenshuo Ma and Zhihui He and Tianchi Cai and Weize Chen and Yuxiang Huang and Yuanqian Zhao and Bokai Xu and Junbo Cui and Yingjing Xu and Liqing Ruan and Luoyuan Zhang and Hanyu Liu and Jingkun Tang and Hongyuan Liu and Qining Guo and Wenhao Hu and Bingxiang He and Jie Zhou and Jie Cai and Ji Qi and Zonghao Guo and Chi Chen and Guoyang Zeng and Yuxuan Li and Ganqu Cui and Ning Ding and Xu Han and Yuan Yao and Zhiyuan Liu and Maosong Sun},
year={2025},
eprint={2509.18154},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://arxiv.org/abs/2509.18154},
}
@article{yao2024minicpm,
title={MiniCPM-V: A GPT-4V Level MLLM on Your Phone},
author={Yao, Yuan and Yu, Tianyu and Zhang, Ao and Wang, Chongyi and Cui, Junbo and Zhu, Hongji and Cai, Tianchi and Li, Haoyu and Zhao, Weilin and He, Zhihui and others},
journal={Nat Commun 16, 5509 (2025)},
year={2025}
}