Jintao Tong¹, Jiaqi Gu², Yujing Lou², Lubin Fan^2✉, Yue Wu²,
Jieping Ye², Ruixuan Li^1✉, Yixiong Zou<sup>1✉

1</sup>Huazhong University of Science and Technology
²Alibaba Cloud Computing

🔥 News

• 2026.02.05 🤗 The checkpoints of SkiLa 7B is released!
• 2026.02.03 🚀 Code is released ！
• 2025.12.16 📝 We release our latest work Sketch-in-Latents (SkiLa), a novel unified reasoning MLLMs to flexibly and seamlessly interleave multi-step explicit textual thoughts and latent visual thoughts.

💡 Highlights

TLDR: We propose SkiLa (Sketch-in-Latents), a unified multimodal reasoning paradigm that enables MLLMs to autoregressively generate continuous visual embeddings as visual thoughts alongside text tokens. The model alternates between textual thinking and visual sketching during multi step reasoning, and uses a semantic reconstruction mechanism to keep the latent sketches grounded.

🛠 Preparation

1. Code

git clone https://github.com/TungChintao/SkiLa.git
cd SkiLa

pip install -r requirements.txt
pip install qwen-vl-utils
pip install flash-attn --no-build-isolation

2. Training Data

Download Datasets of Zebra-CoT

🎯 Training

To run the training script, use the following command:

bash scripts/train_skila.sh

📖 Evaluation

We adopt VLMEvalKit to conduct the evaluation. You can get started as follows:

1. Install

cd VLMEvalKit
pip install -e.

2. Inference

bash test.sh

See here [QuickStar | 快速开始] for more details about arguments.

🔑 License

• This project is released under the Apache 2.0 license.

📌 Citation

If you find this project useful in your research, please consider citing:

@article{tong2025sketch,
  title={Sketch-in-latents: Eliciting unified reasoning in mllms},
  author={Tong, Jintao and Gu, Jiaqi and Lou, Yujing and Fan, Lubin and Zou, Yixiong and Wu, Yue and Ye, Jieping and Li, Ruixuan},
  journal={arXiv preprint arXiv:2512.16584},
  year={2025}
}

👍 Acknowledgment

• We sincerely thank Qwen-VL-Series-Finetune, LVR, Zebra-CoT and others for their contributions, which have provided valuable insights.