Squant
July 3, 2025 ยท View on GitHub
Official repo for the paper: Squat: Quant Small Language Models on the Edge
Implementation
Follow the instructions of the BabyLLaMA to implement the training environment, and BabyLM Challenge to implement the evaluation environment.
Usage
- Download dataset from BabyLM Challenge
- Clean the dataset according to BabyLLaMA
- Pretrain teacher model
- Download FP16 LLaMA-58M model from BabyLLaMA
- QAT with scripts in
distill_train/scripts/ - Evaluation with scripts in
evaluation_pipeline/
Citation
@article{shen2025squant,
title={Squat: Quant Small Language Models on the Edge},
author={Shen, Xuan and Peiyan, Dong and Kong, Zhenglun and Gong, Yifan and Yang, Changdi and Han, Zhaoyang and Xie, Yanyue and Lu, Lei and others},
journal={arXiv preprint arXiv:2402.10787},
year={2025}
}