This repository corresponds to the paper:

On Predictability of Reinforcement Learning Dynamics for Large Language Models
https://arxiv.org/pdf/2510.00553

---

Installation

# clone codebase
git clone https://github.com/caiyuchen-ustc/Alpha-RL.git && cd Alpha-RL

# prepare environment
conda create -y -n AlphaRL python=3.11
conda activate AlphaRL
cd Alpha-RL

# install dependencies
pip install -r requirements.txt

Download_Model

You can access the checkpoint at the following link: Hugging Face - caiyuchen

# run
cd eval
sh download_hf.sh

Model Evaluation

sh reasoning_eval.sh

Singular Value Decomposition

sh svd.sh # Obtain the SVD decomposition of each matrix in a model

Obtain a Rank-k Model

sh upd_rank.sh 

t-SNE Visualization of Training Trajectories

cd analysis #eval/analysis
sh extract_rank1_u.sh #Extract U[:,0]
sh visualize_rank1_u_tsne.sh

PLS (Partial Least Squares) Trajectory Fitting

sh AlphaPLS.sh

AlphaRL Predict

sh AlphaPredVector.sh
sh AlphaRLBuildPredictModel.sh

This repository provides an evaluation framework inspired by LIMO, which can be found here.

If you find this project interesting, feel free to ⭐ star the repository or open an issue for discussion!

If you use this code in your research, please cite:

@article{cai2025predictability,
  title={On Predictability of Reinforcement Learning Dynamics for Large Language Models},
  author={Cai, Yuchen and Cao, Ding and Xu, Xin and Yao, Zijun and Huang, Yuqing and Tan, Zhenyu and Zhang, Benyi and Liu, Guiquan and Fang, Junfeng},
  journal={arXiv preprint arXiv:2510.00553},
  year={2025}
}