🧭 Contents

• Introduction
• Main Results
• Getting Started
  • Install
  • Data
  • Train
• Acknowledgments
• License
• Citation

💡 Introduction

Self-play agent training must balance reliable supervision with evolving task coverage. Fixed tasks are verifiable but cannot follow a solver's changing capabilities, while unconstrained generation can introduce invalid or uninformative samples. Skill Self-Play (Skill-SP) closes this loop through skill-routed task generation, automatic validity verification, frontier-based curriculum construction, and feedback-driven skill evolution. It supports tool-call prediction and logical reasoning; skills are used only during training, while the final solver remains prompt-only at inference.

• Evolving skill library: modular skill packages provide task-generation guidance, and a router selects skills for each proposal.
• Verified task generation: the proposer generates tasks conditioned on the selected skill. Validity checks enforce task contracts, while the current solver empirically estimates each task's difficulty and utility.
• Frontier-based co-evolution: tasks near the solver's learning frontier form the training pool. The solver learns from this pool, while the proposer and solver are optimized for their respective objectives.
• Targeted skill evolution: the controller aggregates check failures, novel samples, and success and utility signals to trigger skill refinement, pruning, and induction; the resulting packages update the skill library.

📊 Main Results

Results reported in the paper. Tool-call overall is the combined API-Bank/BFCL average; logical overall is ZebraLogic grid-level accuracy.

Backbone	Tool-call	Gain	Logical	Gain
Qwen3-4B-2507	60.2 -> 66.7	+6.5	72.1 -> 73.5	+1.4
Qwen3-8B	69.4 -> 72.2	+2.8	23.6 -> 32.4	+8.8
Ministral-3-8B	20.7 -> 63.6	+42.9	5.0 -> 11.2	+6.2
Ministral-3-14B	22.2 -> 64.5	+42.3	5.4 -> 17.4	+12.0
Granite-4.1-3B	57.2 -> 62.5	+5.3	11.6 -> 12.6	+1.0

🚀 Getting Started

📦 Install

Python 3.11 is recommended.

python -m pip install -U pip
python -m pip install -r requirements.txt

Install CUDA-compatible PyTorch, vLLM, and FlashAttention builds first when your environment requires custom wheels. Set HF_TOKEN when gated Hugging Face resources require authentication. For experiment logging, set WANDB_API_KEY or use WANDB_MODE=offline.

🗂️ Data

API-Bank, BFCL, and ZebraLogic evaluation data are bundled under benchmarks/, so the provided launchers run without additional dataset setup.

🏋️ Train

Set the model and output roots, then run one launcher with bash <script>. The released launchers assume 8 visible GPUs: the questioner uses GPUs 0-3, and the solver uses all 8 GPUs. Each launcher evaluates the base model and every solver checkpoint automatically.

export SKILL_SP_OUTPUT_DIR=/path/to/output
export SKILL_SP_MODEL_ROOT=/path/to/model/checkpoints
bash train-tool-call-skill-qwen3-4b.sh

Backbone	Tool-call prediction	Logical reasoning
Qwen3-4B-Instruct-2507	train-tool-call-skill-qwen3-4b.sh	train-logical-skill-qwen3-4b.sh
Qwen3-8B	train-tool-call-skill-qwen3-8b.sh	train-logical-skill-qwen3-8b.sh
Ministral-3-8B-Instruct-2512-BF16	train-tool-call-skill-ministral3-8b-bf16.sh	train-logical-skill-ministral3-8b-bf16.sh
Ministral-3-14B-Instruct-2512-BF16	train-tool-call-skill-ministral3-14b-bf16.sh	train-logical-skill-ministral3-14b-bf16.sh
granite-4.1-3b	train-tool-call-skill-granite-4.1-3b.sh	train-logical-skill-granite-4.1-3b.sh

🙏 Acknowledgments

We thank the contributors to VERL and R-Zero for their helpful open-source contributions to this work.

⚖️ License

Apache License 2.0. See LICENSE.

📚 Citation

If you find Skill-SP useful, please consider citing:

@article{huang2026skill,
  title={Skill Self-Play: Pushing the Frontier of LLM Capability with Co-Evolving Skills},
  author={Huang, Siyuan and Cheng, Pengyu and Liu, Haotian and Chen, Tao and Liu, Yihao and Ni, Jingwei and Zhou, Shijie and Yang, Ziyi and Jiang, Gangwei and Zhou, Mengyu and Cheng, Yu and Jiang, Xiaoxi and Jiang, Guanjun},
  journal={arXiv preprint arXiv:2607.22529},
  year={2026},
  url={https://arxiv.org/abs/2607.22529}
}