README.md
April 15, 2026 Β· View on GitHub
LLMEval-Gaokao2024-Math: LLM Evaluation on 2024 Chinese Gaokao Mathematics
Note: For the Chinese version of this README, please refer to README_zh.md.
π News
- π [2024-06] Evaluation results released for the 2024 Gaokao Mathematics papers.
π Overview
The freshly released Chinese National College Entrance Examination (Gaokao) papers possess high originality and confidentiality, making them an excellent benchmark for evaluating large language models. The FDU-NLP LLMEval team presents a series of evaluations on the 2024 Gaokao Mathematics papers.
The leaderboard is continuously updated. Results are for reference only.
Key Features
- Zero contamination β brand-new exam questions that models cannot have seen during pre-training
- Dual prompt formats β LaTeX and escape-character versions to test prompt sensitivity
- Two exam papers β New Paper I (ζ°Iε·) and New Paper II (ζ°IIε·)
- Standardized rubrics β official Gaokao scoring criteria
π Evaluation Design
Each model is evaluated on both papers using two different prompt formats:
| Paper | Format | Description |
|---|---|---|
| New Paper I | LaTeX | Mathematical expressions in standard LaTeX |
| New Paper I | Escape | Mathematical expressions in text-based escape characters |
| New Paper II | LaTeX | Mathematical expressions in standard LaTeX |
| New Paper II | Escape | Mathematical expressions in text-based escape characters |
This dual-format design reveals how sensitive models are to prompt formatting in mathematical contexts.
π Results
New Paper I (ζ°Iε·)
Results are available as ranking images in the repository:
- LaTeX format:
ζ°Iε·/latexζ΅θ―/ - Escape format:
ζ°Iε·/转δΉη¬¦ζ΅θ―/
New Paper II (ζ°IIε·)
- LaTeX format:
ζ°IIε·/latexζ΅θ―/ - Escape format:
ζ°IIε·/转δΉη¬¦ζ΅θ―/
π Related Projects
| Project | Description | Link |
|---|---|---|
| LLMEval (AAAI 2024) | Foundational evaluation methodology paper | arXiv |
| LLMEval-Fair (ACL 2026) | Robust & fair evaluation, 200K+ questions | GitHub |
| LLMEval-Med (EMNLP 2025) | Medical LLM benchmark | GitHub |
| LLMEval-1 | Phase I: General capability evaluation | GitHub |
| LLMEval-2 | Phase II: Professional domain evaluation | GitHub |
| Official Website | All projects & leaderboard | llmeval.com |
π Citation
@misc{llmeval-gaokao2024-math,
author = {LLMEval Team},
title = {LLMEval-Gaokao2024-Math},
year = {2024},
url = {https://github.com/llmeval/Llmeval-Gaokao2024-Math}
}
π Contact Us
- Website: http://llmeval.com/
- Email: mingzhang23@m.fudan.edu.cn
- WeChat: zanyingluan
LLMEval | Fudan University NLP Lab