Simple LLM Evaluation
June 11, 2025 · View on GitHub
Welcome to the simple LLM evaluation framework—simpleval, for short.
simpleval is a Python package designed to make evaluating Large Language Models (LLMs) easier, using the "LLM as a Judge" technique.
It supports a variety of LLM providers, including OpenAI, Google (Gemini API, Vertex), AWS Bedrock, Anthropic, Azure, and more (via LiteLLM).
simpleval also includes several reports to help you analyze, compare, and summarize your evaluation results. See the available reports for more details.
Getting Started
See the 📚 Quickstart Guide 📚
Documentation
Contributing
We appreciate your help in making this project better! ✨
If you would like to contribute to this project, please follow the guidelines outlined in the CONTRIBUTING.md file.
License
simpleval is released under the Apache License. See the LICENSE file for more details.
Contact
If you have any questions or suggestions, feel free to join our GitHub discussions forum 💬
If you want to report a bug or request a feature, please open an issue in the GitHub issues tracker 🐛