README.md

🛝 Playground | 📄 Technical report | 💻 GitHub | 👀 Sign up for the API

Selene Mini

Selene Mini is a state-of-the-art small language model-as-a-judge (SLMJ). Selene Mini achieves comparable performance to models 10x its size, outperforming GPT-4o on RewardBench, EvalBiasBench, and AutoJ.

Post-trained from Llama-3.1-8B across a wide range of evaluation tasks and scoring criteria, Selene Mini outperforms prior small evaluation models overall across 11 benchmarks covering three different types of tasks:

• Absolute scoring, e.g. "Evaluate the harmlessness of this response on a scale of 1-5"
• Classification, e.g. "Does this response address the user query? Answer Yes or No."
• Pairwise preference. e.g. "Which of the following responses is more logically consistent - A or B?"

It is also the #1 8B generative model on RewardBench.

Resources

This repo features prompt templates used during training and hands-on examples for using Selene Mini.

Contact

Get in touch if you have any queries not covered in this repo.