Evaluation Data for Anserini and Pyserini

This repository holds various data and various tools used by Anserini and Pyserini.

Build the included evaluation tools as follows (you might get warnings, but you can ignore):

cd eval && tar xvfz trec_eval.9.0.4.tar.gz && cd trec_eval.9.0.4 && make && cd ../..
cd eval && cd ndeval && make && cd ../..

Topics

Topics for various evaluations are stored in topics/. Metadata is stored in topics.json and aliases in topics-aliases.json. In the aliases file, the keys represent canonical topics.

Qrels

Qrels for various evaluations are stored in qrels/. Metadata is stored in qrels.json and aliases in qrels-aliases.json. In the aliases file, the keys represent canonical topics.