RAG Evaluations in LlamaIndex

March 19, 2024 ยท View on GitHub

Multiple Domains Scenarios in ["zh"]

Embedding ModelsWithoutReranker
[hit_rate/mrr]
CohereRerank
[hit_rate/mrr]
bge-reranker-large
[hit_rate/mrr]
bge-reranker-v2-m3
[hit_rate/mrr]
bce-reranker-base_v1
[hit_rate/mrr]
OpenAI-ada-277.35/56.1985.36/68.1386.19/69.5986.74/70.9488.67/75.26
OpenAI-embed-3-small84.53/62.5190.06/70.7089.78/71.8391.16/73.7292.54/78.36
OpenAI-embed-3-large84.25/60.1088.12/69.9789.23/71.2191.16/73.2692.27/77.28
bge-large-zh-v1.584.81/61.9089.50/69.8189.50/71.0790.61/73.1192.82/77.64
bge-m3-large87.57/65.0191.16/71.3991.44/72.0393.09/74.0294.75/79.46
CohereV3-multilingual82.87/63.2286.19/68.1485.64/68.2686.74/70.2588.40/74.42
JinaAI-v2-Base-zh78.45/56.8084.81/67.3384.81/68.1485.64/69.9288.12/75.09
gte-large-zh77.35/55.3385.36/67.0685.08/68.8385.91/70.0887.85/74.79
e5-large-multilingual87.02/65.2689.78/70.7390.33/71.5190.88/73.7492.82/78.69
bce-embedding-base_v183.70/62.9092.27/71.9492.27/72.7992.54/75.1195.03/79.57