Data Analysis task

eval_GPT.ipynb: A sample code to evaluate GPT series models on our data analysis task.

./save_process/gpt-3.5-turbo-0125: This is evaluation results of gpt-3.5-turbo-0125. It recodes the cost, predicted answer, consuming time.

If you want to evaluate AutoGen with eva_autogen_gpt.ipynb, please install all required environments.

How to get the data

Download all competations

Please download all competations from the ELOQUWNS website. The details and download URL of each competition can be found in data.json file.

Process the downloaded file with our script.

For simplicity, we also provide the processed files by google drive and huggingface.

Non-Infringement: The pre-processed data provided by us is intended solely for educational and research purposes. We do not claim ownership of the original data, and any use of this data should respect the rights of the original creators. Users are responsible for ensuring that their use of the data does not infringe on any copyrights or other intellectual property rights.

How to evaluate GPT on data analysis tasks

1. Unzip data.zip into the current path.

2. Set your OpenAI key in eval_GPT.ipynb.

3. Run eval_GPT.ipynb to evaluate the performance. The results will be saved into the path ./save_process/{model}/

4. Show the evaluation results:

   (1) Set your OpenAI key in computer_answer.py

   (2)

   python compute_answer.py
   

   Determine whether the answers generated by the model are correct and save the result into a JSON file results.json.

   (3)

   python show_result.py
   

   Show the performance of the model based on the results.json and files generated from the evaluation process.