Data Modeling task

eval_code_interpreter.ipynb: A sample code to evaluate Code Interpreter with GPT series models on our data modeling task.

./output_model/gpt-3.5-turbo-0125: This is evaluation results of Code Interpreter with gpt-3.5-turbo-0125. It recodes the cost, predicted answer, consuming time....

If you want to evaluate AutoGen with eva_autogen_gpt.ipynb, please install all required environments.

Download all competations

Please download all competations from the Kaggle website. The details and download url of each competition can be found in data.json file. To simplify, you can use the Kaggle API for data dowloading.

Process the downloaded file with our script.

For simplicity, we also provide the processed files by google drive and huggingface.

Non-Infringement: The pre-processed data provided by us is intended solely for educational and research purposes. We do not claim ownership of the original data, and any use of this data should respect the rights of the original creators. Users are responsible for ensuring that their use of the data does not infringe on any copyrights or other intellectual property rights.

How to evaluate CodeInterpreter on data modeling tasks

1. Unzip data.zip and save_performance.zip into the current path.
2. Set your OpenAI key in eval_code_interpreter.ipynb.
3. Run eval_code_interpreter.ipynb to evaluate the performance.

4. python score4each_com.py
   

   Computer the performance of the model on each competition with the corresponding metric, such as accuracy. Save the results in the directory ./save_performance/gpt-3.5-turbo-0125.

5. python show_result.py
   

   Show the performance of the model based on the results generated from the last step.