Usage

• To perform inference for RAF and the baseline, install the necessary packages by running:

  pip install -r requirements.txt
  

Running Chronos: Baseline vs. Naive RAF Performance Comparison

This codebase is built on top of the main Chronos Forecasting GitHub repository. For more details and updates, please refer to the official repo.

• For the baseline approach, run run_chronos.py:

  python run_chronos.py path/to/config/file.yaml path/to/result/file.csv --no-augment
  

• For RAF, run run_chronos.py:

  python run_chronos.py path/to/config/file.yaml path/to/result/file.csv --augment
  

Fine-tuning Chronos Models for Advanced RAF and Baseline

• The same approach is followed for fine-tuning in the Chronos repository. The configuration files required to fine-tune the models are provided under configs/fine-tune for both the baseline approach and RAF. The same scripts are run for both approaches, using different configuration files.

• The list of arrow files obtained from Benchmark I is also provided. The datasets in RAF_finetune_datasets are augmented via time series retrieval, whereas the datasets in Baseline_finetune_datasets are not. However, these datasets can be replicated or new ones can be generated by running generate_fine_tune_data.py as follows:

  # For augmented datasets via time series retrieval
  python generate_fine_tune_data.py path/to/config/file.yaml  --augment
  
  # For regular datasets with no retrieval
  python generate_fine_tune_data.py path/to/config/file.yaml  --no-augment
  

• Start the fine-tuning for Chronos Base:

  # Fine-tune `amazon/chronos-t5-base` for 1000 steps with initial learning rate of 1e-5
  python chronos_training/train.py --config /path/to/modified/config.yaml \
      --model-id amazon/chronos-t5-base \
      --no-random-init \
      --max-steps 1000 \
      --learning-rate 0.00001
  

• Start the fine-tuning for Chronos Mini:

  # Fine-tune `amazon/chronos-t5-tiny` for 400 steps with initial learning rate of 1e-5
  python chronos_training/train.py --config /path/to/modified/config.yaml \
      --model-id amazon/chronos-t5-tiny \
      --no-random-init \
      --max-steps 400 \
      --learning-rate 0.00001
  

The output and checkpoints will be saved in RAF_models/run-{id}/ for fine-tuning on the RAF_finetune_datasets, and in Baseline_models/run-{id}/ for fine-tuning on the Baseline_finetune_datasets.

Plotting the Results for Baseline and Naive RAF

• In order to visualize the different forecasts obtained from Naive RAF and the baseline, run the following script:

  python plot_chronos_results.py path/to/config/file.yaml
  

This will generate plots for the given dataset using the provided context and prediction lengths specified in the configuration file.

Calculating the Relative Improvements

• Run the following script to compute the aggregate relative WQL and MASE values obtained from Naive RAF and the baseline scores.

  python compute_scores.py path/to/baseline/score/file.csv path/to/RAF/score/file.csv