Documentation :open_book:

• Documentation :open_book:
  • Setup
  • Configuration Parameters
    • Trainer
    • Model
    • Data
  • Datasets
    • Datasets Overview
      • Short-Term Setting
      • Long-Term Setting
    • Data Processing Pipeline
    • Using Build-in Datasets
    • Using Customized Dataset
  • Model
    • Available Models
    • Using Customized Model
  • Training
    • Configuring Optimizers and Learning Rate Schedulers
  • Forecasting with Varied Prediction Lengths
    • Example 1: Varied-Horizon Training
    • Example 2: Validation and Testing with Multiple Horizons

Setup

ProbTS is developed with Python 3.10 and relies on PyTorch Lightning. To set up the environment:

# Create a new conda environment
conda create -n probts python=3.10
conda activate probts

# Install required packages
pip install .
pip uninstall -y probts # recommended to uninstall the root package (optional)

[Optional] For time-series foundation models, you need to install basic packages and additional dependencies:

# Create a new conda environment
conda create -n probts_fm python=3.10
conda activate probts_fm

# Git submodule
git submodule update --init --recursive

# Install additional packages for foundation models
pip install ".[tsfm]"
pip uninstall -y probts # recommended to uninstall the root package (optional)

# For MOIRAI, we fix the version of the package for better performance
cd submodules/uni2ts
git reset --hard fce6a6f57bc3bc1a57c7feb3abc6c7eb2f264301

# For TimesFM, fix the version for reproducibility (optional)
cd submodules/timesfm
git reset --hard 5c7b905

# For Lag-Llama, fix the version for reproducibility (optional)
cd submodules/lag_llama
git reset --hard 4ad82d9

# For TinyTimeMixer, fix the version for reproducibility (optional)
cd submodules/tsfm
git reset --hard bb125c14a05e4231636d6b64f8951d5fe96da1dc

Configuration Parameters

• To print the full pipeline configuration to a file:

  python run.py --print_config > config/pipeline_config.yaml
  

Trainer

Model

Data

Temporal Features

For the datasets used for long-term forecasting scenario, we support three types of time feature encoding

--data.data_manager.init_args.timeenc {the encoding type} # select from [0,1,2]

• [timeenc 0] temporal information

  The dimension of time feature is 5, containing month, day, weekday, hour, minute.

• [timeenc 1] time feature based on frequency Extract time feature using time_features_from_frequency_str() function. The dimensionality follows:

  freq_map = {'h': 4, 't': 5, 's': 6, 'm': 1, 'a': 1, 'w': 2, 'd': 3, 'b': 3}
  

  Note: timeenc = 0 if model.embed != 'timeF' else 1.

• [timeenc 2] Raw date information

  The dimension of time feature is 5, using the following code to recover it to date data type:

  data_stamp = batch_data.past_time_feat.cpu().numpy().astype('datetime64[s]')
  data_stamp = batch_data.future_time_feat.cpu().numpy().astype('datetime64[s]')
  

Datasets

Datasets Overview

Short-Term Setting

Long-Term Setting

Data Processing Pipeline

Using Build-in Datasets

• Short-Term Forecasting: We use datasets from GluonTS. Configure the datasets using --data.data_manager.init_args.dataset {DATASET_NAME} with available DATASET_NAME in short-term setting.

• Long-Term Forecasting: To download the long-term forecasting datasets, please follow these steps:

  bash scripts/prepare_datasets.sh "./datasets"
  

  Configure the datasets using --data.data_manager.init_args.dataset {DATASET_NAME} with available DATASET_NAME in long-term setting.

  Note: When utilizing long-term forecasting datasets, you must explicitly specify the context_length and prediction_length parameters. For example, to set a context length of 96 and a prediction length of 192, use the following command-line arguments:

  --data.data_manager.init_args.context_length 96 \
  --data.data_manager.init_args.prediction_length 192 \
  

• Using Datasets from Monash Time Series Forecasting Repository: To use datasets from the Monash Time Series Forecasting Repository, follow these steps:

  1. Download the Dataset:
  • Navigate to the target dataset, such as the Electricity Hourly Dataset.
  • Download the .tsf file and place it in your local datasets directory (e.g., ./datasets).
  1. Configure the Dataset:

  • Use the following configuration to specify the dataset, file path, and frequency:

    --data.data_manager.init_args.dataset {DATASET_NAME} \
    --data.data_manager.init_args.data_path /path/to/data.csv \
    --data.data_manager.init_args.freq {FREQ} 
    

  • Example Configuration:

    --data.data_manager.init_args.dataset monash_electricity_hourly \
    --data.data_manager.init_args.data_path ./datasets/electricity_hourly_dataset.tsf \
    --data.data_manager.init_args.freq H \
    --data.data_manager.init_args.context_length 96 \
    --data.data_manager.init_args.prediction_length 96 \
    --data.data_manager.init_args.multivariate true
    

  Note: Refer to the Pandas Time Series Offset Aliases for the correct frequency values ({FREQ}) to use in your configuration.

• Using Datasets from GIFT-EVAL Benchmarking: see this page for detailed instructions.

Using Customized Dataset

1. Prepare the Data:

• Format your dataset as a .csv file with the following structure:

  date	VAR1	VAR2	...
  2013-01-01 00:00:00	2611.0	1539.0	...
  2013-01-01 01:00:00	2132.0	1535.0	...

  Note1: The date column represents timestamps.

  Note2: VAR1, VAR2, etc., represent different variables (features) for each timestamp.

• Place the csv file in your local datasets directory (e.g., ./datasets).

1. Configure the Dataset:

• Use the following configuration to specify the dataset, file path, and frequency:

  --data.data_manager.init_args.dataset {DATASET_NAME} \
  --data.data_manager.init_args.data_path /path/to/data_file.tsf \
  --data.data_manager.init_args.freq {FREQ} 
  

• Example Configuration:

  --data.data_manager.init_args.dataset my_data \
  --data.data_manager.init_args.data_path ./datasets/my_data.csv \
  --data.data_manager.init_args.freq H \
  --data.data_manager.init_args.context_length 96 \
  --data.data_manager.init_args.prediction_length 96 \
  --data.data_manager.init_args.multivariate true
  

Note: You can adjust the test instance sampling using the --data.data_manager.init_args.test_rolling_length parameter.

Model

Available Models

ProbTS includes both classical time-series models, specializing in long-term point forecasting or short-term distributional forecasting, and recent time-series foundation models that offer zero-shot and arbitrary-horizon forecasting capabilities for new time series.

Classical Time-series Models

Fundation Models

See the tsfm configuration directory for more details. More models will be added soon—stay tuned!

Using Customized Model

With our platform, you can easily evaluate customized models across various datasets. Follow the steps below to create and evaluate your model.

Step 1: Create a New Python File

Create a new Python file and follow the structure below to define your custom model:

from probts.model.forecaster import Forecaster

class ModelName(Forecaster):
    def __init__(
        self,
        **kwargs
    ):
        """
        Initialize the model with parameters.
        """
        super().__init__(**kwargs)
        # Initialize model parameters here

    def forward(self, inputs):
        """
        Forward pass for the model.

        Parameters:
        inputs [Tensor]: Input tensor for the model.

        Returns:
        Tensor: Output tensor.
        """
        # Perform the forward pass of the model
        return outputs

    def loss(self, batch_data):
        """
        Compute the loss for the given batch data.

        Parameters:
        batch_data [dict]: Dictionary containing input data and possibly target data.

        Returns:
        Tensor: Computed loss.
        """
        # Extract inputs and targets from batch_data
        inputs = batch_data.past_target_cdf[:, -self.context_length:, :] # [batch_size, context_length, var_num]
        target = batch_data.future_target_cdf # [batch_size, prediction_length, var_num]

        # Forward pass
        outputs = self.forward(inputs)
        
        # Calculate loss using a loss function, e.g., Mean Squared Error
        loss = self.loss_function(outputs, future_target_cdf)

        return loss

    def forecast(self, batch_data, num_samples=None):
        """
        Generate forecasts for the given batch data.

        Parameters:
        batch_data [dict]: Dictionary containing input data.
        num_samples [int, optional]: Number of samples per distribution during evaluation. Defaults to None.

        Returns:
        Tensor: Forecasted outputs.
        """
        # Perform the forward pass to get the outputs
        outputs = self(batch_data.past_target_cdf[:, -self.context_length:, :])

        if num_samples is not None:
            # If num_samples is specified, use it to sample from the distribution
            outputs = self.sample_from_distribution(outputs, num_samples)
        else: 
            # If perform point estimation, the num_samples is equal to 1
            outputs = outputs.unsqueeze(1)
        return outputs # [batch_size, num_samples, prediction_length, var_num]

Input Data Format

The batch_data dictionary contains several fields that provide necessary information for the model's operation. Each field is described below:

• target_dimension_indicator:

  • Shape: [var_num]
  • Description: Indicator that specifies which dimension or feature of the target is being referenced.

• {past|future}_time_feat:

  • Shape: [batch_size,length,time_feature_dim]
  • Description: Time features associated with each time step in the past or future. This can include various time-related information such as timestamps, seasonal indicators (e.g., month, day of the week), or other temporal features that provide context to the observations.

• {past|future}_target_cdf:

  • Shape: [batch_size,length,var_num]
  • Description: The observation values of the target variable(s) for past or future time steps.

• {past|future}_observed_values:

  • Shape: [batch_size,length,var_num]
  • Description: Binary masks indicating which values in the past or future target data are observed (1) and which are missing or unobserved (0).

Step 2: Create YAML Configuration File

Create a YAML configuration file (model.yaml) for the customized model:

seed_everything: 1 # random seed
trainer:
  accelerator: gpu
  devices: 1
  strategy: auto
  max_epochs: 50
  use_distributed_sampler: false
  limit_train_batches: 100
  log_every_n_steps: 1
  default_root_dir: ./results # path to the log folder
model:
  forecaster:
    class_path: class.path.to.ModelName
    init_args:
      # init your hyperparameter here
  learning_rate: 0.001 # learning rate
data:
  data_manager:
    class_path: probts.data.data_manager.DataManager
    init_args:
      dataset: solar_nips # dataset name
      split_val: true
      scaler: standard # identity, standard, temporal
  batch_size: 32
  test_batch_size: 32
  num_workers: 8

Step 3: Run the Customized Model

Run the customized model using the configuration file:

python run.py --config config/path/to/model.yaml

Training

Configuring Optimizers and Learning Rate Schedulers

ProbTS supports customizable optimizers and learning rate schedulers. You can specify them directly in the YAML configuration file.

Example Configuration

model:
  forecaster:
    class_path: probts.model.forecaster.point_forecaster.PatchTST
    init_args:
      # Add forecaster-specific parameters here

  optimizer_config:
    class_name: torch.optim.Adam
    init_args:
      weight_decay: 0  # Add optimizer-specific parameters here

  lr_scheduler_config:
    class_name: torch.optim.lr_scheduler.OneCycleLR
    init_args:
      max_lr: 0.0001
      steps_per_epoch: 100
      pct_start: 0.3
      epochs: 50  # Add scheduler-specific parameters here

Example configurations can be found in config/default/patchtst.yaml.

Notes

• If no configuration is provided, ProbTS defaults to the Adam optimizer with a constant learning rate.
• Adjust init_args for both the optimizer and scheduler to suit your specific use case.

Forecasting with Varied Prediction Lengths

Example:

python run.py --config config/multi_hor/elastst.yaml \
                --data.data_manager.init_args.path ./datasets \
                --trainer.default_root_dir /path/to/log_dir/ \
                --data.data_manager.init_args.dataset {DATASET_NAME} \
                --data.data_manager.init_args.context_length ${TEST_CTX_LEN} \
                --data.data_manager.init_args.prediction_length ${TEST_PRED_LEN} \
                --data.data_manager.init_args.train_ctx_len ${TRAIN_CTX_LEN} \
                --data.data_manager.init_args.train_pred_len_list ${TRAIN_PRED_LEN} \
                --data.data_manager.init_args.val_ctx_len ${VAL_CTX_LEN} \
                --data.data_manager.init_args.val_pred_len_list ${VAL_PRED_LEN} 

• DATASET_NAME: Select from datasets used in long-term forecasting scenerios.
• TEST_CTX_LEN: Context length in the testing phase.
• VAL_CTX_LEN (Default: TEST_CTX_LEN): Context length in the validation phase.
• TRAIN_CTX_LEN (Default: TEST_CTX_LEN): Context length in the training phase.
• TEST_PRED_LEN: Forecasting horizons in the testing phase.
• VAL_PRED_LEN (Default: TEST_PRED_LEN): Forecasting horizons for performance validation.
• TRAIN_PRED_LEN (Default: TEST_PRED_LEN): Forecasting horizons in the training phase.

The results across multiple horizons will be saved to:

/path/to/log_dir/{DATASET_NAME}_{MODEL}_{seed}_TrainCTX_{TRAIN_CTX_LEN}_TrainPRED_{TRAIN_PRED_LEN}_ValCTX_{CTX_LEN}_ValPRED_{VAL_PRED_LEN}/horizons_results.csv

Example 1: Varied-Horizon Training

Mode 1: Random sampling from a set of horizons

python run.py --config config/multi_hor/elastst.yaml \
                --data.data_manager.init_args.path ./datasets \
                --trainer.default_root_dir /path/to/log_dir/ \
                --data.data_manager.init_args.dataset ${DATASET} \
                --data.data_manager.init_args.context_length 96 \
                --data.data_manager.init_args.prediction_length 720 \
                --data.data_manager.init_args.train_ctx_len 96 \
                --data.data_manager.init_args.val_pred_len_list 720 \
                # random selection from {96, 192, 336, 720}
                --data.data_manager.init_args.train_pred_len_list 96-192-336-720 \
                --data.data_manager.init_args.continuous_sample false 

Mode 2: Random sampling from a horizon range

python run.py --config config/multi_hor/elastst.yaml \
                --data.data_manager.init_args.path ./datasets \
                --trainer.default_root_dir /path/to/log_dir/ \
                --data.data_manager.init_args.dataset ${DATASET} \
                --data.data_manager.init_args.context_length 96 \
                --data.data_manager.init_args.prediction_length 720 \
                --data.data_manager.init_args.train_ctx_len 96 \
                --data.data_manager.init_args.val_pred_len_list 720 \
                # random sampling from [1, 720]
                --data.data_manager.init_args.train_pred_len_list 1-720 \ 
                --data.data_manager.init_args.continuous_sample true 

Example 2: Validation and Testing with Multiple Horizons

python run.py --config config/multi_hor/elastst.yaml \
                --data.data_manager.init_args.path ./datasets \
                --trainer.default_root_dir /path/to/log_dir/ \
                --data.data_manager.init_args.dataset ${DATASET} \
                --data.data_manager.init_args.context_length 96 \
                --data.data_manager.init_args.train_pred_len_list 720 \ 
                --data.data_manager.init_args.train_ctx_len 96 \
                # validation on {96, 192, 336, 720}
                --data.data_manager.init_args.val_pred_len_list 96-192-336-720 \
                # testing on {24, 96, 192, 336, 720, 1024}
                --data.data_manager.init_args.prediction_length 24-96-192-336-720-1024