Position: Current Benchmarking Hinders Real Progress in Deep Learning for Time Series Forecasting

---

Reference Architecture

The core of the reference architecture is a modular, extensible architecture where different stages (preprocessing, temporal, spatial) can be customized independently and extended to new components with minimal effort.

The diagram above shows the reference architecture used in the paper, decomposed as f(·,·) = f_post ◦ f_proc ◦ f_pre.

Key Modules

Component	Location
Core Backbone	lib/nn/models/reference_architecture.py
Temporal Processing	lib/nn/blocks/temporal
Spatial Processing	lib/nn/blocks/spatial

📁 Project Structure

position_benchmarking/
├── config/                    # Configuration files
├── lib/                       # Core library modules
│   ├── nn/
│   │   ├── models/           # Model implementations
│   │   ├── blocks/           # Reusable neural blocks
│   │   └── ...
│   └── datasets/             # Dataset preprocessing
├── tsl/                       # Time Series Learning utilities
├── experiments/              # Experimental scripts
│   ├── forecasting/          # Forecasting experiments
│   │   └── run_experiment.py
│   └── statforecast/         # Statistical forecasting
│       └── ols_experiment.py
├── conda_env.yml             # Environment specification
└── README.md

⚙️ Installation

Create a virtual environment using conda:

conda env create -f conda_env.yml
conda activate position_env

Usage

Running Experiments

Default configuration:

python experiments/forecasting/run_experiment.py

Custom configuration:

python experiments/forecasting/run_experiment.py model=patchtst dataset=weather

Statistical forecasting with OLS/Ridge:

python experiments/statforecast/ols_experiment.py

Configuration

Experiments are configured using YAML files in experiments/forecasting/config. Customize:

• Models and hyperparameters
• Datasets and preprocessing
• Training parameters (batch size, learning rate, epochs)
• And more...

Datasets

We support common forecasting benchmarks:

• Weather - Historical weather measurements
• Electricity - Power consumption data
• Traffic - Traffic speed measurements
• Solar - Solar power generation

See lib/datasets for implementation details and preprocessing scripts.

👥 Authors

Author	GitHub
Valentina Moretti	@valentina-moretti
Andrea Cini	@andreacini
Ivan Marisca	@marshka
Cesare Alippi	-

📖 Citation

If you find this code useful, please consider citing our paper:

@inproceedings{moretti2026DeepForecastingBenchmarkingIssues,
    title={Position: Current Benchmarking Hinders Real Progress in Deep Learning for Time Series Forecasting},
    author={Moretti, Valentina and Marisca, Ivan and Alippi, Cesare and Cini, Andrea},
    year={2026},
    booktitle={Forty-third International Conference on Machine Learning Position Paper Track},
    url={https://openreview.net/forum?id=gtwbLmO7Wb}
}

---