This repository presents CTBench, the first comprehensive Time Series Generation benchmark tailored for the cryptocurrency domain. Thank you for your interest!

Table of Contents

• Overview of CTBench
  • Crypto-Centric Datasets
  • Dual-Task Evaluation
  • Trading Strategy
  • Financial Metric Suite
  • TSG Model Zoo
  • Benchmarking Results
    • Predictive Utility Task
    • Statistical Arbitrage Task
  • Recommendations
• Getting Started with CTBench
  • Environment Setup and Configuration
  • Benchmark Tasks
    • Predictive Utility Task
    • Statistical Arbitrage Task
    • Evaluation
  • Running CTBench
    • Creating a Custom Model
    • Output Structure

Overview of CTBench

Crypto-Centric Datasets

CTBench presents a meticulously curated, publicly available cryptocurrency dataset collected from major global exchanges. The data undergoes a standardized preprocessing pipeline with configurable options and feature selections tailored to the unique dynamics of crypto markets.

Dual-Task Evaluation

To bridge synthetic TSG with practical financial use, CTBench introduces dual-task benchmarks assessing both statistical similarity and the functional realism and trading utility of synthetic data. These tasks probe complementary aspects of TSG models: generation quality through predictive utility and reconstruction fidelity via tradable residual signals.

Trading Strategy

CTBench is explicitly designed to be strategy-agnostic, evaluating TSG models across a spectrum of trading paradigms to ensure broad applicability. We summarize three canonical strategies widely used in crypto trading:

• S1: Cross-Sectional Momentum (CSM)
• S2: Long-Only Top-Quantile (LOTQ)
• S3: Proportional-Weighting (PW)

Financial Metric Suite

CTBench organizes eleven well-established evaluation metrics into five categories, each answering a distinct question practitioners face when considering synthetic data for production.

Error-based Evaluation

• E1: Mean Squared Error (MSE)
• E2: Mean Absolute Error (MAE)

Rank-based Evaluation

• E3: Information Coefficient (IC)
• E4: Information Ratio (IR)

Trading Performance

• E5: Compound Annual Growth Rate (CAGR)
• E6: Sharpe Ratio (SR)

Risk Assessment Metrics

• E7: Maximum Drawdown (MDD)
• E8: Value at Risk (VaR)
• E9: Expected Shortfall (ES)

Efficiency

• E10: Training Time
• E11: Inference Time

TSG Model Zoo

CTBench summarizes popular TSG methods with their backbone models and financial datasets used.

Year	Method	Backbone	Financial Datasets Used
2016	C-RNN-GAN	GAN	/
2017	RCGAN	GAN	/
2018	T-CGAN	GAN	/
2019	TimeGAN	GAN	Stocks
2019	WaveGAN	GAN	/
2020	COT-GAN	GAN	/
2020	DoppelGANger	GAN	/
2020	Quant-GAN	GAN	SPX
2020	SigCWGAN	GAN	SPX & DJI
2020	TSGAN	GAN	/
2021	RTSGAN	GAN	Stocks
2021	Sig-WGAN	GAN	SPX & DJI
2021	TimeGCI	GAN	/
2022	CEGEN	GAN	Stocks & Electric Price
2022	COSCI-GAN	GAN	/
2022	PSA-GAN	GAN	/
2022	TsT-GAN	GAN	Stocks
2022	TTS-GAN	GAN	/
2023	AEC-GAN	GAN	/
2023	TT-AAE	GAN	Stocks
2021	TimeVAE	VAE	Stocks
2023	CRVAE	VAE	/
2023	TimeVQVAE	VAE	/
2024	KoVAE	VAE	Stocks
2023	DiffTime	Diffusion	Stocks
2023	TSGM	Diffusion	Stocks
2024	Diffusion-TS	Diffusion	Stocks
2024	FIDE	Diffusion	Stocks
2024	ImagenTime	Diffusion	Stocks
2024	SDformer	Diffusion	Stocks
2025	PaD-TS	Diffusion	Stocks
2020	CTFP	Flow	/
2021	Fourier-Flow	Flow	Stocks
2024	FlowTS	Flow	Stocks
2018	Neural ODE	ODE + RNN	/
2019	ODE-RNN	ODE + RNN	/
2021	Neural SDE	ODE + GAN	Stocks
2022	GT-GAN	ODE + GAN	Stocks
2023	LS4	ODE + VAE	/
2024	TimeLDM	Diffusion + VAE	Stocks

Benchmarking Results

Predictive Utility Task

Statistical Arbitrage Task

Recommendations

CTBench reveal a four-way trade-off among TSG model families:

1. VAE-based models ensure stable reconstruction but might under-react to fast-changing regimes.
2. GAN-based approaches extract trend alpha but suffer from volatility-induced instability.
3. Diffusion models handle regime clustering and fat tails well, but degrade under low signal regimes.
4. Flow-based models prioritize likelihood but offer limited utility, while mixed-type ones are efficient but inconsistent in risk–return.

Based on these findings, CTBench distills them into scenario-based actionable recommendations for the end-users.

Scenario	Recommended TSG Models	Rationale
Trend-following / Directional Markets	COSCI-GAN, KoVAE	COSCI-GAN amplifies trend and dispersion; KoVAE offers alpha with higher drawdowns
Mean-reverting / Range-bound Regimes	TimeVAE, Fourier-Flow, Diffusion-TS	TimeVAE/Fourier-Flow provide balance; Diffusion-TS preserves rank order
Fee-sensitive / Low-turnover Settings	TimeVAE, Diffusion-TS	Smooth residuals, stable Sharpe under transaction costs
Risk Tolerance / Portfolio Design	KoVAE, LS4, TimeVAE, Diffusion-TS, FIDE	KoVAE/LS4 maximize returns with risk; TimeVAE/Diffusion-TS balance Sharpe and drawdown; FIDE is defensive
Deployment Efficiency	TimeVAE, LS4	Fast retraining and low-latency inference; diffusion models better suited for offline use

Getting Started with CTBench

Environment Setup and Configuration

We recommend using conda:

conda create -n ctbench python=3.11
conda activate ctbench
pip install -r requirements.txt

Edit the file: config/config.yaml

Key configuration sections:

api:
data:
model:
cuda:
predictive_utility: 
statistical_arbitrage:
evaluation:

Benchmark Tasks

1. Predictive Utility Task (predictive_utility.py)

Evaluate how well the generated synthetic data supports predictive alpha modeling and trading.

• Trains a time series forecasting model on features extracted from synthetic data.
• Predicts log returns and simulates trading strategies.
• Logs performance measures.
• Saves predictions and equity curves.

Key config entries:

• predictive_utility.do_predictive_utility: enable this module
• predictive_utility.strategy: choose trading strategy (long_only_20, long_short_50, etc.)
• predictive_utility.use_fee: whether to simulate transaction costs

2. Statistical Arbitrage Task (statistical_arbitrage.py)

Tests the ability to reconstruct residuals and generate tradeable mean-reverting signals.

• Uses your model to reconstruct real returns.
• Fits an OU (Ornstein-Uhlenbeck) model.
• Calculates S-scores and makes long/short decisions.
• Tracks portfolio equity over time.

Key config entries:

• statistical_arbitrage.do_statistical_arbitrage: enable this module
• statistical_arbitrage.use_fee: whether to include transaction costs

3. Evaluation (evaluate.py)

Analyze performance using standard financial metrics.

• Automatically loads curve files from trading modules.

• Supports yearly or full-period evaluation.

• Outputs metrics.

  • CAGR (Compound Annual Growth Rate)
  • Sharpe Ratio
  • Max Drawdown
  • VaR (Value at Risk)
  • ES (Expected Shortfall)
  • MSE / MAE/ IC / IR (for predictive tasks)

Config entries:

• evaluation.do_evaluation: run after strategies complete
• evaluation.evaluate_by_year: yearly vs aggregate
• evaluation.file_pattern: filter curve files

Running CTBench

To run strategies defined in your config:

python main.py

This will:

• Load config/config.yaml
• Execute predictive_utility, statistical_arbitrage, or both
• Run evaluation if enabled

Creating a Custom Model

1. Add a new file in model/, e.g. model/mymodel.py
2. Define a class mymodelModel with the required interface:

class mymodelModel(nn.Module):
    def __init__(self, window_size, num_features, device, scaler, **kwargs): ...
    def fit(self, windows, num_features): ...
    def generate(self, num_samples): ...
    def reconstruct_single_window(self, window_tensor): ...
    def compute_windowed_residuals_for_training(self, data): ...

3. Set model.name: mymodel in config/config.yaml.

Output Structure

• xgboost_predictions/: predictive regression results
• feature_importances/: saved XGBoost feature importances
• performance_logs/: TensorBoard logs (optional)

Citation

@inproceedings{
  ang2025ctbench,
  title={CTBench: Cryptocurrency Time Series Generation Benchmark},
  author={Ang, Yihao and Wang, Qiang and Huang, Qiang and Bao, Yifan and Xi, Xinyu and Tung, Anthony KH and Jin, Chen and Huang, Zhiyong},
  booktitle={The Fourteenth International Conference on Learning Representations},
  year={2026},
  url={https://openreview.net/forum?id=RzT2sombPD}
}