This repository provides model weights, TSUMM-Suite data utilities, training and inference scripts, and evaluation tools for TimeOmni-VL.

TimeOmni-VL is a vision-centric time-series multimodal model. It represents time series as structured images and supports both time-series understanding and time-series generation in a unified framework.

🎬 Demo Video

▶️ Click the teaser to watch the full demo video. It walks through TimeOmni-VL across all of its abilities: TS-image-based understanding (variable counting, Y-range, cycle localization, mean comparison, anomaly detection, trend analysis), generation (forecasting and imputation by editing the TS-image), and text-only reasoning.

---

🚩 Updates/News

🚩 News (Jun. 2026): We have added a demo video showing how TimeOmni-VL works.

🚩 News (May 2026): We release the TimeOmni-VL checkpoint and TSUMM-Suite generation training samples on Hugging Face: TimeOmni-VL and TSUMM-Suite_Training.

🚩 News (May 2026): TimeOmni-VL has been accepted to ICML 2026.

🔎 Overview

TimeOmni-VL contains two main components:

1. TSUMM-Suite: A data pipeline covering time series understanding and generation tasks.
2. TimeOmni-VL: A unified vision-language generation model trained on time series images, text instructions, and reasoning data.

🛠️ Environment Setup

# Create a new conda environment
conda create -n timeomni_vl python=3.10
conda activate timeomni_vl

# Install TimeOmni-VL dependencies.
pip install -r training/requirements.txt

FlashAttention installation:

# Download the FlashAttention wheel
wget https://github.com/Dao-AILab/flash-attention/releases/download/v2.5.8/flash_attn-2.5.8+cu122torch2.3cxx11abiFALSE-cp310-cp310-linux_x86_64.whl

# Install FlashAttention
pip install flash_attn-2.5.8+cu122torch2.3cxx11abiFALSE-cp310-cp310-linux_x86_64.whl

🧱 TSUMM-Suite

TSUMM-Suite includes understanding, generation, and reasoning data.

Understanding tasks

• Variable counting
• Variable Y-Range identification
• Cycle bounding box localization
• Mean comparison
• Anomaly detection
• Trend analysis

Generation tasks

• Multivariate time series forecasting
• Multivariate time series imputation

Reasoning tasks

• Text-only time series reasoning samples that complement the vision-centric tasks.

🧪 Training Data

The TSUMM-Suite generation training samples (80k forecasting + imputation samples) are available at TSUMM-Suite_Training. In the paper, the main experiments use 5k samples per generation task for training; the remaining samples are released for community exploration.

🧬 Evaluation Data

Before generating evaluation samples, download the GIFT-Eval data into data_pipeline/GiftEval, then install the GIFT-Eval requirements:

git clone https://github.com/SalesforceAIResearch/gift-eval.git && cd gift-eval && pip install -e .

The following commands demonstrate how to generate 10 evaluation samples.

Generate forecasting evaluation samples:

python data_pipeline/gen_test_data/gen_gifteval_forecasting_test.py \
  --output-root data_pipeline/forecast_benchmark_samples \
  --term short medium long \
  --max-total-samples 10

Generate imputation evaluation samples:

python data_pipeline/gen_test_data/gen_gifteval_imputation_test.py \
  --output-root data_pipeline/imputation_benchmark_samples \
  --term short medium long \
  --max-total-samples 10

🤖 TimeOmni-VL

TimeOmni-VL is a unified vision-language generation model trained on time series images, text instructions, and reasoning data. It supports time series forecasting, imputation, visual understanding, and text reasoning through a shared multimodal interface.

📦 Model Download

Create a local checkpoint folder and place the downloaded model under it:

mkdir -p checkpoint

Download the TimeOmni-VL checkpoint from TimeOmni-VL.

🚀 Inference

Demo-level samples are already included in:

data_pipeline/demo_level_samples/

📈 Forecasting

CUDA_VISIBLE_DEVICES=0 python eval/generation_inference.py \
  --base_model checkpoint/TimeOmni-VL \
  --jsonl data_pipeline/demo_level_samples/forecast_samples_thinking_gen.jsonl \
  --input-root data_pipeline/demo_level_samples \
  --output-root eval/outputs/forecasting_demo \
  --output-name edit.png \
  --metrics-csv eval/outputs/forecasting_demo/metrics.csv \
  --device-ids 0 \
  --no-shuffle

🧩 Imputation

CUDA_VISIBLE_DEVICES=0 python eval/generation_inference.py \
  --base_model checkpoint/TimeOmni-VL \
  --jsonl data_pipeline/demo_level_samples/imputation_samples_thinking_gen.jsonl \
  --input-root data_pipeline/demo_level_samples \
  --output-root eval/outputs/imputation_demo \
  --output-name edit.png \
  --metrics-csv eval/outputs/imputation_demo/metrics.csv \
  --device-ids 0 \
  --no-shuffle

👁️ Understanding

CUDA_VISIBLE_DEVICES=0 python eval/understanding_inference.py \
  --base_model checkpoint/TimeOmni-VL \
  --image data_pipeline/demo_level_samples/sample_understanding/image_full.png \
  --qa-json data_pipeline/demo_level_samples/sample_understanding/qa_pairs.json \
  --qa-index 0 \
  --output-root eval/outputs/understanding_demo \
  --device-ids 0

Add --think to enable explicit thinking output.

🏋️ Training

Dataset configuration:

training/data/configs/example.yaml

Training entry:

bash training/scripts/train.sh

Before launching training, update the machine-specific paths, distributed settings, dataset paths, and GPU count in the config and script.

🖼️ Task Examples

Understanding

Forecasting

Imputation

✍️ Citation

@inproceedings{
guan2026timeomnivl,
title={TimeOmni-{VL}: Unified Models for Time Series Understanding and Generation},
author={Tong Guan and Sheng Pan and Johan Barthelemy and Zhao Li and Yujun Cai and Cesare Alippi and Ming Jin and Shirui Pan},
booktitle={Forty-third International Conference on Machine Learning},
year={2026},
url={https://openreview.net/forum?id=uUnIAtZUfX}
}