PhenoYieldNet: Learning Crop-Aware Phenological Responses for Multi-Crop Yield Prediction

Official implementation of $\texttt{PhenoYieldNet}$, a multi-modal model that predicts county-level yield for multiple crops.

Training

We train in two stages: self-supervised temporal-contrastive adaptation of the backbone (train_da.py), then supervised fine-tuning of the full model (train_ft.py).

1. Preparation of the Environment:

   git clone https://github.com/roroyo/PhenoYieldNet
   cd PhenoYieldNet
   
   conda create -n PhenoYieldNet python=3.10 -y
   conda activate PhenoYieldNet
   pip install -r requirements.txt
   

2. Preparation of the Data:

   Download the Tiny-CropNet dataset (Sentinel-2 imagery, WRF-HRRR weather and USDA county yield statistics) released with MMST-ViT / CropNet, and point --root_dir at the directory you unpack it into.

   Then download SpectralGPT+.pth from SpectralGPT — it is the Stage-1 backbone initialization. Move it into pretrained/.

3. Generation of the config files:

   A config is a JSON array with one record per (county, year, crop), holding the county's identifiers and the relative paths to its imagery, weather and yield files:

   python configs/build_config_multicrop.py \
       --county-csv configs/input/multicrop_2020_county_info.csv \
       --year 2020 \
       --out configs/multicrop/multicrop_2020_train.json
   
   python configs/build_config_multicrop.py \
       --county-csv configs/input/multicrop_2021_county_info.csv \
       --year 2021 \
       --out configs/multicrop/multicrop_2021_val.json
   

4. Stage 1 — Temporal contrast adaptation:

   python train_da.py \
       --pretrain_model ./pretrained/SpectralGPT+.pth \
       --croptype multicrop \
       --model_type tensor \
       --embed_dim 768 \
       --model vit_base_patch8_128_channel3_frame1_tpatch1 \
       --root_dir /path/to/Tiny-CropNet \
       -dft configs/multicrop/multicrop_2020_train.json \
       -dfv configs/multicrop/multicrop_2021_val.json \
       --output_dir ./output_dir/stage1_pretrain
   

   Checkpoints land in ./output_dir/stage1_pretrain/<timestamp>_multicrop/checkpoint-<epoch>.pth every --save_freq epochs.

5. Stage 2 — Fine-tuning:

   python train_ft.py \
       --pretrain_model path/to/pretrained_model.pth \
       --croptype multicrop \
       --model_type tensor \
       --embed_dim 768 \
       --model vit_base_patch8_128_channel3_frame1_tpatch1 \
       --root_dir /path/to/Tiny-CropNet \
       -dft configs/multicrop/multicrop_2020_train.json \
       -dfv configs/multicrop/multicrop_2021_val.json \
       --batch_size 16 \
       --grad_checkpointing \
       --blr 3e-4 \
       --epochs 60 \
       --output_dir ./output_dir/stage2_finetune
   

   We provide scripts/run_finetune.sh, which wraps exactly this command.

Evaluation

Point --test_model_path at one of the per-crop checkpoints written by Stage 2:

python test.py \
    --test_model_path path/to/best_model.pth \
    --croptype multicrop \
    --model_type tensor \
    --embed_dim 768 \
    --model vit_base_patch8_128_channel3_frame1_tpatch1 \
    --root_dir /path/to/Tiny-CropNet \
    -dfv configs/multicrop/multicrop_2021_val.json \
    --output_dir ./test_results

This reports per-crop RMSE, MAE, R², Pearson correlation and MAPE, and writes evaluation_results.csv under --output_dir.

Acknowledgement

Our project benefits from SpectralGPT, MMST-ViT. Per-file attribution and the corresponding licenses are listed in LICENSE.

Citation

If you find our project helpful, please cite our paper as:

@InProceedings{Luo_2026_CVPR,
    author    = {Luo, Yu and Zhu, Xiaogang and Zeng, Shan and Xiang, Wei and Bishop, Thomas Francis and Wang, Zhiyong and Hu, Kun},
    title     = {PhenoYieldNet: Learning Crop-Aware Phenological Responses for Multi-Crop Yield Prediction},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month     = {June},
    year      = {2026},
    pages     = {15870-15879}
}