README.md

June 22, 2026 · View on GitHub

Unaligned RGBT Tracking Project (UA-RGBT Tracking)

🔗 Quick Navigation

Release

MUART244 LUART LasHeR-UA

Evaluation Results Trackers Citation

This repository contains multiple research modules related to multi-modal tracking, RGB–TIR fusion, and unaligned cross-modal UAV tracking.
Among them, our recent work:

“Progressive Multi-cue Alignment for Unaligned RGBT Tracking”
has been accepted by CVPR 2026 🎉.

“Unaligned UAV RGBT Tracking: A Largescale Benchmark and A Novel Approach”
has been accepted by AAAI 2026 🎉.

📢 Public Release

We will progressively release the following resources to support reproducible research on unaligned RGBT tracking.

🚧 Coming Soon

  • PMATrack README
  • SFCATrack Training Steps
  • New Works for UA-RGBT Tracking
  • Standard Baseline (OSTrack-/LoRAT-Style) with unaligned data augmentation for UA-RGBT Tracking

✅ Released Resources

📦 MUART244 Dataset

Multi-platform Unaligned RGBT Tracking Dataset

MUART244 is a high-quality multi-platform benchmark for unaligned RGBT tracking.
Different from existing aligned RGBT datasets, MUART244 preserves the original spatial misalignment between RGB and TIR modalities without manual pre-alignment, cropping, or rescaling.

It includes:

  • 244 RGBT video pairs
    • 143 ground-view sequences
    • 101 aerial-view sequences
  • 205K RGBT image pairs
  • Average 844 frames per video
  • 26 object categories
  • 22 challenge attributes
  • Precise dual-modal bounding-box annotations
  • Original heterogeneous resolutions:
    • RGB: from 1600×1200 to 3840×2160
    • TIR: from 640×512 to 1280×1024

MUART244 provides a realistic benchmark for evaluating unaligned RGBT tracking under large spatial offsets, scale variations, multi-platform viewpoints, and modality-specific challenges.

ResourceBaidu CloudAccess Code
MUART244 DatasetDownloadam6y
MUART244 Tracking ResultsDownload4prb

📦 LUART Dataset

Unaligned UAV RGBT Tracking Dataset

LUART is the first large-scale benchmark focusing on unaligned UAV visible–thermal tracking.
It includes:

  • 1,453 RGB–TIR sequence pairs
  • 1.02M dual-modality frames
  • 42 object categories
  • 22 challenge attributes
  • Original UAV resolutions:
    • RGB: 1920×1080
    • TIR: 640×512

ResourceBaidu CloudAccess Code
LUART DatasetDownloader4r
LUART Repair PackageDownloadgd4y
LUART Tracking ResultsDownloadpi2i

Important

Some files may be missing or corrupted after decompression. Please download the LUART Repair Package and replace the affected files.

Known Issues

  • 2026.06.16: Partial sequences may contain missing files or corrupted images.
  • 2026.06.18: Car_153–Car_164 may encounter decompression errors.

If you encounter problems with other sequences, please contact us.

📦 LasHeR-Unaligned

We also provide LasHeR-Unaligned, a derived benchmark based on
LasHeR, where spatial alignment assumptions are explicitly removed to support fair evaluation of unaligned RGBT trackers.

ResourceBaidu CloudAccess Code
LasHeR-Unaligned DatasetDownloadmmic
LasHeR-Unaligned Tracking ResultsDownloaddhjx

🧪 Evaluation Toolkit

We will release a unified evaluation toolkit for unaligned RGBT tracking based on the standard RGBT evaluation library.

The toolkit supports:

  • MUART244
  • LasHeR-Unaligned
  • LUART
  • One Pass Evaluation protocol
  • Precision Rate (PR)
  • Normalized Precision Rate (NPR)
  • Success Rate (SR)
  • Unified result format for fair comparison across different unaligned RGBT datasets

Evaluation Toolkit

  • Baidu Cloud: https://pan.baidu.com/s/1gtoEsZPTCz_CDPhuc518jg?pwd=k2hp
  • Access Code: k2hp

📊 Benchmark Results

⭐ Overall Comparison on MUART244 / LasHeR-Unaligned / LUART

TrackerPublicationMUART244 PR ↑MUART244 NPR ↑MUART244 SR ↑LasHeR-UA PR ↑LasHeR-UA NPR ↑LasHeR-UA SR ↑LUART PR ↑LUART NPR ↑LUART SR ↑
mfDiMPICCVW 2019------41.640.133.5
MANetICCVW 2019---32.926.624.1---
MaCNetSensors 2020---38.430.727.0---
CATECCV 2020---36.329.925.342.839.834.4
FANetTIV 2021---32.826.622.7---
ADRNetIJCV 2021---34.529.223.844.643.133.0
MANet++TIP 2021---30.123.920.3---
APFNetAAAI 2022---40.332.429.1---
DMCNetTNNLS 2022---35.127.725.7---
HMFTCVPR 2022------44.541.535.7
ToMPCVPR 2022---46.341.436.0---
OSTrackECCV 202245.640.433.559.253.846.7---
Baseline (Single-modal)ECCV 2022------45.441.735.6
Baseline (Multi-modal)ECCV 2022------48.645.338.3
SeqTrackv2CVPR 2023------48.345.237.5
TBSICVPR 202353.145.737.660.355.247.752.248.541.4
ViPTCVPR 202353.447.739.755.251.144.252.148.641.3
SDSTrackCVPR 202446.841.834.557.652.545.350.046.339.7
UnTrackCVPR 202454.147.939.956.551.544.753.348.841.7
BATAAAI 202444.539.732.860.555.147.749.645.939.5
GMMTAAAI 202451.044.136.258.453.345.7---
NATCISE 2024---58.152.344.8---
AFterTIP 202542.535.528.457.552.344.8---
SUTrackAAAI 202549.540.933.557.452.545.054.749.642.6
CAFormerAAAI 202546.541.934.359.053.846.752.748.841.6
AINetAAAI 202557.350.441.161.455.748.3---
STTrackAAAI 2025------53.649.642.2
SFCATrackAAAI 2026---60.755.147.957.351.944.6
PMATrackCVPR 202662.755.945.864.458.750.6---

⭐ MUART244

TrackerPublicationPR ↑NPR ↑SR ↑
OSTrackECCV 202245.640.433.5
TBSICVPR 202353.145.737.6
ViPTCVPR 202353.447.739.7
SDSTrackCVPR 202446.841.834.5
UnTrackCVPR 202454.147.939.9
BATAAAI 202444.539.732.8
GMMTAAAI 202451.044.136.2
AFterTIP 202542.535.528.4
SUTrackAAAI 202549.540.933.5
CAFormerAAAI 202546.541.934.3
AINetAAAI 202557.350.441.1
PMATrackCVPR 202662.755.945.8

⭐ LasHeR-Unaligned

TrackerPublicationPR ↑NPR ↑SR ↑FPS ↑
MANetICCVW 201932.926.624.11
MaCNetSensors 202038.430.727.00.8
CATECCV 202036.329.925.320
FANetTIV 202132.826.622.719
ADRNetIJCV 202134.529.223.825
MANet++TIP 202130.123.920.325.4
APFNetAAAI 202240.332.429.11.3
DMCNetTNNLS 202235.127.725.72.3
ToMPCVPR 202246.341.436.034
OSTrackECCV 202259.253.846.744.4
TBSICVPR 202360.355.247.736.2
ViPTCVPR 202355.251.144.224.8
SDSTrackCVPR 202457.652.545.320.9
UnTrackCVPR 202456.551.544.7-
BATAAAI 202460.555.147.7-
GMMTAAAI 202458.453.345.7-
AFterTIP 202557.552.344.823.0
SUTrackAAAI 202557.452.545.055
CAFormerAAAI 202559.053.846.786.3
AINetAAAI 202561.455.748.338.1
NATCISE 202458.152.344.819
SFCATrackAAAI 202660.755.147.9-
PMATrackCVPR 202664.458.750.628.0

⭐ LUART

TrackerPublicationPR ↑NPR ↑SR ↑
mfDiMPICCVW 201941.640.133.5
CATECCV 202042.839.834.4
ADRNetIJCV 202144.643.133.0
HMFTCVPR 202244.541.535.7
SeqTrackv2CVPR 202348.345.237.5
ViPTCVPR 202352.148.641.3
TBSICVPR 202352.248.541.4
BATAAAI 202449.645.939.5
SDSTrackCVPR 202450.046.339.7
UnTrackCVPR 202453.348.841.7
CAFormerAAAI 202552.748.841.6
STTrackAAAI 202553.649.642.2
SUTrackAAAI 202554.749.642.6
Baseline (Single-modal)ECCV 202245.441.735.6
Baseline (Multi-modal)ECCV 202248.645.338.3
SFCATrackAAAI 202657.351.944.6

💻 Open-source Trackers

We organize open-source trackers according to their alignment strategy.

1. Early-stage Alignment

Trackers that perform cross-modal alignment before or at the early feature extraction stage.

TrackerVenueCodeStatus
SFCATrackAAAI 2026Link✅ Released
Coming soon——🚧 Coming Soon

2. Middle-stage Alignment

Trackers that perform alignment during feature interaction, fusion, or representation learning.

TrackerVenueCodeStatus
PMATrackCVPR 2026Link✅ Released
Coming soon——🚧 Coming Soon

3. Post-stage Alignment

Trackers that perform alignment after prediction or use post-processing-based alignment.

TrackerVenueCodeStatus
Coming soon——🚧 Coming Soon

4. Efficient Alignment

Trackers designed for efficient unaligned RGBT tracking with reduced computation or faster inference.

TrackerVenueCodeStatus
Coming soon——🚧 Coming Soon

5. Alignment-free Tracker

TrackerVenueCodeStatus
Coming soon——🚧 Coming Soon

📚 Citation

If you find this repository or the LUART dataset useful for your research,
please consider citing our AAAI 2026 paper:

@inproceedings{jin2026progressive,
    author    = {Jin, Jiandong and Li, Chenglong and Feng, Hao and Lu, Andong and Huang, Lili and Tang, Jin},
    title     = {Progressive Multi-cue Alignment for Unaligned RGBT Tracking},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    year      = {2026},
    pages     = {35207-35216}
}

@inproceedings{xiao2026unaligned,
  title={Unaligned UAV RGBT Tracking: A Largescale Benchmark and a Novel Approach},
  author={Xiao, Yun and Wang, Yuhang and Jin, Jiandong and Zhang, Wankang and Li, Chenglong},
  booktitle={Proceedings of the AAAI Conference on Artificial Intelligence},
  volume={40},
  number={13},
  pages={11014--11022},
  year={2026}
}

@article{li2021lasher,
  title={LasHeR: A large-scale high-diversity benchmark for RGBT tracking},
  author={Li, Chenglong and Xue, Wanlin and Jia, Yaqing and Qu, Zhichen and Luo, Bin and Tang, Jin and Sun, Dengdi},
  journal={IEEE Transactions on Image Processing},
  volume={31},
  pages={392--404},
  year={2021},
  publisher={IEEE}
}