DRGBT-1K: A Large-scale High-quality Benchmark for Dynamic RGBT Tracking
July 15, 2026 · View on GitHub
Online Evaluation Platform: https://www.codabench.org/competitions/17422/
1. Motivation
RGBT tracking receives a surge of interest in the computer vision community, but existing RGBT benchmarks assume a single, fixed observation platform with synchronized RGB and thermal sensors. In real-world collaborative perception systems, however, the target is often observed by multiple heterogeneous platforms (e.g., UAVs and ground cameras) that carry different sensors and may hand off tracking responsibility over time. This leads to Dynamic RGBT (DRGBT) tracking, where both the available modalities and observation viewpoints change dynamically.
The existing DRGBT dataset (DRGBT603) is limited in scale and contains many synthetically constructed cross-platform sequences. To address these limitations, we present DRGBT-1K — the first large-scale, fully real-captured DRGBT tracking benchmark.
2. About DRGBT-1K Benchmark
Highlights
- Fully Real-Captured: All 1,045 sequences are captured from real-world scenarios using real cross-platform handoffs, avoiding synthetic construction.
- Large Scale: Contains 1,045 real sequences and 795K frame pairs in total (2,090 RGBT sequences considering multi-view perspectives).
- High-Quality Annotations: Contains over 799K densely annotated target bounding boxes across multi-view sequences.
- Multi-Platform Imaging Devices: Data collected using DJI Mavic 3T (UAV) and Hikvision RGBT Camera (handheld/ground), preserving genuine viewpoint discontinuities.
- Rich Scenes and Categories: Covers 24 target categories across diverse scenarios (daytime, nighttime, highways, parks, campuses, etc.).
- Real-World Challenges: Annotated with 15 challenge attributes reflecting practical DRGBT tracking difficulties.
Data Samples
Below are representative samples from DRGBT-1K, showcasing various scenes, categories, and cross-platform viewpoints (RGB and thermal pairs from both UAV and ground perspectives):
3. Dataset Statistics
Comparison with Existing Benchmarks
| Dataset | Pub. Info | Task Type | View Num. | Sequence Num. | Total Frames | Object Classes | Attr. | Dynamic Modality | Cross Platform | Real Cross-platform Seq. Num. | Synthetic Cross-platform Seq. Num. |
|---|---|---|---|---|---|---|---|---|---|---|---|
| GTOT | TIP 2017 | RGBT | 1 | 50 | 7.8K | 9 | 7 | ✕ | ✕ | 0 | 0 |
| RGBT210 | CVPR 2018 | RGBT | 1 | 210 | 104.7K | 22 | 12 | ✕ | ✕ | 0 | 0 |
| RGBT234 | TPAMI 2019 | RGBT | 1 | 234 | 116.7K | 22 | 12 | ✕ | ✕ | 0 | 0 |
| LasHeR | TIP 2021 | RGBT | 1 | 1224 | 734.8K | 32 | 19 | ✕ | ✕ | 0 | 0 |
| VTUAV | CVPR 2022 | RGBT | 1 | 500 | 1.7M | 13 | 13 | ✕ | ✕ | 0 | 0 |
| DRGBT603 | TIP 2026 | DRGBT | 2 | 603 | 1.49M | 29 | 12 | ✓ | ✓ | 203 | 400 |
| DRGBT-1K (Ours) | — | DRGBT | 2 | 1045 (2090)† | 795K | 24 | 15 | ✓ | ✓ | 1045 | 0 |
†indicates that DRGBT contains 2,090 RGBT tracking sequences in total.
Distribution Statistics
The following figure shows: (a) Category distribution across daytime and nighttime scenarios, (b) Visualization of subset and challenge attribute co-occurrence relationships, and (c) Distribution of challenge attributes with train/test splits.
The frame distribution between UAV and ground viewpoints is highly balanced, accounting for 50.7% and 49.3% respectively, which significantly reduces viewpoint bias in cross-platform evaluation. In addition, quantitative analysis shows that platform transitions induce severe target scale variations: Ground-to-UAV handoffs result in target area reduction in 64.8% of cases, while UAV-to-Ground handoffs lead to target area expansion in 62.8% of cases.
15 Challenge Attributes
DRGBT-1K annotates each sequence with 15 challenge attributes reflecting real-world tracking scenarios:
| Attr | Full Name | Description |
|---|---|---|
| HO | Heavy Occlusion | The target is severely occluded. |
| PO | Partial Occlusion | The target is slightly or partially occluded. |
| LI | Low Illumination | The sequence is captured under low-light conditions (e.g., nighttime). |
| LR | Low Resolution | The target region is blurred or has low visual resolution. |
| BC | Background Clutter | The background is complex and contains many interfering objects. |
| HI | High Illumination | Strong illumination or glare appears in the scene. |
| SA | Similar Appearance | Similar objects appear near the target, making it hard to distinguish. |
| FL | Frame Lost | Several consecutive frames are completely identical. |
| SO | Small Object | The target is very small in the image. |
| FM | Fast Motion | The target moves rapidly between adjacent frames. |
| OV | Out-of-View | The target leaves the camera field of view and later reappears. |
| SV | Scale Variation | The target undergoes significant scale changes. |
| CM | Camera Moving | The camera has obvious motion or shake. |
| TC | Thermal Crossover | Target and background have similar temperatures (low thermal contrast). |
| ARC | Aspect Ratio Change | The aspect ratio of the bounding box is outside the range [0.5, 2]. |
4. Train/Test Split
We split DRGBT-1K into training and testing subsets according to the target category distribution, ensuring balanced representation across all 24 classes:
| Split | Sequences | Usage |
|---|---|---|
| Training | 800 | For training deep DRGBT trackers |
| Testing | 245 | For benchmarking and evaluation |
| Total | 1045 | — |
The training and testing sets maintain similar distributions in terms of target categories, challenge attributes, and day/night ratios.
5. Dataset Download
- DRGBT-1K (Aligned): BaiduNetdisk（Password: g13h）
- DRGBT-1K (Unaligned): BaiduNetdisk（Password: 6xxh）
Note: Due to the physical structure of heterogeneous devices and sensor mechanisms, real-world data naturally exhibits modality offsets between RGB and TIR under different viewpoints. We provide this unaligned version (where the spatial deviation is constrained within [0.01D, 0.17D] and the initial bias is randomly sampled within [2%, 17%]D, with D being the image diagonal length) to facilitate research on spatial perturbation and cross-modality alignment.
6. Dataset File Structure
sequence_name/
├── ground_viewq/
│ ├── RGB/
│ │ ├── 000001.jpg
│ │ └── ...
│ ├── TIR/
│ │ ├── 000001.jpg
│ │ └── ...
│ └── init.txt
├── uav_viewq/
│ ├── RGB/
│ │ ├── 000001.jpg
│ │ └── ...
│ ├── TIR/
│ │ ├── 000001.jpg
│ │ └── ...
│ └── init.txt
├── challenges.txt
├── modality.txt
├── platforms.txt
├── scene_class.txt
├── target_class.txt
7. Benchmark Results
Evaluation on DRGBT-1K
We evaluate multiple categories of tracking methods on DRGBT-1K, including RGBT trackers, modality-missing RGBT (MMRGBT) trackers, and DRGBT trackers, under the One-Pass Evaluation (OPE) protocol. The primary evaluation metrics are:
- Precision Rate (PR): Percentage of frames whose center location error is within 20 pixels.
- Normalized Precision Rate (NPR): Center location error normalized by target size.
- Success Rate (SR): Bounding box overlap (IoU) evaluated using Area Under Curve (AUC).
Overall Performance on DRGBT-1K
Below are the baseline results of representative state-of-the-art trackers evaluated on the DRGBT-1K test set:
| Method | Source | PR (%) | NPR (%) | SR (%) |
|---|---|---|---|---|
| RGBT Trackers | ||||
| OSTrack | ECCV'22 | 47.39 | 41.47 | 34.46 |
| TBSI | CVPR'23 | 43.89 | 38.81 | 32.34 |
| TATrack | AAAI'24 | 47.82 | 41.17 | 34.40 |
| BAT | AAAI'24 | 45.84 | 41.08 | 33.86 |
| PURA | CVPR'24 | 48.32 | 40.73 | 33.98 |
| SDSTrack | CVPR'24 | 40.66 | 33.64 | 28.23 |
| MMLoRAT | ECCV'24 | 47.76 | 43.06 | 34.92 |
| CKD | ACM MM'24 | 45.67 | 40.97 | 33.75 |
| AINet | AAAI'25 | 44.83 | 40.42 | 33.14 |
| STTrack | AAAI'25 | 22.57 | 17.38 | 16.61 |
| CAFormer | AAAI'25 | 46.06 | 39.80 | 33.51 |
| FMTrack | TCSVT'25 | 48.64 | 41.58 | 34.91 |
| QSTNet | TIP'25 | 44.21 | 39.20 | 32.27 |
| MRTTrack | PR'25 | 43.72 | 38.76 | 32.04 |
| UATrack | IJCV'26 | 46.92 | 40.77 | 34.02 |
| GOLA | CVPR'26 | 51.18 | 46.42 | 38.01 |
| MMRGBT Trackers | ||||
| IPT | IJCV'25 | 46.52 | 39.89 | 33.46 |
| TMKD | PR'26 | 48.06 | 42.95 | 35.52 |
| SCDT | CVPR'26 | 45.57 | 36.41 | 29.91 |
| DRGBT Trackers | ||||
| CMRL | TIP'26 | 43.03 | 38.48 | 31.82 |
Attribute- and Subset-based Performance
To analyze tracker performance under various scenarios, we evaluate the trackers across 15 challenge attributes and 3 variation subsets: Modality Variation Only (MVO, 209 sequences), Platform Variation Only (PVO, 209 sequences), and Combined Variation (CV, 627 sequences).
Each cell reports PR/SR (%). The best and second-best results under each row are highlighted in red and blue font, respectively. Based on the attribute- and subset-specific evaluation, we can observe that:
- The dedicated DRGBT tracker CMRL achieves outstanding performance on the platform transition subsets (PVO and CV), demonstrating its robustness in handling viewpoint and platform switches. It also exhibits superior robustness under degradation-related attributes, achieving the best PR and SR scores under LR, HI, FL, OV, and CM.
- GOLA achieves the best success rates (SR) across the largest number of individual challenge attributes (including PO, LI, SO, FM, SV, and ARC) and the MVO subset, reflecting its strong generalization in handling diverse challenge factors and dynamic modality variations.
- Other trackers also show distinct strengths: TATrack achieves the highest Precision Rate (PR) under BC, SO, SV, and ARC, while the modality-missing tracker TMKD performs competitively under SA (similar appearance) and TC (thermal crossover).
Attribute- and Subset-based Performance on DRGBT-1K Test Set (Part 1)
| Attr | OSTrack | TBSI | TATrack | BAT | PURA | SDSTrack | MMLoRAT | CKD | AINet | STTrack |
|---|---|---|---|---|---|---|---|---|---|---|
| HO | 47.97/34.64 | 39.38/30.55 | 47.41/34.02 | 42.73/32.60 | 44.72/31.56 | 33.00/25.14 | 42.33/30.90 | 43.36/32.80 | 39.42/31.08 | 13.41/8.73 |
| PO | 39.01/32.05 | 36.61/30.00 | 39.05/31.84 | 36.74/30.64 | 36.63/29.95 | 32.79/26.40 | 38.22/30.60 | 37.05/30.55 | 36.26/30.08 | 7.79/8.62 |
| LI | 34.72/29.56 | 35.62/30.07 | 38.13/30.91 | 38.17/31.16 | 36.47/30.00 | 30.13/23.84 | 36.20/30.34 | 35.11/29.83 | 34.95/29.72 | 9.97/7.94 |
| LR | 36.17/27.70 | 33.56/26.44 | 36.93/28.40 | 36.06/28.27 | 35.57/26.41 | 31.23/23.65 | 34.85/27.55 | 34.17/27.46 | 33.59/27.13 | 13.49/8.01 |
| BC | 44.04/33.73 | 35.94/29.49 | 44.82/34.68 | 41.37/32.57 | 40.97/30.47 | 33.12/26.03 | 38.66/30.40 | 39.56/31.82 | 36.25/30.04 | 15.47/9.37 |
| HI | 33.41/24.90 | 31.51/22.72 | 28.56/22.22 | 29.78/20.53 | 24.43/18.97 | 25.61/17.75 | 24.14/18.17 | 34.72/24.54 | 26.51/20.06 | 4.46/6.08 |
| SA | 40.91/32.99 | 37.60/30.84 | 39.85/31.74 | 40.47/32.64 | 38.92/29.38 | 29.22/23.85 | 37.54/29.63 | 37.97/31.33 | 35.31/30.12 | 12.25/9.16 |
| FL | 35.82/25.15 | 34.22/24.77 | 36.98/26.96 | 34.56/24.87 | 32.62/22.86 | 27.21/20.44 | 34.55/24.96 | 34.05/25.05 | 32.17/23.76 | 6.69/6.55 |
| SO | 41.10/30.99 | 36.97/28.83 | 42.33/31.75 | 38.21/29.49 | 38.40/27.96 | 32.29/24.31 | 38.84/29.13 | 38.22/29.53 | 35.90/28.35 | 14.88/8.41 |
| FM | 38.39/32.59 | 37.32/32.06 | 39.95/33.64 | 40.33/34.22 | 38.17/31.84 | 31.42/26.67 | 36.70/31.83 | 37.27/32.24 | 36.05/31.54 | 5.79/9.19 |
| OV | 16.78/15.26 | 14.25/13.48 | 16.06/14.45 | 17.90/14.61 | 18.90/15.78 | 12.41/11.62 | 16.76/14.98 | 16.81/14.24 | 15.40/13.80 | 7.27/7.05 |
| SV | 40.48/32.36 | 36.97/30.34 | 41.62/33.22 | 39.21/31.78 | 39.22/30.43 | 32.83/26.18 | 38.74/30.94 | 38.10/31.27 | 36.69/30.32 | 11.54/8.79 |
| CM | 41.00/34.23 | 38.34/33.05 | 42.60/35.59 | 44.45/36.67 | 39.91/31.59 | 35.05/28.75 | 37.16/31.96 | 40.46/34.05 | 40.62/34.15 | 6.10/9.42 |
| TC | 25.77/21.71 | 24.86/20.57 | 27.99/22.06 | 25.22/20.94 | 29.84/22.53 | 19.93/15.83 | 31.42/23.87 | 26.77/22.23 | 23.78/19.91 | 4.50/7.26 |
| ARC | 39.49/31.77 | 35.95/29.81 | 41.19/32.96 | 37.74/30.94 | 38.07/29.83 | 32.04/25.62 | 37.79/30.37 | 37.11/30.62 | 35.80/29.73 | 11.09/8.56 |
| MVO | 77.25/54.24 | 65.01/48.22 | 78.72/56.21 | 69.83/52.08 | 70.42/49.35 | 51.20/36.79 | 69.09/50.30 | 69.34/51.29 | 64.67/48.24 | 21.01/11.41 |
| PVO | 35.12/30.14 | 35.71/30.29 | 36.97/30.67 | 37.02/31.09 | 37.74/29.91 | 35.24/27.70 | 35.81/30.06 | 36.11/30.86 | 35.53/30.64 | 11.11/8.46 |
| CV | 30.19/25.36 | 28.32/24.18 | 30.92/25.76 | 29.55/24.65 | 29.75/24.18 | 26.34/21.86 | 29.89/24.55 | 28.72/24.42 | 28.20/24.12 | 8.92/8.00 |
Attribute- and Subset-based Performance on DRGBT-1K Test Set (Part 2)
| Attr | CAFormer | IPT | FMTrack | QSTNet | UATrack | CMRL | MRTTrack | TMKD | SCDT | GOLA |
|---|---|---|---|---|---|---|---|---|---|---|
| HO | 39.64/30.46 | 37.03/27.82 | 40.95/30.98 | 38.79/29.71 | 46.84/33.91 | 46.71/34.56 | 40.02/30.04 | 46.10/34.83 | 24.74/18.14 | 45.30/34.15 |
| PO | 37.34/30.13 | 33.87/28.05 | 38.15/29.81 | 35.90/29.51 | 39.58/31.95 | 41.13/33.82 | 34.26/28.69 | 38.99/32.07 | 29.53/21.35 | 41.70/33.91 |
| LI | 36.47/30.22 | 34.38/29.14 | 36.53/29.74 | 34.23/28.89 | 37.26/30.29 | 36.65/30.29 | 35.38/29.39 | 35.74/30.30 | 28.76/22.87 | 37.43/31.35 |
| LR | 35.92/27.62 | 34.71/26.32 | 35.87/27.14 | 33.63/26.58 | 37.42/28.31 | 37.91/29.46 | 35.04/27.48 | 36.42/28.17 | 30.45/20.74 | 37.55/29.26 |
| BC | 38.08/30.92 | 38.84/29.55 | 40.19/30.93 | 37.44/29.75 | 41.19/31.84 | 40.55/32.14 | 37.87/30.57 | 43.53/33.86 | 27.17/20.17 | 43.34/33.81 |
| HI | 31.63/22.51 | 34.30/22.52 | 35.40/25.85 | 33.63/23.42 | 32.41/23.42 | 40.43/31.01 | 20.96/17.41 | 32.18/22.64 | 31.30/19.26 | 26.71/20.72 |
| SA | 37.95/30.81 | 40.22/31.04 | 36.90/30.43 | 39.57/30.78 | 41.40/32.49 | 36.83/30.74 | 40.82/31.47 | 41.67/33.64 | 29.55/22.39 | 36.81/31.07 |
| FL | 35.34/25.39 | 28.99/21.41 | 35.52/24.97 | 33.36/24.01 | 37.15/26.70 | 39.47/28.73 | 31.51/22.73 | 36.51/25.99 | 30.61/18.59 | 35.61/26.25 |
| SO | 38.37/29.43 | 36.60/27.32 | 38.82/28.98 | 37.84/29.13 | 41.59/31.01 | 39.39/30.45 | 35.63/27.80 | 40.85/31.16 | 29.89/21.52 | 41.52/31.80 |
| FM | 38.72/33.09 | 36.14/30.94 | 38.01/31.43 | 36.28/31.14 | 40.59/33.90 | 41.52/34.91 | 36.34/31.38 | 39.82/34.01 | 28.70/23.30 | 40.90/35.13 |
| OV | 15.28/13.49 | 15.94/13.79 | 16.79/13.27 | 15.50/13.06 | 17.21/13.87 | 25.47/19.35 | 14.70/13.86 | 16.79/14.90 | 12.48/10.71 | 17.55/15.29 |
| SV | 38.06/30.82 | 37.70/29.65 | 38.88/30.53 | 37.22/30.21 | 40.37/32.02 | 41.31/33.46 | 35.97/29.58 | 40.73/32.72 | 30.46/22.71 | 41.48/33.46 |
| CM | 41.25/34.92 | 41.23/32.33 | 37.28/31.52 | 39.37/32.69 | 40.44/33.74 | 46.91/39.21 | 37.85/32.15 | 44.13/36.00 | 28.82/23.45 | 42.43/35.60 |
| TC | 25.78/21.94 | 26.43/20.71 | 33.30/22.69 | 25.35/20.62 | 28.29/22.38 | 22.14/19.00 | 25.56/21.20 | 37.19/26.58 | 22.13/17.01 | 30.06/24.62 |
| ARC | 36.92/30.08 | 36.58/29.05 | 37.71/30.09 | 36.48/29.83 | 39.47/31.49 | 40.64/32.93 | 34.65/28.82 | 39.80/32.30 | 29.44/22.18 | 40.68/33.12 |
| MVO | 66.82/49.39 | 61.21/45.12 | 66.99/48.60 | 62.90/46.87 | 75.12/53.85 | 61.07/44.87 | 62.22/46.37 | 74.29/54.80 | 44.56/32.22 | 77.61/57.24 |
| PVO | 37.01/30.91 | 37.08/28.90 | 36.71/29.86 | 35.60/30.38 | 35.76/30.15 | 42.90/35.96 | 35.61/29.91 | 37.91/31.40 | 35.87/28.29 | 36.25/30.93 |
| CV | 29.35/24.51 | 29.90/24.25 | 30.48/24.40 | 29.43/24.42 | 30.63/25.13 | 34.61/28.46 | 28.05/23.88 | 30.52/25.42 | 23.15/16.70 | 31.45/26.12 |
Qualitative Results
8. UGVT-1K: UAV-Ground Collaborative Visual Tracking Benchmark
As a derivative of DRGBT-1K, we also release UGVT-1K — a UAV-Ground collaborative RGB-only tracking benchmark. UGVT-1K contains the same cross-platform sequences (split into 800 sequences for training and 245 for testing) but uses only the visible (RGB) modality, enabling research on multi-view visual tracking without thermal data.
License
This dataset is released under the Creative Commons Attribution-ShareAlike 4.0 International License (CC BY-SA 4.0).