SVTRv2: CTC Beats Encoder-Decoder Models in Scene Text Recognition

[Paper] [Doc] [Model] [Datasets] [Config, Training and Inference] [Benchmark]

Introduction

Connectionist temporal classification (CTC)-based scene text recognition (STR) methods, e.g., SVTR, are widely employed in OCR applications, mainly due to their simple architecture, which only contains a visual model and a CTC-aligned linear classifier, and therefore fast inference. However, they generally have worse accuracy than encoder-decoder-based methods (EDTRs), particularly in challenging scenarios. In this paper, we propose SVTRv2, a CTC model that beats leading EDTRs in both accuracy and inference speed. SVTRv2 introduces novel upgrades to handle text irregularity and utilize linguistic context, which endows it with the capability to deal with challenging and diverse text instances. First, a multi-size resizing (MSR) strategy is proposed to adaptively resize the text and maintain its readability. Meanwhile, we introduce a feature rearrangement module (FRM) to ensure that visual features accommodate the alignment requirement of CTC well, thus alleviating the alignment puzzle. Second, we propose a semantic guidance module (SGM). It integrates linguistic context into the visual model, allowing it to leverage language information for improved accuracy. Moreover, SGM can be omitted at the inference stage and would not increase the inference cost. We evaluate SVTRv2 in both standard and recent challenging benchmarks, where SVTRv2 is fairly compared with 24 mainstream STR models across multiple scenarios, including different types of text irregularity, languages, and long text. The results indicate that SVTRv2 surpasses all the EDTRs across the scenarios in terms of accuracy and speed.

A Unified Training and Evaluation Benchmark for Scene Text Recognition

Recent research shows that STR models' performance can be been significantly improved by leveraging large-scale real-world datasets. However, many previous methods are trained on synthetic datasets, which fail to reflect their performance in real-world scenarios. Additionally, recent approaches have used different real-world datasets and inconsistent evaluation protocols, making it difficult to compare their performance.

To this end, we established a unified benchmark to re-train and evaluate mainstream STR methods.

First, to evaluate the performance of STR methods across diverse scenarios, we selected Union14M-Benchmarks as the test set. This benchmark includes a variety of complex scenarios. Additionally, we reported results on six test sets (Common Benchmarks) used in previous studies. For the training set, we used the large-scale real-world training dataset Union14M-L. To avoid data leakage, we filtered out the overlapping samples between Union14M-L (training set) and Union14M-L-Benchmark (Test set), resulting in the Union14M-L-Filter training dataset.

Furthermore, previous methods used inconsistent hyperparameter settings during training, which contributed to variations in their performance. To ensure reliable evaluation, we standardized key settings that significantly affect accuracy, such as the number of training epochs, data augmentation strategies, input size, and evaluation strategies. This ensures the reliability of our results.

Subsequently, we trained 24 reproduced STR methods and SVTRv2 using the Union14M-L-Filter dataset and evaluated their performance on both Common Benchmarks and Union14M-L-Benchmark.

Dataset Details

Training Dataset

• Unified Training Set: All models are trained from scratch on a unified dataset named Union14M-L-Filter. This dataset was derived from Union14M-L, with certain adjustments for filtering.
• Composition of Datasets: The training dataset includes different categories of samples, categorized as Easy, Medium, Hard, Norm, and Challenging.
  • Union14M-L: Contains 3,230,742 images in total.
  • Union14M-L-Filter: The filtered version contains 3,224,143 images.
  • Overlap with Union14M-Benchmarks: Only 6,599 images overlap between Union14M-L and the benchmark datasets used for evaluation.

Test Datasets

1. Common Test Set: Previous methods have primarily been evaluated on this set. While it includes some irregular text, the datasets are not highly challenging (e.g., little curva or rotation). Models trained on synthetic datasets often perform well here.
   • Includes 6 subsets with regular and irregular samples.
   • Example datasets: IIIT5K, SVT, IC13, IC15, SVTP, and CUTE80.

2. Challenging Test Set (Union14M-L-Benchmark): Introduced to test the full capabilities of STR (Scene Text Recognition) models. This set includes significantly more difficult text samples, featuring extreme curva, rotation, artistic styles, overlapping text, and other challenges.
   • Includes datasets such as Curve, Multi-Oriented, Artistic, Contextless, Salient, Multi-word, and General.

3. Special Test Sets: Designed to evaluate specific model capabilities beyond the above datasets.
   • LTB (Long Text Benchmark): Evaluates performance on long texts (25–36 characters), as models are typically trained on short texts (≤25 characters).
   • OST (Occluded Scene Text): Tests the model’s ability to infer text from damaged or partially erased samples.

Note: Both Union14M-L-Filter and Union14M-L-Benchmark are based on Union14M-L and therefore comply with its copyright. Additionally, Common Benchmarks and OST are derived from PARSeq and VisionLAN, respectively.

Implementation Details

The optimal training hyperparameters for all models are usually not fixed. However, key setting such as Training Epochs, Data Augmentation, Input Size, Data Type, and Evaluation Protocols—which significantly impact accuracy—must be strictly standardized to ensure fair and unbiased comparisons of model performance. By following these standardizations, the results can accurately reflect the true capabilities of the models, unaffected by experimental inconsistencies. The specific setting include:

Get Started with training a SoTA Scene Text Recognition model from scratch.

Installation

• PyTorch version >= 1.13.0
• Python version >= 3.7

git clone https://github.com/Topdu/OpenOCR.git
cd OpenOCR
# Ubuntu 20.04 Cuda 11.8
conda create -n openocr python==3.8
conda activate openocr
conda install pytorch==2.2.0 torchvision==0.17.0 torchaudio==2.2.0 pytorch-cuda=11.8 -c pytorch -c nvidia
pip install -r requirements.txt

Downloading Datasets

All data can be downloaded from huggingface.

The structure of Datasets and OpenOCR code will be organized as follows:

```text
benchmark_bctr # Chinese text datasets, optional
├── benchmark_bctr_test
│   ├── document_test
│   ├── handwriting_test
│   ├── scene_test
│   └── web_test
└── benchmark_bctr_train
    ├── document_train
    ├── handwriting_train
    ├── scene_train
    └── web_train
evaluation
├── CUTE80
├── IC13_857
├── IC15_1811
├── IIIT5k
├── SVT
└── SVTP
iiit5k_test_images # for Latency Measurement, optional
ltb # Long Text Benchmark
OpenOCR
OST
synth # optional
├── MJ
│   ├── test
│   ├── train
│   └── val
└── ST
test # Common Benchmarks from PARSeq
├── ArT
├── COCOv1.4
├── CUTE80
├── IC13_1015
├── IC13_1095
├── IC13_857
├── IC15_1811
├── IC15_2077
├── IIIT5k
├── SVT
├── SVTP
└── Uber
u14m # lmdb format Union14M-Benchmark
├── artistic
├── contextless
├── curve
├── general
├── multi_oriented
├── multi_words
└── salient
Union14M-L-LMDB-Filtered # lmdb format Union14M-L-Filtered
├── train_challenging
├── train_easy
├── train_hard
├── train_medium
└── train_normal
```

Datasets used during Training

If you have downloaded Union14M-L, you can use the filtered list of images to create an LMDB of the training set Union14M-L-Filter (detailed in create_lmdb_dataset.py).

Test Set

Note: Both Union14M-L-Filter and Union14M-L-Benchmark are based on Union14M-L and therefore comply with its copyright. Common Benchmarks and OST are derived from PARSeq and VisionLAN, respectively.

Training & Evaluation & Inference & Latency Measurement

Note: Take SVTRv2 as an example here. The execution commands for each model are listed in detail on their readme pages.

Training

# Multi GPU training
CUDA_VISIBLE_DEVICES=0,1,2,3 python -m torch.distributed.launch --nproc_per_node=4 tools/train_rec.py --c configs/rec/svtrv2/svtrv2_smtr_gtc_rctc.yml
# For Multi RTX 4090
NCCL_P2P_DISABLE=1 CUDA_VISIBLE_DEVICES=0,1,2,3 python -m torch.distributed.launch --nproc_per_node=4 tools/train_rec.py --c configs/rec/svtrv2/svtrv2_smtr_gtc_rctc.yml
# 20epoch runs for about 6 hours

Evaluation

# short text: Common, Union14M-Benchmark, OST
python tools/eval_rec_all_en.py --c configs/rec/svtrv2/svtrv2_smtr_gtc_rctc.yml

After a successful run, the results are saved in a csv file in output_dir in the config file.

Inference

python tools/infer_rec.py --c configs/rec/svtrv2/svtrv2_smtr_gtc_rctc.yml --o Global.infer_img=/path/img_fold or /path/img_file

Latency Measurement

Firstly, downloading the IIIT5K images from Google Drive. Then, running the following command:

python tools/infer_rec.py --c configs/rec/svtrv2/svtrv2_rctc.yml --o Global.infer_img=../iiit5k_test_image

Results (Benchmark) & Configs & Checkpoints:

(TODO) Downloading all model checkpoints from Google Drive and Baidu Yun.

Note: TF$\_n$ denotes the $n$-layer Transformer block. $Size$ denotes the number of parameters ($M$). $Latency$ is measured on one NVIDIA 1080Ti GPU with Pytorch dynamic graph mode.

Results when trained on synthetic datasets ($ST$ + $MJ$).

Note: * represents that the results on Union14M-Benchmarks are evaluated using the model they released.

Citation

If you find our method useful for your reserach, please cite:

@inproceedings{Du2025SVTRv2,
      title={SVTRv2: CTC Beats Encoder-Decoder Models in Scene Text Recognition},
      author={Yongkun Du and Zhineng Chen and Hongtao Xie and Caiyan Jia and Yu-Gang Jiang},
      booktitle={ICCV},
      year={2025},
      pages={20147-20156}
}