Towards Accurate Scene Text Recognition with Semantic Reasoning Networks

Unofficial PyTorch implementation of the paper, which integrates not only global semantic reasoning module but also parallel visual attention module and visual-semantic fusion decoder.the semanti reasoning network(SRN) can be trained end-to-end.

At present, the accuracy of the paper cannot be achieved. And i borrowed code from deep-text-recognition-benchmark

model

result

IIIT5k_3000	SVT	IC03_860	IC03_867	IC13_857	IC13_1015	IC15_1811	IC15_2077	SVTP	CUTE80
84.600	83.617	92.907	92.849	90.315	88.177	71.010	68.064	71.008	68.641

total_accuracy: 80.597

---

Feature

• predict the character at once time
• DistributedDataParallel training

---

Requirements

Pytorch >= 1.1.0

Test

1. download the evaluation data from deep-text-recognition-benchmark

2. download the pretrained model from Baidu, Password: d2qn

3. test on the evaluation data

python test.py --eval_data path-to-data --saved_model path-to-model

---

Train

1. download the training data from deep-text-recognition-benchmark

2. training from scratch

python train.py --train_data path-to-train-data --valid-data path-to-valid-data

Reference

1. bert_ocr.pytorch
2. deep-text-recognition-benchmark
3. 2D Attentional Irregular Scene Text Recognizer
4. Towards Accurate Scene Text Recognition with Semantic Reasoning Networks

difference with the origin paper

• use resnet for 1D feature not resnetFpn 2D feature
• use add not gated unit for visual-semanti fusion decoder

other

It is difficult to achieve the accuracy of the paper, hope more people to try and share