Text spotting requires both accurate text recognition and precise spatial localization. Current specialised spotters excel at predicting tight bounding boxes in natural scenes, but falter on complex or artistic text, whereas multimodal large language models (MLLMs) possess strong recognition capabilities yet remain weak at localisation. To equip the text spotter with general and powerful recognition capabilities and to maximize its localization ability, we explore two MLLM-based fine-tuning methods: Supervised Fine-Tuning (SFT) and reinforcement learning fine-tuning based on Group Relative Policy Optimisation (GRPO). An interesting finding is that SFT is less effective than GRPO at enhancing recognition, while GRPO is less effective than SFT at enhancing detection. To compensate for each other's shortcomings, we introduce a joint training strategy, SupGRPO, which simultaneously optimizes the model using both SFT and GRPO. SupGRPO employs the specially designed reward functions and develops a matching-based online SFT applied solely to coordinate tokens. It both mitigates the reward sparsity problem of GRPO and avoids the instance order dependency problem of SFT. To evaluate particularly challenging cases, we curate ATS, a dataset for artistic text spotting. Experiments demonstrate that SupGRPO improves both text recognition and detection, and attains superior performance. Our code and dataset will be released at https://github.com/Psycho-9/SupGRPO.
Figures & tables
Figure 1: Comparison with other models and other training strategies.
Figure 2: The overall framework of SupGRPO.
Figure 3: Examples of different types for artistic text spotting from our ATS dataset
Method
Total-Text
ICDAR 2015
CTW1500
ATS
None
Full
S
W
G
None
Full
None
Mask TextSpotter v3 [ 21 ]
71.2
78.4
83.3
78.1
74.2
−
−
56.9
ABCNet v2 [ 25 ]
70.4
78.1
82.7
78.5
73.0
57.5
77.2
53.4
GLASS [ 31 ]
79.9
86.2
84.7
80.1
76.3
−
−
−
TESTR [ 45 ]
73.3
83.9
85.2
79.4
73.6
56.0
81.5
55.0
SwinTextSpotter [ 15 ]
74.3
84.1
77.3
70.5
−
51.8
77.0
55.4
Table 1: Standard end-to-end text spotting results.
Method
Total-Text
ICDAR 2015
CTW1500
ATS
None
Full
S
W
G
None
Full
None
Qwen2.5-VL-7B [ 2 ]
78.1
87.0
89.0
83.5
79.2
73.2
80.4
72.7
InternVL3.5-8B [ 37 ]
81.7
86.7
87.8
84.7
81.5
71.5
78.5
72.1
Gemini 3.1-Pro [ 30 ]
84.0
90.0
91.2
86.6
83.2
78.4
85.1
81.6
TextMonkey [ 26 ]
67.4
76.9
72.9
66.7
60.6
55.2
76.5
50.9
GOT-OCR [ 39 ]
70.3
79.2
78.9
73.2
69.8
59.5
80.2
56.7
Table 2: Recognition-only word-level F1 results of MLLM-based methods.
Method
Det.
ABCNet v2 [ 25 ]
87.5
ABINet++ [ 45 ]
88.2
TESTR [ 45 ]
86.4
SwinTextSpotter [ 15 ]
85.9
DeepSolo [ 42 ]
86.7
Qwen2.5-VL-7B [ 2 ]
27.4
Table 3: Detection results on ATS.
Method
Total
IC15
TextPerceptron [ 29 ]
85.2
87.1
PGNet [ 36 ]
86.1
88.2
ABCNet v2 [ 25 ]
87.0
88.1
DeepSolo [ 42 ]
87.3
90.0
Bridge [ 14 ]
89.2
90.5
Qwen2.5-VL-7B [ 2 ]
23.2
19.2
Table 4: Detection results on Total-Text and ICDAR 2015.
Training Data
Total (Det./Rec.)
IC15 (Det./Rec.)
ATS (Det./Rec.)
CTW (Rec.)
Without ATS
65.8/83.9
67.2/82.8
65.3/75.7
77.9
With ATS (Ours)
68.4/86.0
71.6/84.7
71.6/84.4
81.8
Table 5: Ablation study on introducing the ATS dataset during training.
Det.
Rec.
SFT
63.3
77.1
SFT + Aug
65.1
77.4
GRPO
62.0
80.0
SupGRPO
71.6
84.4
Table 6: Training strategy ablation.
Det.
Rec.
GRPO
62.0
80.0
+ All Tokens SFT
64.0
81.2
+ Text Tokens SFT
60.8
84.1
+ Location Tokens SFT
71.6
84.4
Table 7: SupGRPO for different tokens.
Det.
Rec.
All Tokens SFT
64.0
81.2
IoU-based
66.5
81.7
Text-based
68.6
82.8
IoU & Text
71.6
84.4
Table 8: Different matching mechanisms for online SFT location tokens.
Det.
Rec.
Baseline
26.3
70.2
Text
61.1
83.2
Text & F
69.3
83.5
Text & P & R
71.6
84.4
Table 9: Effect of different reward function designs.
Figure 4: Training curve comparison between the vanilla GRPO and our SupGRPO.