TextBoost: Boosting Scene Text Fidelity in Ultra-low Bitrate Image Compression
Authors: Bingxin Wang, Yuan Lan, Zhaoyi Sun, Yang Xiang, Jie Sun
Organizations: Department of Mathematics, The Hong Kong University of Science and Technology (HKUST), Clear Water Bay, Sai Kung District, Hong Kong SAR, China · Huawei Hong Kong Research Center, 8/F, 2 Science Park West Avenue, Hong Kong Science Park, Sha Tin, Hong Kong SAR, China
Ultra-low bitrate image compression faces a critical challenge: preserving small-font scene text while maintaining overall visual quality. Region-of-interest (ROI) bit allocation can prioritize text but often degrades global fidelity, leading to a trade-off between local accuracy and overall image quality. Instead of relying on ROI coding, we incorporate auxiliary textual information extracted by OCR and transmitted with negligible overhead, enabling the decoder to leverage this semantic guidance. Our method, TextBoost, operationalizes this idea through three strategic designs: (i) adaptively filtering OCR outputs and rendering them into a guidance map; (ii) integrating this guidance with decoder features in a calibrated manner via an attention-guided fusion block; and (iii) enforcing guidance-consistent reconstruction in text regions with a regularizing loss that promotes natural blending with the scene. Extensive experiments on TextOCR and ICDAR 2015 demonstrate that TextBoost yields up to 60.6% higher text-recognition F1 at comparable Peak Signal-to-Noise Ratio (PSNR) and bits per pixel (bpp), producing sharper small-font text while preserving global image quality and effectively decoupling text enhancement from global rate-distortion optimization.
Figures & tables
Figure 1: TextBoost preserves small-text fidelity at ultra-low bitrates. Visual comparison on the TextOCR validation set [ 29 ] against ELIC [ 15 ] , TACO [ 20 ] , and LIC - TCM [ 23 ] . Our method delivers clearly better text reconstruction at similar or lower bitrates, with improved preservation of fine typographic details.
Figure 2: Overall pipeline of TextBoost. The framework is built upon a learned image compression backbone (comprising the Image Encoder, Hyperprior network, and Image Decoder). In parallel, the text branch extracts and transmits OCR information, which is processed by the Rendering-and-Alignment module to generate a visual guidance map. Finally, the Fusion Block integrates this guidance with the features decoded by the baseline network to produce the final output. SCCTX refers to the space-channel contextual model.
Figure 3: Examples of guidance maps generated from OCR information on the TextOCR dataset [ 29 ] . The first row shows the original scene images and the second row the corresponding rendering-and-alignment results. (1) and (3) illustrate robust handling of diverse in-plane orientations. By internally normalizing text to a horizontal layout, our method accurately renders text at distinct angles (e.g., vertical in (1) and slanted in (3)). (2) and (4) demonstrate selective transmission and precise rendering: large-font text is filtered out according to the average character-area criterion, while small-font content is rendered with accurate geometry and spatial placement.
Figure 4: Fusion block architecture and intermediate visualizations. Top: the proposed fusion block couples the auxiliary guidance with decoder features through element-wise modulation, channel expansion and concatenation, an attention module, and a final 1×1 projection to RGB. Bottom: visualizations aligned with each stage. (a) Hadamard product map : element-wise multiplication where white glyphs in the auxiliary map inherit color from the decoder features. (b) Concatenated features (13 + 3) : the first 13 channels (blue branch) are projected from the decoder output, while the last 3 channels (green branch) correspond to the Hadamard product map. (c) Attention heatmap : activations of the attention block, mostly concentrated on small-font text areas. (d) Reconstruction : the final image after the 1×1 projection.
Figure 5: Quantitative evaluation of text spotting performance. Rate-distortion curves showing Text Detection (DET) and End-to-End Recognition (E2E) F-measures on TextOCR [ 29 ] and ICDAR 2015 [ 18 ] . TextBoost (red curves) yields significant improvements over state-of-the-art learned compression methods (e.g., LIC-TCM, ELIC) and ROI-based baselines, demonstrating superior text preservation capabilities at ultra-low bitrates.
Figure 6: Additional visual comparisons across diverse street scenes on TextOCR. TextBoost produces sharper glyphs and cleaner fine strokes while retaining the global scene appearance.Notably, our method achieves this superior text fidelity at a lower average bitrate (0.025 bpp) compared to the baselines (0.027–0.029 bpp).
Figure 7: Global quality consistency on TextOCR: PSNR, LPIPS, and MS-SSIM. Comparison of rate-distortion curves against state-of-the-art baselines. TextBoost maintains competitive global fidelity, achieving PSNR/MS-SSIM comparable to ELIC and LIC-TCM while offering better perceptual quality (lower LPIPS). With the LIC-TCM backbone ( TextBoost-TCM ), our method effectively matches the original LIC-TCM in reconstruction metrics, demonstrating that our text enhancement incurs negligible cost to global scene statistics.
Figure 8: Generalization to ICDAR 2015: PSNR, LPIPS, and MS-SSIM. Consistent with TextOCR results, TextBoost maintains high global quality on the ICDAR 2015 dataset [ 18 ] . Notably, TextBoost achieves strong PSNR and LPIPS performance at ultra-low bitrates, indicating that the proposed guidance-based fusion mechanism generalizes well across different scene distributions.
Method
bpp
PSNR
MS-SSIM
LPIPS
ELIC
0.027
24.22
0.8110
0.42
TACO
0.028
23.04
0.7720
0.09
LIC-TCM
0.028
24.53
0.8197
0.55
TextBoost
0.028
24.70
0.8150
0.42
Table 1: Performance comparison on the Kodak dataset. Our method achieves the best reconstruction quality in terms of PSNR and maintains competitive structural (MS-SSIM) and perceptual (LPIPS) fidelity compared to state-of-the-art baselines.
Figure 9: Ablation study: Effectiveness of key components. Quantitative comparison of DET and E2E performance on TextOCR. We observe that both the Fusion Block (purple) and the Guidance-consistent Loss (green) individually improve upon the ELIC baseline (cyan). However, the full TextBoost framework (red) yields the highest scores, demonstrating the necessity and complementary nature of combining adaptive feature integration with loss-based constraints.
Scene Text Image Super-Resolution (STISR) aims to restore high-resolution details in low-resolution text images, which is crucial for both human readability and machine recognition. Existing methods, however, often depend on external Optical Character Recognition (OCR) models for textual priors or rely on complex multi-component architectures that are difficult to train and reproduce. In this paper, we introduce DualTSR, a unified end-to-end framework that addresses both issues. DualTSR employs a single multimodal transformer backbone trained with a dual diffusion objective. It simultaneously models the continuous distribution of high-resolution images via Conditional Flow Matching and the discrete distribution of textual content via discrete diffusion. This shared design enables visual and textual information to interact at every layer, allowing the model to infer text priors internally instead of relying on an external OCR module. Compared with prior multi-branch diffusion systems, DualTSR offers a simpler end-to-end formulation with fewer hand-crafted components. Experiments on synthetic Chinese benchmarks and a curated real-world evaluation protocol show that DualTSR achieves strong perceptual quality and text fidelity.
Axi Niu, Kang Zhang, Qingsen Yan +3
Northwestern Polytechnical University · Korea Advanced Institute of Science and Technology
Text image super-resolution (TSR) aims to recover visually faithful and readable text under unknown degradations. Existing diffusion-based methods typically rely on multi-step prediction of either the high-resolution image or its text prior, resulting in prohibitive computational cost and inference latency. More critically, an erroneous text prior may be repeatedly injected into the denoising process, causing image and text predictions to reinforce each other and progressively amplify an early recognition error into a sharp yet semantically incorrect character. To address these limitations, we propose TOLA, a Text-aware One-step Latent Adaptation framework without iterative image-text diffusion. TOLA consists of two key modules. First, a confidence-weighted text conditioning module constructs the semantic condition only once and suppresses unreliable OCR predictions before they contaminate image reconstruction. Second, a lightweight latent residual correction module explicitly estimates and corrects the structured residual errors to recover missing or distorted stroke details. Extensive experiments demonstrate our state-of-the-art performance across all evaluation metrics on both CTR-TSR-Test (×4) and RealCE-200 benchmarks. It is worth noting that our TOLA consistently surpasses existing diffusion-based TSR methods by at least 2.72 dB in PSNR on CTR-TSR-Test.
Yike Xu, Yue Shi, Yong Guo +1
Shanghai Jiao Tong University · University of Chinese Academy of Sciences · Shanghai AI Laboratory
Accurate text recognition in low-light environments is essential for intelligent systems in applications ranging from autonomous vehicles to smart surveillance. However, challenges such as poor illumination and noise interference remain underexplored. To address this gap, we introduce LSTR, a large-scale Low-light Scene Text Recognition dataset comprising 11,273 low-light images generated from well-lit datasets (ICDAR2015, IIIT5K, and WordArt), along with ESTR, which includes 60 real nighttime street-scene images in English and Spanish for exclusive evaluation. We explore two solution strategies: (1) employing Optical Character Recognition (OCR) models with fine-tuning and LoRA-based fine-tuning and (2) a joint training strategy that integrates a low-light image enhancement (LLIE) module with an OCR model. In particular, we propose a novel re-render LLIE (RLLIE) module, which demonstrates improved performance on real-world data. Through extensive experimentation, we analyze various training strategies and address a key research question: How bright is bright enough for effective scene text recognition? Our results indicate that standalone LLIE or OCR models perform inadequately under low-light conditions, highlighting the advantages of specialized, jointly trained text-centric approaches. Additionally, we provide a comprehensive benchmark to support future research in robust low-light scene text recognition. https://huggingface.co/datasets/lumimusta/Low-light_Scene_Text_Dataset.