TextBoost: Boosting Scene Text Fidelity in Ultra-low Bitrate Image Compression

Bingxin Wang; Yuan Lan; Zhaoyi Sun; Yang Xiang; Jie Sun

cs.CVMar 4, 2026

TextBoost: Boosting Scene Text Fidelity in Ultra-low Bitrate Image Compression

Authors: Bingxin Wang, Yuan Lan, Zhaoyi Sun, Yang Xiang, Jie Sun

Organizations: Department of Mathematics, The Hong Kong University of Science and Technology (HKUST), Clear Water Bay, Sai Kung District, Hong Kong SAR, China · Huawei Hong Kong Research Center, 8/F, 2 Science Park West Avenue, Hong Kong Science Park, Sha Tin, Hong Kong SAR, China

Abstract

Ultra-low bitrate image compression faces a critical challenge: preserving small-font scene text while maintaining overall visual quality. Region-of-interest (ROI) bit allocation can prioritize text but often degrades global fidelity, leading to a trade-off between local accuracy and overall image quality. Instead of relying on ROI coding, we incorporate auxiliary textual information extracted by OCR and transmitted with negligible overhead, enabling the decoder to leverage this semantic guidance. Our method, TextBoost, operationalizes this idea through three strategic designs: (i) adaptively filtering OCR outputs and rendering them into a guidance map; (ii) integrating this guidance with decoder features in a calibrated manner via an attention-guided fusion block; and (iii) enforcing guidance-consistent reconstruction in text regions with a regularizing loss that promotes natural blending with the scene. Extensive experiments on TextOCR and ICDAR 2015 demonstrate that TextBoost yields up to 60.6% higher text-recognition F1 at comparable Peak Signal-to-Noise Ratio (PSNR) and bits per pixel (bpp), producing sharper small-font text while preserving global image quality and effectively decoupling text enhancement from global rate-distortion optimization.

Figures & tables

Figure 1: TextBoost preserves small-text fidelity at ultra-low bitrates. Visual comparison on the TextOCR validation set [ 29 ] against ELIC [ 15 ] , TACO [ 20 ] , and LIC - TCM [ 23 ] . Our method delivers clearly better text reconstruction at similar or lower bitrates, with improved preservation of fine typographic details.

Figure 2: Overall pipeline of TextBoost. The framework is built upon a learned image compression backbone (comprising the Image Encoder, Hyperprior network, and Image Decoder). In parallel, the text branch extracts and transmits OCR information, which is processed by the Rendering-and-Alignment module to generate a visual guidance map. Finally, the Fusion Block integrates this guidance with the features decoded by the baseline network to produce the final output. SCCTX refers to the space-channel contextual model.

Figure 3: Examples of guidance maps generated from OCR information on the TextOCR dataset [ 29 ] . The first row shows the original scene images and the second row the corresponding rendering-and-alignment results. (1) and (3) illustrate robust handling of diverse in-plane orientations. By internally normalizing text to a horizontal layout, our method accurately renders text at distinct angles (e.g., vertical in (1) and slanted in (3)). (2) and (4) demonstrate selective transmission and precise rendering: large-font text is filtered out according to the average character-area criterion, while small-font content is rendered with accurate geometry and spatial placement.

Figure 4: Fusion block architecture and intermediate visualizations. Top: the proposed fusion block couples the auxiliary guidance with decoder features through element-wise modulation, channel expansion and concatenation, an attention module, and a final 1×1 projection to RGB. Bottom: visualizations aligned with each stage. (a) Hadamard product map : element-wise multiplication where white glyphs in the auxiliary map inherit color from the decoder features. (b) Concatenated features (13 + 3) : the first 13 channels (blue branch) are projected from the decoder output, while the last 3 channels (green branch) correspond to the Hadamard product map. (c) Attention heatmap : activations of the attention block, mostly concentrated on small-font text areas. (d) Reconstruction : the final image after the 1×1 projection.

Figure 5: Quantitative evaluation of text spotting performance. Rate-distortion curves showing Text Detection (DET) and End-to-End Recognition (E2E) F-measures on TextOCR [ 29 ] and ICDAR 2015 [ 18 ] . TextBoost (red curves) yields significant improvements over state-of-the-art learned compression methods (e.g., LIC-TCM, ELIC) and ROI-based baselines, demonstrating superior text preservation capabilities at ultra-low bitrates.

Figure 6: Additional visual comparisons across diverse street scenes on TextOCR. TextBoost produces sharper glyphs and cleaner fine strokes while retaining the global scene appearance.Notably, our method achieves this superior text fidelity at a lower average bitrate (0.025 bpp) compared to the baselines (0.027–0.029 bpp).

Figure 7: Global quality consistency on TextOCR: PSNR, LPIPS, and MS-SSIM. Comparison of rate-distortion curves against state-of-the-art baselines. TextBoost maintains competitive global fidelity, achieving PSNR/MS-SSIM comparable to ELIC and LIC-TCM while offering better perceptual quality (lower LPIPS). With the LIC-TCM backbone ( TextBoost-TCM ), our method effectively matches the original LIC-TCM in reconstruction metrics, demonstrating that our text enhancement incurs negligible cost to global scene statistics.

Figure 8: Generalization to ICDAR 2015: PSNR, LPIPS, and MS-SSIM. Consistent with TextOCR results, TextBoost maintains high global quality on the ICDAR 2015 dataset [ 18 ] . Notably, TextBoost achieves strong PSNR and LPIPS performance at ultra-low bitrates, indicating that the proposed guidance-based fusion mechanism generalizes well across different scene distributions.

Method	bpp	PSNR	MS-SSIM	LPIPS
ELIC	0.027	24.22	0.8110	0.42
TACO	0.028	23.04	0.7720	0.09
LIC-TCM	0.028	24.53	0.8197	0.55
TextBoost	0.028	24.70	0.8150	0.42

Table 1: Performance comparison on the Kodak dataset. Our method achieves the best reconstruction quality in terms of PSNR and maintains competitive structural (MS-SSIM) and perceptual (LPIPS) fidelity compared to state-of-the-art baselines.

Figure 9: Ablation study: Effectiveness of key components. Quantitative comparison of DET and E2E performance on TextOCR. We observe that both the Fusion Block (purple) and the Guidance-consistent Loss (green) individually improve upon the ELIC baseline (cyan). However, the full TextBoost framework (red) yields the highest scores, demonstrating the necessity and complementary nature of combining adaptive feature integration with loss-based constraints.

Learned Image Compression

Learned image compression uses neural encoders, quantization, and entropy models to reduce image bitrates while balancing fidelity, perceptual quality, and task utility.

See all 120 papers

DualTSR: Unified Dual-Diffusion Transformer for Scene Text Image Super-Resolution
Mar 15, 2026Axi Niu, Kang Zhang, Qingsen Yan +3Image Super-Resolution Diffusion Transformer
Northwestern Polytechnical University · Korea Advanced Institute of Science and Technology
TOLA: Text-aware One-Step Latent Adaptation for Diffusion-based Text Image Super-Resolution
Sep 24, 2026Yike Xu, Yue Shi, Yong Guo +1Diffusion Models for Image Restoration Image Super-Resolution
Shanghai Jiao Tong University · University of Chinese Academy of Sciences · Shanghai AI Laboratory
Reading in the Dark: Low-light Scene Text Recognition
Apr 26, 2026Xuanshuo Fu, Lei Kang, Ernest Valveny +2Scene Text Recognition Low-Light Image Enhancement
Computer Vision Center, Barcelona, Spain

Mar 15, 2026J/K move · Enter open · S save

DualTSR: Unified Dual-Diffusion Transformer for Scene Text Image Super-Resolution

Axi Niu, Kang Zhang, Qingsen Yan, Hao Jin+2

Northwestern Polytechnical University · Korea Advanced Institute of Science and Technology

Image Super-Resolution Diffusion Transformer Diffusion Models Multimodal Diffusion Models

Open paper AskPDF arXiv

Scene Text Image Super-Resolution (STISR) aims to restore high-resolution details in low-resolution text images, which is crucial for both human readability and machine recognition. Existing methods, however, often depend on external Optical Character Recognition (OCR) models for textual priors or rely on complex multi-component architectures that are difficult to train and reproduce. In this paper, we introduce DualTSR, a unified end-to-end framework that addresses both issues. DualTSR employs a single multimodal transformer backbone trained with a dual diffusion objective. It simultaneously models the continuous distribution of high-resolution images via Conditional Flow Matching and the discrete distribution of textual content via discrete diffusion. This shared design enables visual and textual information to interact at every layer, allowing the model to infer text priors internally instead of relying on an external OCR module. Compared with prior multi-branch diffusion systems, DualTSR offers a simpler end-to-end formulation with fewer hand-crafted components. Experiments on synthetic Chinese benchmarks and a curated real-world evaluation protocol show that DualTSR achieves strong perceptual quality and text fidelity.

Abstract

Learned Image Compression

Explore similar work

DualTSR: Unified Dual-Diffusion Transformer for Scene Text Image Super-Resolution

TOLA: Text-aware One-Step Latent Adaptation for Diffusion-based Text Image Super-Resolution

Reading in the Dark: Low-light Scene Text Recognition