DDVR-Net: Degradation-Aware Decoupling and Visibility-Guided Restoration for Historical Document Recognition

Jiu Sha; Yu Weng; Mengxiao Zhu

cs.CVOct 12, 2026

DDVR-Net: Degradation-Aware Decoupling and Visibility-Guided Restoration for Historical Document Recognition

Authors: Jiu Sha, Yu Weng, Mengxiao Zhu

Organizations: Institute of National Security, Minzu University of China, Beijing, China · School of Information Engineering, Minzu University of China, Beijing, China · School of Artificial Intelligence and Computer Science, North China University of Technology, Beijing, China

Abstract

Historical document analysis is crucial for cultural heritage preservation and low-resource language digitization. However, real historical documents often suffer from severe degradation, such as ink erosion and structural damage, causing mixed-visibility character sequences. Conventional OCR systems struggle in such evidence-limited scenarios, while contextual language models may over-correct visually clear characters. To address these challenges, we propose DDVR-Net, a degradation-aware decoupling and visibility-guided restoration framework for historical document recognition. DDVR-Net decouples intrinsic glyph structures from physical corruption at the feature level and predicts local visibility to estimate character reliability. A contextual decoder is selectively activated through a visibility-guided gate, restoring degraded characters while preserving clear visual recognition. Furthermore, we construct a low-resource multilingual benchmark covering Tibetan, Mongolian, and Uyghur scripts with 187000 document pairs, combining large-scale synthetic data and progressive training with real-world annotations to reduce the synthetic-to-real gap. Extensive experiments demonstrate that DDVR-Net consistently outperforms existing methods across three languages and diverse reading orders, providing an effective framework and benchmark for low-resource historical document recognition. Code and Benchmark: https://github.com/Shajiu/DDVR-Net

Figures & tables

Figure 1. Overview of benchmark construction.

Lang.	Subset	Imgs	Dam./Im	Tot. Dam.	Char/Im	Cls.	Train	Test
Tibetan	Manual	2000	105.40	210800	512.35	1120	1400	600
	Semi-Auto	12000	62.15	745800	485.20	3250	8400	3600
	Fully Auto	50000	58.20	2910000	498.60	3480	35000	15000
Mongolian	Manual	1500	88.65	132975	355.40	650	1050	450
	Semi-Auto	10000	45.30	453000	320.15	1840	7000	3000
	Fully Auto	50000	42.80	2140000	332.40	1950	35000	15000

Table 1. Statistics of the multilingual historical document restoration dataset. “Dam./Im” denotes average damaged characters per image, “Tot. Dam.” the total damaged characters, and “Cls.” the number of character categories.

Figure 2. Overall framework of DDVR-Net.

Panel of Figure 3 — Figure 3. Analysis of DDVR-Net: (a) visibility stratification, (b) progressive curriculum, and (c) case study.

Lang.	Models	Character Recognition & Restoration				Sequence-Level		Visibility Estimation			Degradation-Type Classification
Lang.	Models	$\mathrm{Acc}_{\mathrm{char}}$	$\mathrm{Acc}_{\mathrm{rest}}$	$\mathrm{Acc}_{\mathrm{sev}}$	$\mathrm{CER}\downarrow$	$\mathrm{LEM}$	$\mathrm{SeqAcc}$	$\mathrm{Acc}_{\mathrm{clear}}$	$\mathrm{Acc}_{\mathrm{partial}}$	$\mathrm{Acc}_{\mathrm{severe}}$	$\mathrm{Micro\mbox{-}F1}$	$\mathrm{Macro\mbox{-}F1}$
Tibetan	GPT-4o	58.72	38.41	22.15	28.43	12.36	7.84	66.51	47.62	32.75	46.85	44.29
	Gemini 3.1 Pro	57.84	37.66	21.43	29.15	11.89	7.41	65.26	46.19	31.84	45.38	43.12
	Qwen3-VL	61.29	42.18	25.84	26.87	14.72	9.65	68.14	49.37	34.22	48.73	46.51
	dots.ocr	52.34	28.15	12.39	45.67	4.51	2.18	60.12	37.53	21.65	36.42	33.87
	PaddleOCR 3.0	55.42	32.47	15.63	42.14	6.82	3.59	63.85	41.26	24.81	40.19	37.43

Table 2. Overall results of different models on Tibetan, Mongolian, and Uyghur historical documents.

Lang.	Training Paradigm	Character Recognition & Restoration				Sequence-Level
Lang.	Training Paradigm	$\mathrm{Acc}_{\mathrm{char}}$	$\mathrm{Acc}_{\mathrm{rest}}$	$\mathrm{Acc}_{\mathrm{sev}}$	$\mathrm{CER}\downarrow$	$\mathrm{SeqAcc}$
Tibetan	Full DDVR-Net	86.47	74.29	61.54	9.83	36.75
	Confidence-based Correction	85.10	73.60	62.10	10.90	32.60
	Post-hoc Correction	84.80	72.90	59.80	11.20	31.80
	Uniform Context	84.20	72.80	60.90	11.70	29.40
	Local-only	83.90	68.90	54.60	12.80	30.70

Table 3. Fair same-backbone comparison across languages.

Lang.	Models	Character Recognition & Restoration				Visibility Estimation		Degradation-Type Classification
Lang.	Models	$\mathrm{Acc}_{\mathrm{char}}$	$\mathrm{Acc}_{\mathrm{rest}}$	$\mathrm{Acc}_{\mathrm{sev}}$	$\mathrm{CER}\downarrow$	$\mathrm{Acc}_{\mathrm{partial}}$	$\mathrm{Acc}_{\mathrm{severe}}$	$\mathrm{Micro\mbox{-}F1}$	$\mathrm{Macro\mbox{-}F1}$
Tibetan	Full DDVR-Net	86.47	74.29	61.54	9.83	78.46	69.18	81.24	79.67
	w/o Decoupling	78.50	58.20	42.30	15.50	62.10	51.40	65.20	63.10
	w/o Ortho	84.10	70.50	57.20	10.90	75.30	66.10	77.80	76.20
	w/o Glyph Consistency	82.30	68.40	55.10	11.80	76.50	67.40	79.10	78.00
	w/o Deg Cls	83.50	64.20	48.60	13.20	58.40	46.20	-	-

Table 4. Ablation: Representation Decoupling

Lang.	Models	Character Recognition & Restoration				Sequence-Level		Clear Chars
Lang.	Models	$\mathrm{Acc}_{\mathrm{char}}$	$\mathrm{Acc}_{\mathrm{rest}}$	$\mathrm{Acc}_{\mathrm{sev}}$	$\mathrm{CER}\downarrow$	$\mathrm{LEM}$	$\mathrm{SeqAcc}$	$\mathrm{Acc}_{\mathrm{clear-char}}$
Tibetan	Full DDVR-Net	86.47	74.29	61.54	9.83	45.62	36.75	93.50
	Uniform Context for All	84.10	74.50	61.80	11.50	42.10	33.50	89.20
	w/o Visibility Head	83.20	68.50	55.40	12.30	40.50	31.80	91.00
	Hard Switch instead of Soft Gate	85.10	71.80	58.90	10.60	43.80	34.90	92.80
	Local-only	75.40	35.20	12.50	22.40	25.60	18.50	93.80

Table 5. Effect of Visibility-Guided Selective Restoration

Lang.	Training Paradigm	Character Recognition & Restoration				Sequence-Level
Lang.	Training Paradigm	$\mathrm{Acc}_{\mathrm{char}}$	$\mathrm{Acc}_{\mathrm{rest}}$	$\mathrm{Acc}_{\mathrm{sev}}$	$\mathrm{CER}\downarrow$	$\mathrm{SeqAcc}$
Tibetan	Full Curriculum (Auto $\rightarrow$ Semi $\rightarrow$ Manual)	86.47	74.29	61.54	9.83	36.75
	Mixed Training (Random Shuffle)	83.15	69.80	56.20	12.10	32.40
	Reverse Curriculum (Manual $\rightarrow$ Semi $\rightarrow$ Auto)	80.40	65.30	52.10	14.50	28.50
	w/o Manual Fine-tune (Auto $\rightarrow$ Semi)	81.20	66.80	54.50	13.80	29.80
	Manual Only	76.50	58.40	41.20	18.20	22.10

Table 6. Effect of Progressive Curriculum over Heterogeneous Data Sources (Evaluated on Manual Real-World Test Set)

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Total Params.	Trainable Params.	Latency (ms/page)	Peak Memory (GB)	Training Cost (GPU-hours)
Local-only	407M	407M	286.4	7.8	4,096
Uniform Context	913M	415M	612.7	12.6	5,888
DDVR-Net	921M	423M	438.5	10.4	6,144

Appendix

Table 7. Efficiency and computational cost. Latency and memory are measured on one NVIDIA A100 GPU with batch size 1. Training cost is reported as the number of GPUs multiplied by wall-clock training time.

Lang.	$\mathbf{Acc_{char}}$	$\mathbf{Acc_{rest}}$	$\mathbf{Acc_{sev}}$	$\mathbf{CER}\downarrow$	$\mathbf{SeqAcc}$
Tibetan	86.47	74.29	61.54	9.83	36.75
Mongolian	84.35	71.18	58.42	11.23	32.49
Uyghur	85.34	72.87	59.61	10.45	34.29

Appendix

Table 8. Performance of the full model on the manual real-world test subset. Higher is better except CER.

Lang.	$\mathbf{Acc_{char}}$	$\mathbf{Acc_{rest}}$	$\mathbf{Acc_{sev}}$	$\mathbf{CER}\downarrow$	$\mathbf{SeqAcc}$
Tibetan	89.12	78.96	67.84	8.21	42.63
Mongolian	87.45	75.83	64.29	9.56	38.47
Uyghur	88.31	77.14	65.72	8.88	40.15

Appendix

Table 9. Performance of the full model on the semi-automatic test subset. Higher is better except CER.

Lang.	$\mathbf{Acc_{char}}$	$\mathbf{Acc_{rest}}$	$\mathbf{Acc_{sev}}$	$\mathbf{CER}\downarrow$	$\mathbf{SeqAcc}$
Tibetan	91.36	82.74	72.18	6.94	47.58
Mongolian	89.84	79.61	68.43	8.07	43.26
Uyghur	90.52	80.88	69.95	7.42	45.11

Appendix

Table 10. Performance of the full model on the fully automatic test subset. Higher is better except CER.

Lang.	Training Paradigm	Character Recognition & Restoration				Sequence-Level
Lang.	Training Paradigm	$\mathrm{Acc}_{\mathrm{char}}$	$\mathrm{Acc}_{\mathrm{rest}}$	$\mathrm{Acc}_{\mathrm{sev}}$	$\mathrm{CER}\downarrow$	$\mathrm{SeqAcc}$
Tibetan	Full DDVR-Net	86.47	74.29	61.54	9.83	36.75
	Confidence-based Correction	85.10	73.60	62.10	10.90	32.60
	Post-hoc Correction	84.80	72.90	59.80	11.20	31.80
	Uniform Context	84.20	72.80	60.90	11.70	29.40
	Local-only	83.90	68.90	54.60	12.80	30.70

Appendix

Table 11. Fair comparison under the same backbone and training data across different languages. Higher is better except CER.

Lang.	Models	Character Recognition & Restoration				Visibility Estimation		Degradation-Type Classification
Lang.	Models	$\mathrm{Acc}_{\mathrm{char}}$	$\mathrm{Acc}_{\mathrm{rest}}$	$\mathrm{Acc}_{\mathrm{sev}}$	$\mathrm{CER}\downarrow$	$\mathrm{Acc}_{\mathrm{partial}}$	$\mathrm{Acc}_{\mathrm{severe}}$	$\mathrm{Micro\mbox{-}F1}$	$\mathrm{Macro\mbox{-}F1}$
Tibetan	Full DDVR-Net	86.47	74.29	61.54	9.83	78.46	69.18	81.24	79.67
	w/o Decoupling	78.50	58.20	42.30	15.50	62.10	51.40	65.20	63.10
	w/o Ortho	84.10	70.50	57.20	10.90	75.30	66.10	77.80	76.20
	w/o Glyph Consistency	82.30	68.40	55.10	11.80	76.50	67.40	79.10	78.00
	w/o Deg Cls	83.50	64.20	48.60	13.20	58.40	46.20	-	-

Appendix

Table 12. Effect of Degradation-Aware Representation Decoupling

Lang.	Models	Character Recognition & Restoration				Sequence-Level		Clear Chars
Lang.	Models	$\mathrm{Acc}_{\mathrm{char}}$	$\mathrm{Acc}_{\mathrm{rest}}$	$\mathrm{Acc}_{\mathrm{sev}}$	$\mathrm{CER}\downarrow$	$\mathrm{LEM}$	$\mathrm{SeqAcc}$	$\mathrm{Acc_{clear-char}}$
Tibetan	Full DDVR-Net	86.47	74.29	61.54	9.83	45.62	36.75	93.50
	Uniform Context for All	84.10	74.50	61.80	11.50	42.10	33.50	89.20
	w/o Visibility Head	83.20	68.50	55.40	12.30	40.50	31.80	91.00
	Hard Switch instead of Soft Gate	85.10	71.80	58.90	10.60	43.80	34.90	92.80
	Local-only	75.40	35.20	12.50	22.40	25.60	18.50	93.80

Appendix

Table 13. Effect of Visibility-Guided Selective Restoration

Panel of Figure 4 — Figure 4. Detailed analysis of DDVR-Net components. (a) Stratifying performance by visibility explicitly demonstrates the trade-off resolved by our visibility-guided selective restoration: it preserves high accuracy on Clear characters while fully recovering Severe characters, and optimally fuses signals for Partial characters. (b) Progressive integration of heterogeneous datasets continuously improves performance, especially on severe degradation.

Lang.	Training Paradigm	Character Recognition & Restoration				Sequence-Level
Lang.	Training Paradigm	$\mathrm{Acc}_{\mathrm{char}}$	$\mathrm{Acc}_{\mathrm{rest}}$	$\mathrm{Acc}_{\mathrm{sev}}$	$\mathrm{CER}\downarrow$	$\mathrm{SeqAcc}$
Tibetan	Full Curriculum (Auto $\rightarrow$ Semi $\rightarrow$ Manual)	86.47	74.29	61.54	9.83	36.75
	Mixed Training (Random Shuffle)	83.15	69.80	56.20	12.10	32.40
	Reverse Curriculum (Manual $\rightarrow$ Semi $\rightarrow$ Auto)	80.40	65.30	52.10	14.50	28.50
	w/o Manual Fine-tune (Auto $\rightarrow$ Semi)	81.20	66.80	54.50	13.80	29.80
	Manual Only	76.50	58.40	41.20	18.20	22.10

Appendix

Table 14. Effect of Progressive Curriculum over Heterogeneous Data Sources (Evaluated on Manual Real-World Test Set)

Figure 5. Qualitative comparison on a mixed-visibility text line. To intuitively illustrate error locations, recognition errors and hallucinations in lightly and moderately damaged regions are denoted by ‘ xx ’ and ‘ yy ’, respectively, compared to the Ground-Truth. Our DDVR-Net successfully handles varying degradations.

Explore similar work

TongGuOCR: A Layout-Aware and Token-Augmented OCR MLLM for Chinese Historical Documents
Aug 8, 2026Zhongheng Zhou, Yi Sun, Huiguo He +6Multimodal Large Language Models Optical Character Recognition
School of Electronic and Information Engineering, South China University of Technology, Guangzhou 510641, China · Huawei Technologies Co., Ltd., Dongguan 511700, China
DocRevive: A Unified Pipeline for Document Text Restoration
Apr 11, 2026Kunal Purkayastha, Ayan Banerjee, Josep Llados +1Document Image Understanding
Computer Vision Center · Indian Statistical Institute, Kolkata
Beyond Pixel Reconstruction: Retrieval-Guided Glyph-Aware Restoration for Low-Resource Manchu Historical Documents
Sep 29, 2026Ting Huang, Dongdong Wang, Mingqiu Liang +1Image Restoration Historical Document Processing
School of Computer Science and Technology, Beijing Jiaotong University, Beijing, China · College of Design, Construction, and Planning, University of Florida, Gainesville, FL, USA

Aug 8, 2026J/K move · Enter open · S save

TongGuOCR: A Layout-Aware and Token-Augmented OCR MLLM for Chinese Historical Documents

Zhongheng Zhou, Yi Sun, Huiguo He, Yuyi Zhang+5

School of Electronic and Information Engineering, South China University of Technology, Guangzhou 510641, China · Huawei Technologies Co., Ltd., Dongguan 511700, China

Multimodal Large Language Models Optical Character Recognition Historical Document Processing

Open paper AskPDF arXiv

Chinese historical documents preserve valuable cultural heritage, but many collections remain accessible only as scanned page images, preventing full-text retrieval, collation, and computational analysis. Optical character recognition (OCR) can bridge this gap, but accurate transcription remains challenging because historical documents often contain complex layouts, rare characters, and nontrivial reading orders. We propose TongGuOCR, a layout-aware and token-augmented multimodal large language model (MLLM) for OCR of Chinese historical documents. First, a Layout-Aware Preprocessing module constructs and refines locally coherent recognition blocks to preserve local context while reducing interference across regions. Second, a Token-Augmented Recognition module augments the transcription target at two complementary levels: character-level vocabulary expansion gives each rare glyph a direct one-token representation and shortens its decoding path, while line-to-line transition modeling injects discrete spatial displacement tokens that guide the decoder along complex reading paths without requiring precise coordinates. Experiments on two Chinese historical document OCR benchmarks show that TongGuOCR outperforms representative traditional task-specific OCR models, general-purpose MLLMs, and OCR-oriented MLLMs. On the more challenging M5HisDoc benchmark, TongGuOCR achieves 93.76 AR and reduces NED from 10.43 to 6.15 and RO-ED from 7.53 to 3.49 relative to the best competing score for each metric. An online demo is available at https://jzzh2004.github.io/TongGuOCR.

Abstract

Explore similar work

TongGuOCR: A Layout-Aware and Token-Augmented OCR MLLM for Chinese Historical Documents

DocRevive: A Unified Pipeline for Document Text Restoration

Beyond Pixel Reconstruction: Retrieval-Guided Glyph-Aware Restoration for Low-Resource Manchu Historical Documents