A Multi-Stage Framework for Kuzushiji Character Recognition in Japanese Historical Documents

Rui-Yang Ju; Kohei Yamashita; Hirotaka Kameko; Shinsuke Mori

cs.CVFeb 22, 2026

A Multi-Stage Framework for Kuzushiji Character Recognition in Japanese Historical Documents

Authors: Rui-Yang Ju, Kohei Yamashita, Hirotaka Kameko, Shinsuke Mori

Organizations: Graduate School of Informatics, Kyoto University, Kyoto, Japan · Academic Center for Computing and Media Studies, Kyoto University, Kyoto, Japan

Abstract

Kuzushiji was a widely used cursive writing system in pre-modern Japan. Due to simplification and glyph variation, most modern Japanese readers cannot read Kuzushiji characters. Consequently, recent studies have developed optical character recognition (OCR) systems for Kuzushiji. Despite recent progress, Kuzushiji character recognition (KCR) in Japanese historical documents remains challenging because of seal-character overlap and complex layouts, which interfere with character recognition and hinder accurate reconstruction of the reading order. To address these challenges, we propose a multi-stage KCR framework comprising character detection, cropping, classification, ordering, and large language model (LLM)-based post-OCR correction. Specifically, we employ a synthetic data augmentation strategy to improve character detection robustness against seal interference and introduce an adaptive column clustering algorithm to reconstruct the reading order. Finally, we leverage the contextual capabilities of the LLM to correct OCR errors. In addition, we correct annotation omissions, reconstruct the benchmark dataset, and introduce a synthetic test set with simulated seal interference and an out-of-domain (OOD) test set for evaluation. Compared with the conventional character-level OCR baseline, our framework achieves relative CER reductions of 43.48%, 46.02%, and 39.11% on the real, synthetic, and OOD test sets, respectively.

Figures & tables

Figure 1: Overview of the proposed multi-stage KCR framework for Japanese historical documents, comprising five stages: character detection, cropping, classification, ordering, and LLM-based post-OCR correction.

Set	Real Pages		Synth. Pages		Total Pages
Set	#Pages	S/P	#Pages	S/P	Total Pages
Train	800	$\leq 2$	800	$\approx$ 10	1,600
Validation	100	$\leq 2$	100	$\approx$ 10	200
Real Test	100	$\leq 2$	–	–	100
Synth. Test	–	–	100	$\approx$ 10	100
OOD Test	1,090	$\leq 2$	–	–	1,090

Table 1: Statistics of the dataset splits and the proposed SDA strategy, where S/P denotes seals per page.

Figure 2: Effect of confidence thresholding on detection.

Figure 3: Examples of character ordering with AdaCC, with green annotations indicating the AdaCC ordering. The two left examples show correct ordering in regular layouts, whereas the right example shows an ordering in a complex layout, with the correct order shown in blue.

Method	Model Complexity		Real Test Set				Synthetic Test Set
Method	Params	FLOPs	P	R	$\textbf{AP}_{\textbf{50}}$	$\textbf{AP}_{\textbf{50:95}}$	P	R	$\textbf{AP}_{\textbf{50}}$	$\textbf{AP}_{\textbf{50:95}}$
RT-DETR-R50	41.94M	125.6G	93.6	90.3	94.3	70.1	91.1	85.0	90.8	65.2
+ Data Duplication	41.94M	125.6G	95.8	92.2	95.1	69.5	93.7	87.0	92.1	64.8
+ SDA (Ours)	41.94M	125.6G	96.3	93.1	95.6	70.5	95.6	92.2	95.0	68.4
YOLOv9-C	25.32M	102.3G	97.8	93.3	96.5	82.6	95.8	88.0	93.8	77.8
+ Data Duplication	25.32M	102.3G	97.6	93.5	96.5	82.8	95.6	88.4	94.0	78.1

Table 2: Quantitative comparison of character detectors under three data configurations on both test sets. Within each detector group, the best result is shown in bold , and the best overall result across all detectors is underlined .

Test Set	Top-1 Acc.	Top-5 Acc.
Real	97.40%	99.38%
Synth.	93.93%	97.48%

Table 3: Character classification performance of our framework on the Real and Synthetic Test Sets.

Model	Real Test Set	Synth. Test Set
GPT-5.4	$14.02\pm 0.18$	$18.56\pm 0.03$
GPT-5.5	$11.22\pm 0.03$	$14.80\pm 0.05$
GPT-5.6-Sol	$14.34\pm 0.02$	$18.53\pm 0.42$
GPT-6-Astra	$11.55\pm 0.03$	$14.83\pm 0.09$

Table 4: Post-OCR CER (%) of different LLMs over three runs, reported as mean

\pm

standard deviation.

Method	Component			Performance (CER)
Method	SDA	AdaCC	Post-OCR	Real	Synth.	OOD
Our Framework				19.85	27.42	20.66
	✓			19.30	22.78	19.74
		✓		12.01	20.26	14.28
	✓	✓		11.59	15.29	13.42
	✓	✓	✓	11.22	14.80	12.58

Table 5: Ablation study of the proposed components and comparison with existing systems in terms of CER (%).

Figure 4: Visualization of the input images (top) and recognition results overlaid on the corresponding images (bottom). The font size is uniformly set to 64 pixels.

Figure 5: Qualitative comparison of images from the Synthetic Test Set (top) and their restored images (bottom).

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

$\boldsymbol{\tau_{r}}$	$\boldsymbol{(\tau_{rg},\,\tau_{rb})}$	Validation Set		Test Set
$\boldsymbol{\tau_{r}}$	$\boldsymbol{(\tau_{rg},\,\tau_{rb})}$	PSNR	SSIM	PSNR	SSIM
–	–	29.15	0.9655	28.71	0.9639
80	(1.2, 1.2)	29.76	0.9470	29.61	0.9465
80	(1.3, 1.3)	33.64	0.9736	33.73	0.9731
80	(1.4, 1.4)	33.87	0.9756	33.77	0.9745
80	(1.5, 1.5)	31.97	0.9717	31.68	0.9706

Appendix

Table 6: Parameter sensitivity analysis of

\tau_{r}

and

(\tau_{rg},\tau_{rb})

for restoration on the validation and test sets.

Method	Real Test Set	Synth. Test Set
w/o Restoration	11.59	15.29
w/ Restoration	11.98	13.67

Appendix

Table 7: Effect of document restoration on KCR before post-OCR correction, measured by CER (%).

Index	NIJL ID	Book Title	Pages	Chars.	Chars/Page
1	100241706	Usonarubeshi (虚南留別志)	67	8,565	127.8
2	100249376	Gozenkashi Hidenshō (御前菓子秘伝抄)	104	11,841	113.9
3	100249416	Mochigashi Sokuseki Teseishū (餅菓子即席／手製集)	58	7,967	137.4
4	100249476	Meshi Hyakuchinden (飯百珍伝)	46	7,842	170.5
5	200006663	Jiguchi (ぢぐち)	8	121	15.1
6	200015843	Nippon Eitaigura (日本永代蔵)	180	50,251	279.2

Appendix

Table 8: Statistics of the benchmark dataset reconstructed from 13 Japanese historical books. “Pages” denotes the number of annotated page images, and “Chars/Page” denotes the average number of characters per page.

Index	NIJL ID	Book Title	Pages	Chars.	Chars/Page
1	100249371	Teisa Hiroku (鼎左秘録)	83	9,580	115.4
2	100249537	Ryōri Chinmishū (料理珍味集)	159	12,358	77.7
3	200003076	Kōshoku Ichidai Otoko (好色一代男)	282	63,959	226.8
4	200010454	Genji Monogatari (源氏物語)	71	11,566	162.9
5	200014740	Ugetsu Monogatari (雨月物語)	154	44,832	291.1
6	200015779	Ukiyoburo (浮世風呂)	268	60,381	225.3

Appendix

Table 9: Statistics of the OOD Test Set reconstructed from seven book-disjoint Japanese historical books. “Pages” denotes the number of annotated page images, and “Chars/Page” denotes the average number of characters per page.

Figure 6: Examples of annotation corrections. Green bounding boxes indicate annotations originally included in the dataset, while red bounding boxes indicate missing character annotations added in this work.

Figure 7: Examples of real and synthetic seal interference. The top row shows pages with naturally occurring seals, while the bottom row shows pages from the Synthetic Test Set with synthetically overlaid seals.

Explore similar work

TongGuOCR: A Layout-Aware and Token-Augmented OCR MLLM for Chinese Historical Documents
Aug 8, 2026Zhongheng Zhou, Yi Sun, Huiguo He +6Multimodal Large Language Models Optical Character Recognition
School of Electronic and Information Engineering, South China University of Technology, Guangzhou 510641, China · Huawei Technologies Co., Ltd., Dongguan 511700, China
DDVR-Net: Degradation-Aware Decoupling and Visibility-Guided Restoration for Historical Document Recognition
Oct 12, 2026Jiu Sha, Yu Weng, Mengxiao ZhuOptical Character Recognition Historical Document Processing
Institute of National Security, Minzu University of China, Beijing, China · School of Information Engineering, Minzu University of China, Beijing, China · School of Artificial Intelligence and Computer Science, North China University of Technology, Beijing, China
Koshur Pixel: a large-scale synthetic ocr dataset for kashmiri
Jun 22, 2026Haq Nawaz Malik, Faizan Iqbal, Nahfid NissarSynthetic Data Generation Low-Resource Language Processing

Aug 8, 2026J/K move · Enter open · S save

TongGuOCR: A Layout-Aware and Token-Augmented OCR MLLM for Chinese Historical Documents

Zhongheng Zhou, Yi Sun, Huiguo He, Yuyi Zhang+5

School of Electronic and Information Engineering, South China University of Technology, Guangzhou 510641, China · Huawei Technologies Co., Ltd., Dongguan 511700, China

Multimodal Large Language Models Optical Character Recognition Historical Document Processing

Open paper AskPDF arXiv

Chinese historical documents preserve valuable cultural heritage, but many collections remain accessible only as scanned page images, preventing full-text retrieval, collation, and computational analysis. Optical character recognition (OCR) can bridge this gap, but accurate transcription remains challenging because historical documents often contain complex layouts, rare characters, and nontrivial reading orders. We propose TongGuOCR, a layout-aware and token-augmented multimodal large language model (MLLM) for OCR of Chinese historical documents. First, a Layout-Aware Preprocessing module constructs and refines locally coherent recognition blocks to preserve local context while reducing interference across regions. Second, a Token-Augmented Recognition module augments the transcription target at two complementary levels: character-level vocabulary expansion gives each rare glyph a direct one-token representation and shortens its decoding path, while line-to-line transition modeling injects discrete spatial displacement tokens that guide the decoder along complex reading paths without requiring precise coordinates. Experiments on two Chinese historical document OCR benchmarks show that TongGuOCR outperforms representative traditional task-specific OCR models, general-purpose MLLMs, and OCR-oriented MLLMs. On the more challenging M5HisDoc benchmark, TongGuOCR achieves 93.76 AR and reduces NED from 10.43 to 6.15 and RO-ED from 7.53 to 3.49 relative to the best competing score for each metric. An online demo is available at https://jzzh2004.github.io/TongGuOCR.

Abstract

Explore similar work

TongGuOCR: A Layout-Aware and Token-Augmented OCR MLLM for Chinese Historical Documents

DDVR-Net: Degradation-Aware Decoupling and Visibility-Guided Restoration for Historical Document Recognition

Koshur Pixel: a large-scale synthetic ocr dataset for kashmiri