cs.CVAug 27, 2026

Ancient-Bench: A Comprehensive Multi-millennial, Multi-medium, and Multi-script Benchmark for Ancient Chinese Artifact Text Recognition

Authors: Hiuyi Cheng, Nuo Xu, Yuyi Zhang, Xuhan Zheng, Wei Pan, Jing Zhang, Dezhi Peng, Minghui Liao, +4 more

Organizations: South China University of Technology · Huawei Technologies Co., Ltd.

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Ancient Chinese artifact text recognition is fundamental to heritage digitization, and benchmarks for ancient texts are essential for evaluating current model capabilities. However, existing benchmarks suffer from ''fragmentation'', manifested in limited temporal coverage, limited medium diversity, and incomplete script types. Therefore, we present Ancient-Bench, a comprehensive benchmark of 2,700 images for ancient Chinese artifact text recognition, featuring three dimensions: Multi-millennial (spanning 3,000 years of character evolution), Multi-medium (covering nine artifact categories), and Multi-script (encompassing seven historical script forms). To enable consistent and fair evaluation across heterogeneous media, we further define three annotation standards tailored to the medium-specific characteristics of ancient texts: symbol standardization, character standardization, and parsing standardization. Extensive experiments on Ancient-Bench covering general Vision-Language Models (VLMs) and OCR-specialist models reveal that ancient Chinese artifact text recognition remains fundamentally unsolved, with persistent challenges in variant characters, specialized symbols, and hallucination. The dataset is available at https://github.com/SCUT-DLVCLab/Ancient_Bench.

Figures & tables

BenchmarkMedium Types
OracleBronzeSlipSilkSealSteleCliffEditionCalligraphy
MTH1000 ( Yang et al., 2018 )       ✓ 
M 5 HisDoc ( Shi et al., 2023 )       ✓ 
HisDoc1B ( Shi et al., 2025 )       ✓ 
OBI-Bench ( Chen et al., 2025 )✓        
DeepJianDu ( Liu et al., 2025 )  ✓      
Table 1: Comparison of ancient artifact benchmarks across media types, script forms, and timeline coverage, illustrating the scope of Ancient-Bench.
Figure 1: Overview of the Ancient-Bench dataset. Ancient-Bench comprises 2,700 meticulously annotated images, encompassing heterogeneous ancient artifact media, mainstream Chinese scripts, and over 3,000 years of character evolution. It provides character-level distinctions among paleographic transcriptions, simplified/traditional/rare characters, and variants, and additionally annotates special symbols, reading order, and layout information—establishing a unified standard for recognition across ancient Chinese artifact types.
MediumImagesResolution (px)CharsInstitution
Oracle300600 ×\times 8001-681
Bronze25041 ×\times 80-1024 ×\times 10241-772
Slip30029 ×\times 224-700 ×\times 68201-1073
Silk30046 ×\times 124-1330 ×\times 20671-7842
ß Seal350100 ×\times 150-1333 ×\times 13411-404
Steles30085 ×\times 155-5795 ×\times 167454-8164
Table 2: Dataset Statistics for Each Medium Category
Figure 2: Overview of Ancient-Bench Dataset Statistics.
Model (NED(%) ↑\uparrow / F1(%) ↑\uparrow )OracleBronzeSlipSilkSealSteleCliffEditionsCalligraphyOverall
General VLMs: Closed-Source Models
GPT-5-2025-08-07 Singh et al. (2025)0.79 / 1.333.59 / 4.817.23 / 8.7015.27 / 19.394.25 / 5.6926.92 / 35.7736.36 / 46.1421.42 / 29.6028.15 / 35.7916.00 / 20.80
GPT-4o-2024-08-06 Achiam et al. (2023)0.89 / 2.055.60 / 10.3911.37 / 14.2215.84 / 22.503.60 / 6.4623.72 / 43.0330.41 / 45.8022.98 / 44.4228.63 / 39.1015.90 / 25.33
Gemini-3.1-pro-preview Google (2026a)10.31 / 15.7526.47 / 31.1634.03 / 36.8339.60 / 50.2816.96 / 19.2081.24 / 86.1264.48 / 74.9690.88 / 93.1367.57 / 73.4047.95 / 53.42
Gemini-3.5-flash Google (2026b)6.20 / 10.9817.29 / 21.9629.41 / 32.1133.18 / 41.6213.69 / 16.0975.96 / 82.8061.90 / 71.8789.11 / 92.0761.91 / 67.9043.18 / 48.60
Claude-Opus-4-7 Anthropic (2026)4.73 / 8.6210.74 / 15.7923.55 / 26.5928.65 / 40.578.38 / 11.0151.10 / 69.3751.89 / 64.2278.34 / 88.3258.74 / 65.3535.12 / 43.31
Table 3: Evaluation results on ancient artifact recognition task. Bold denotes SOTA (State-of-the-Art) performance, and underline‾\underline{underline} indicates second-best performance. Results are reported separately for General VLMs and OCR Specialist Models.
Figure 3: Qualitative visualization of medium-specific errors . Red, Green, and Blue indicate deletions , insertions , and substitutions , respectively.
Figure 4: Qualitative visualization of hallucinations in general VLMs on the calligraphy subset. Red, Green, and Blue indicate deletions , insertions , and substitutions , respectively.
Appendix figures & tables21 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Oracle-Bone Example Annotation Workflow
Figure 9
Figure 10
Figure 11
Figure 12
Figure 13
Figure 14
Figure 15
Figure 16
Figure 17
Figure 18
Figure 19
Figure 20
Figure 21
Figure 22
Figure 23
Figure 24
Figure 25
Figure 26
Figure 27
Figure 28

Explore similar work

CardsList

  1. Chronicles-OCR: A Cross-Temporal Perception Benchmark for the Evolutionary Trajectory of Chinese Characters

    May 12, 2026Gengluo Li, Shangpin Peng, Xingyu Wan +16VLM EvaluationVLM Robustness

    Institute of Information Engineering, Chinese Academy of Sciences · Tencent · Anyang Normal University +2

  2. JieZi: A Large-Scale Expert-Audited Dataset and Benchmark for Ancient Chinese Character Exegesis

    Aug 12, 2026Ran Li, Huiguo He, Jiahuan Cao +3VLM EvaluationVLM Reasoning

    South China University of Technology Guangzhou, China

  3. Enhancing Multimodal Large Language Models for Ancient Chinese Character Evolution Analysis via Glyph-Driven Fine-Tuning

    Apr 13, 2026Rui Song, Lida Shi, Ruihua Qi +2Multimodal Large Language ModelsLLM Fine-Tuning

    College of Computer Science and Technology, Jilin University, China · Key Laboratory of Ancient Chinese Script, Culture Relics and Artificial Intelligence, Jilin University, China · School of Artificial Intelligence, Jilin University, China +1