Organizations: School of CS and Engineering, Vellore Institute of Technology, Vellore, India · Technology Innovation Hub, Indian Institute of Technology, India · College of Technology and Engineering, WestCliff University, USA · School of Science & Technology, York St John University, London, UK · Department of Information Security and Communication Technology, Norwegian University of Science and Technology, Norway
Automated recognition of ancient cuneiform script poses a compound signal-degradation problem: the three-dimensional relief of clay tablets creates spatially varying illumination and cast shadows, surface erosion introduces structured noise that overlaps with genuine sign impressions, and severe class imbalance across 141 sign categories undermines classifier reliability. We introduce EpigraphNet, a segmentation-guided transformer pipeline evaluated on the Persepolis Fortification Archive. From 1,239 annotated tablet images, brightness-adaptive morphological preprocessing and zero-shot SAM2-Large segmentation generate clean binary symbol masks, which a fine-tuned Vision Transformer (ViT-B/16) with inverse-frequency class weighting then classifies. EpigraphNet reaches 86.41% top-1 accuracy on a 132-class benchmark, a 17.21 percentage-point gain over the strongest CNN baseline (ResNet-101, 69.20%) and 5.31-12.91% over four modern backbones (DeiT-B/16, Swin-B, ConvNeXt-B, EfficientNet-B4) under identical conditions. The full pipeline runs at approximately 18 ms per sign on an NVIDIA A100 GPU. A lower Spearman correlation between sign frequency and per-class performance indicates more balanced recognition across frequent and rare classes. Implementation is available at: github.com/r11up/sam-guided-vit
Figures & tables
Fig. 1: Overview of recognition approaches for degraded cuneiform symbols. (a) 3D relief creates spatially varying illumination and cast shadows (red) [ 5 ] , surface erosion structured noise overlapping (yellow) [ 26 ] , and poor stroke boundaries (blue) [ 31 ] complicate recognition. (b) CNN-based methods [ 12 ] ,missing the global arrangement of wedges that defines each sign [ 6 ] . (c) Severe class imbalance across 141 sign categories (bar chart) [ 32 ] . (d) EpigraphNet (ours): SAM2-Large [ 24 ] with morphological operations and fine-tuned ViT-B/16 [ 23 ] with inverse-frequency weighting classifies.
Research
Yr.
Task
Method / Key Result
Cuneiform and Elamite
Kriege et al. [ 9 ]
2018
Sign retrieval
Graph-based matching.
Dencker et al. [ 10 ]
2020
Sign detection
Weak supervision using transliteration alignment; mAP evaluation.
Rest et al. [ 11 ]
2022
Sign detection
Illumination augmentation; mAP evaluation.
Bogacz & Mara [ 43 ]
2020
Period classification
Geometric neural network on 3D meshes; per-class sample cap.
Mahmood et al. [ 8 ]
2023
Language identification
DNN: 93.0%; RF: 95.46%; balanced dataset.
TABLE I: Comparison of deep learning methods for cuneiform and ancient script recognition.
Ibi(x,y)={10if Ii(x,y)>Tgotherwise
Algorithm 1 Morphological Image Enhancement for SAM2 Preprocessing
Fig. 2: Morphological enhancement results for tablets with different brightness levels. (a) Processing pipeline applied to a lighter tablet. (b) Processing pipeline applied to a darker tablet.
Fig. 3: SAM2-Large segmentation across illumination groups (Section III-B ). Triplets show the bounding box, binary mask, and extracted symbol. Rows 1–2: success cases (well-preserved/eroded). Row 3: fallback to raw crop due to exceeded centroid tolerance δ on degraded tablets.
Fig. 4: EpigraphNet pipeline: morphological enhancement → SAM2-Large segmentation → mask refinement → symbol extraction → ViT-B/16 classification. Slice Voting : the [CLS] token aggregates evidence from all 196 patch tokens via multi-head self-attention (Section IV-C ).
Fig. 5: SAM2 model architecture with mask tokens integrated with image features via cross-attention to predict masks of cuneiform symbols.
Fig. 6: Per-class precision, recall, and F1 of EpigraphNet, sorted by descending class frequency. Class names follow the OCHRE transliteration index; rare signs ( ≤30 training samples) appear on the right.
Component
Configuration
GPU
NVIDIA A100 80 GB
Framework
PyTorch 2.1 / CUDA 12.1
ViT variant
ViT-B/16, ImageNet-21k pretrained
Optimizer
AdamW ( λ=0.01 )
Learning rate
1×10−4 , cosine annealing
Batch size
32
TABLE II: Experimental Configuration
Method
Top-1 (%)
Top-3 (%)
Top-5 (%)
Params (M)
Time (ms)
Crop-only baselines (no segmentation)
DeepScribe (ResNet-18) [ 12 ]
65.70
83.70
88.80
11.7
3
DeepScribe (ResNet-50) [ 12 ]
68.30
85.10
89.70
25.6
5
DeepScribe (ResNet-101) [ 12 ]
69.20
85.90
90.20
44.5
9
YOLOv7 [ 34 ]
76.84
83.41
87.93
36.9
12
EfficientNet-B4 [ 35 ]
73.50
83.10
87.00
19.3
5
TABLE III: Top- k accuracy on PFA test set: crop-only baselines vs. full SAM2-Large + transformer pipeline.
Architecture
Mean Recall
Prec. ρ
Rec. ρ
DS RN-18 [ 12 ]
0.459 (0.013)
0.503 (0.027)
0.773 (0.040)
DS RN-50 [ 12 ]
0.483 (0.018)
0.509 (0.049)
0.794 (0.042)
DS RN-101 [ 12 ]
0.504 (0.022)
0.519 (0.028)
0.763 (0.039)
ViT-B/16 (ours)
0.623 (0.016)
0.318 (0.031)
0.492 (0.038)
TABLE IV: Spearman ρ Correlation Between Sign Frequency and Per-Class Performance (Lower Indicates More Balanced Recognition Across Frequencies).
Fig. 7: Confusion matrix for the 10 most confused sign pairs. Errors concentrate on phonologically and visually similar pairs (e.g., MA/NA, KU/GU) [ 39 ] .
Learning to read cuneiform tablets is an extremely demanding task; consequently, of the roughly half million excavated tablets, only a small fraction has been analysed by Assyriologists. Computer vision offers a promising avenue for decipherment but requires large, densely annotated datasets. To address this limitation, the largest annotated cuneiform sign dataset to date is used, and a Deformable Detection Transformer (DETR)-based object detection model is evaluated under two class granularities of 173 and 106 classes. The proposed system integrates automatic tablet-side extraction, heuristic line grouping, and n-gram-based textual similarity evaluation to bridge visual sign detection and textual structure, and achieves consistent improvements of up to 28-37% over prior work on COCO-style detection metrics. At inference, the method is applied to 87,668 tablet fragments from the Electronic Babylonian Library (eBL) corpus, producing nearly 2.9 million sign detections. Although the approach operates without linguistic priors and remains sensitive to tablet damage and layout variability, it provides a scalable and interpretable foundation for corpus-wide cuneiform analysis and supports future integration with multimodal and linguistic modelling frameworks.
Wentao Che, Esteban Garcés Arias, Asim Niaz +2
Institute of Assyriology and Hittite Studies, LMU Munich, Germany · Department of Statistics, LMU Munich, Germany · Munich Center for Machine Learning (MCML), Germany
In this paper, we present a network structure for classifying metadata of cuneiform tablets. The problem is of practical importance, as the size of the existing corpus far exceeds the number of experts available to analyze it. But the task is made difficult by the combination of limited annotated datasets and the high-resolution point-cloud representation of each tablet. To address this, we develop a convolution-inspired architecture that gradually down-scales the point cloud while integrating local neighbor information. The final down-scaled point cloud is then processed by computing neighbors in the feature space to include global information. Our method is compared with the state-of-the-art transformer-based network Point-BERT, and consistently obtains the best performance. Source code and data available at github.com/fhagelskjaer/cuneiform3d
Frederik Hagelskjær
SDU Robotics · The Mærsk Mc-Kinney Møller Institute · University of Southern Denmark +1
Stone inscriptions are invaluable sources of historical and linguistic knowledge, yet their automated analysis remains a major challenge due to surface irregularities, erosion, and low visual contrast. Conventional document and handwriting analysis techniques fail to perform well in these scenarios. In this work, we propose character detection as a core strategy for robust inscription analysis. We introduce EpiSAM, a prompt-guided transformer framework for character segmentation in stone inscriptions. Rather than treating characters in isolation, EpiSAM employs a novel neighbor-aware strategy, explicitly predicting adjacent characters alongside the target. These contextual cues resolve boundary ambiguities, improving mask generation and enabling more accurate character segmentation. Furthermore, we expand an existing stone inscription dataset by adding dense polygonal annotations for characters, thereby enabling comprehensive research on Southeast Asian epigraphy. Experimental results show that EpiSAM achieves consistent improvements over existing baselines, while also exhibiting strong zero-shot generalization in challenging epigraphic scenarios.
Arnav Sharma, Pratyush Jena, Amal Joseph +1
Center for Visual Information Technology, International Institute of Information Technology, Hyderabad, Gachibowli, Hyderabad, Telangana 500032, India