Organizations: School of CS and Engineering, Vellore Institute of Technology, Vellore, India · Technology Innovation Hub, Indian Institute of Technology, India · College of Technology and Engineering, WestCliff University, USA · School of Science & Technology, York St John University, London, UK · Department of Information Security and Communication Technology, Norwegian University of Science and Technology, Norway
Automated recognition of ancient cuneiform script poses a compound signal-degradation problem: the three-dimensional relief of clay tablets creates spatially varying illumination and cast shadows, surface erosion introduces structured noise that overlaps with genuine sign impressions, and severe class imbalance across 141 sign categories undermines classifier reliability. We introduce EpigraphNet, a segmentation-guided transformer pipeline evaluated on the Persepolis Fortification Archive. From 1,239 annotated tablet images, brightness-adaptive morphological preprocessing and zero-shot SAM2-Large segmentation generate clean binary symbol masks, which a fine-tuned Vision Transformer (ViT-B/16) with inverse-frequency class weighting then classifies. EpigraphNet reaches 86.41% top-1 accuracy on a 132-class benchmark, a 17.21 percentage-point gain over the strongest CNN baseline (ResNet-101, 69.20%) and 5.31-12.91% over four modern backbones (DeiT-B/16, Swin-B, ConvNeXt-B, EfficientNet-B4) under identical conditions. The full pipeline runs at approximately 18 ms per sign on an NVIDIA A100 GPU. A lower Spearman correlation between sign frequency and per-class performance indicates more balanced recognition across frequent and rare classes. Implementation is available at: github.com/r11up/sam-guided-vit
Figures & tables
Fig. 1: Overview of recognition approaches for degraded cuneiform symbols. (a) 3D relief creates spatially varying illumination and cast shadows (red) [ 5 ] , surface erosion structured noise overlapping (yellow) [ 26 ] , and poor stroke boundaries (blue) [ 31 ] complicate recognition. (b) CNN-based methods [ 12 ] ,missing the global arrangement of wedges that defines each sign [ 6 ] . (c) Severe class imbalance across 141 sign categories (bar chart) [ 32 ] . (d) EpigraphNet (ours): SAM2-Large [ 24 ] with morphological operations and fine-tuned ViT-B/16 [ 23 ] with inverse-frequency weighting classifies.
Research
Yr.
Task
Method / Key Result
Cuneiform and Elamite
Kriege et al. [ 9 ]
2018
Sign retrieval
Graph-based matching.
Dencker et al. [ 10 ]
2020
Sign detection
Weak supervision using transliteration alignment; mAP evaluation.
Rest et al. [ 11 ]
2022
Sign detection
Illumination augmentation; mAP evaluation.
Bogacz & Mara [ 43 ]
2020
Period classification
Geometric neural network on 3D meshes; per-class sample cap.
Mahmood et al. [ 8 ]
2023
Language identification
DNN: 93.0%; RF: 95.46%; balanced dataset.
TABLE I: Comparison of deep learning methods for cuneiform and ancient script recognition.
Ibi(x,y)={10if Ii(x,y)>Tgotherwise
Algorithm 1 Morphological Image Enhancement for SAM2 Preprocessing
Fig. 2: Morphological enhancement results for tablets with different brightness levels. (a) Processing pipeline applied to a lighter tablet. (b) Processing pipeline applied to a darker tablet.
Fig. 3: SAM2-Large segmentation across illumination groups (Section III-B ). Triplets show the bounding box, binary mask, and extracted symbol. Rows 1–2: success cases (well-preserved/eroded). Row 3: fallback to raw crop due to exceeded centroid tolerance δ on degraded tablets.
Fig. 4: EpigraphNet pipeline: morphological enhancement → SAM2-Large segmentation → mask refinement → symbol extraction → ViT-B/16 classification. Slice Voting : the [CLS] token aggregates evidence from all 196 patch tokens via multi-head self-attention (Section IV-C ).
Fig. 5: SAM2 model architecture with mask tokens integrated with image features via cross-attention to predict masks of cuneiform symbols.
Fig. 6: Per-class precision, recall, and F1 of EpigraphNet, sorted by descending class frequency. Class names follow the OCHRE transliteration index; rare signs ( ≤30 training samples) appear on the right.
Component
Configuration
GPU
NVIDIA A100 80 GB
Framework
PyTorch 2.1 / CUDA 12.1
ViT variant
ViT-B/16, ImageNet-21k pretrained
Optimizer
AdamW ( λ=0.01 )
Learning rate
1×10−4 , cosine annealing
Batch size
32
TABLE II: Experimental Configuration
Method
Top-1 (%)
Top-3 (%)
Top-5 (%)
Params (M)
Time (ms)
Crop-only baselines (no segmentation)
DeepScribe (ResNet-18) [ 12 ]
65.70
83.70
88.80
11.7
3
DeepScribe (ResNet-50) [ 12 ]
68.30
85.10
89.70
25.6
5
DeepScribe (ResNet-101) [ 12 ]
69.20
85.90
90.20
44.5
9
YOLOv7 [ 34 ]
76.84
83.41
87.93
36.9
12
EfficientNet-B4 [ 35 ]
73.50
83.10
87.00
19.3
5
TABLE III: Top- k accuracy on PFA test set: crop-only baselines vs. full SAM2-Large + transformer pipeline.
Architecture
Mean Recall
Prec. ρ
Rec. ρ
DS RN-18 [ 12 ]
0.459 (0.013)
0.503 (0.027)
0.773 (0.040)
DS RN-50 [ 12 ]
0.483 (0.018)
0.509 (0.049)
0.794 (0.042)
DS RN-101 [ 12 ]
0.504 (0.022)
0.519 (0.028)
0.763 (0.039)
ViT-B/16 (ours)
0.623 (0.016)
0.318 (0.031)
0.492 (0.038)
TABLE IV: Spearman ρ Correlation Between Sign Frequency and Per-Class Performance (Lower Indicates More Balanced Recognition Across Frequencies).
Fig. 7: Confusion matrix for the 10 most confused sign pairs. Errors concentrate on phonologically and visually similar pairs (e.g., MA/NA, KU/GU) [ 39 ] .
Institute of Assyriology and Hittite Studies, LMU Munich, Germany · Department of Statistics, LMU Munich, Germany · Munich Center for Machine Learning (MCML), Germany