TEXMET: A Deep Learning Benchmark for Archaeological Textile Reconstruction
Authors: Muhammad Hamza Zafar, Davit Gigilashvili, Jon Yngve Hardeberg
Organizations: Norwegian University of Science and Technology (NTNU), Department of Computer Science, The Norwegian Colour and Visual Computing Laboratory (Colourlab), Gjøvik, Norway
The reconstruction of fragmented historical textiles presents a significant challenge in computational archaeology. The Oseberg ship burial, dating to 834 CE, yielded hundreds of degraded textile fragments that have resisted systematic computational analysis. While deep learning has revolutionized image-based puzzle solving, progress on archaeological textiles has been limited by the lack of domain-specific datasets and systematic evaluation of modern architectures. To address this gap, we present TEXMET, a curated dataset of 18,644 historical textile images from the Metropolitan Museum of Art's Open Access collection, representing the first textile-specific benchmark for puzzle-solving research. Using TEXMET, we conduct a systematic evaluation of three state-of-the-art puzzle-solving models: JPDVT, FCViT, and DiffAssemble. Our experiments reveal that the transformer-based models pre-trained on standard datasets fail to generalize to textiles, but domain-specific fine-tuning proves highly effective. The graph-based DiffAssemble model, when fine-tuned on TEXMET, achieves 85.89% puzzle accuracy on textile data. Most significantly, applying this fine-tuned model to 45 highly degraded Oseberg fragments yields a performance increase from 26.67% to 64.44% puzzle accuracy. These results establish TEXMET as a new benchmark for textile reconstruction and demonstrate that domain adaptation is essential for applying puzzle-solving methods to imagery of real archaeological materials.
Figures & tables
Figure 1. Examples of textile fragments from the Oseberg ship burial (834 CE). These degraded fragments present significant challenges for computational reconstruction. The fragments do not correspond to their true scale. Images by George Alexis Pantos, Museum of Cultural History, University of Oslo. Photographs of six archaeological textile fragments from the Oseberg ship burial, arranged on a black background. Each fragment is irregular in outline, torn at the edges and heavily faded, with woven figures, animals and geometric borders still partly visible in brown, gray and reddish tones.
Metric
Count
Objects queried
33,588
Objects with images
27,373
Primary images downloaded
27,360
Additional images downloaded
27,647
Total images
55,007
Download success rate
99.95%
Table 1. TEXMET data collection summary from the MET archive.
Figure 2. UMAP projection of CLIP embeddings for the textile collection, colored by MET curatorial department, as displayed in the FiftyOne curation interface. Natural clusters emerge, enabling systematic identification of outliers and quality issues. Scatter plot of a two-dimensional UMAP projection of CLIP image embeddings on a dark background, where each point is one textile image colored by MET curatorial department. The points form one large dense cluster and one smaller separate cluster with scattered outliers around them. A legend lists fifteen departments, including Asian Art, Costume Institute, Islamic Art, Medieval Art and The Cloisters.
Figure 3. Overview of the final curated TEXMET dataset showing 18,644 high-quality textile images spanning diverse historical periods and cultural traditions. A dense grid of thumbnail images from the curated TEXMET dataset, each labeled with its object title. The thumbnails show tapestries, woven panels, rank badges, embroidered hangings and textile fragments from many periods and cultures, in both color and black and white.
Property
Value
Total images
18,644
Train / Val / Test split
14,915 / 1,864 / 1,865
Average dimensions
1,557 × 1,631 pixels
Average resolution
3.3 MP
Color images (RGB)
92.0%
Black and white images
8.0%
Table 2. TEXMET dataset statistics.
Department
Samples
%
European Sculpture and Decorative Arts
8,516
45.7
Asian Art
5,229
28.0
Islamic Art
3,029
16.2
Other departments (11)
1,870
10.0
Total
18,644
100.0
Table 3. Department distribution in TEXMET.
Dataset
Domain
Size
Fragmented
License
ImageNet
Natural images
1.28M
No
Research
CelebA-HQ
Faces
30k
No
Research
TextileNet
Modern fabrics
760k
No
Academic
RePAIR
Frescoes
1k+
Yes
Restricted
TEXMET
Historical textiles
18,644
Simulated
CC0
Table 4. Comparison of TEXMET with related datasets.
Figure 4. JPDVT architecture showing the Vision Transformer backbone integrated with the diffusion process for position prediction. Figure by the authors, created using Canva. Two-part pipeline diagram of JPDVT. On the left, the training path: image patches are shuffled, their position embeddings are noised by forward diffusion, and the patches pass through a linear projection of flattened patches and reverse diffusion transformer layers that output the predicted patch ordering. On the right, the testing path: visible patches, missing patches and Gaussian noise pass through the same transformer layers for T iterations, generating the missing patches and reassembling the complete textile image.
Figure 5. FCViT architecture illustrating the coordinate regression process where fragment positions are directly predicted. Figure by the authors, created using Canva. Flow diagram of FCViT. An input image is divided into a 3 by 3 grid of fragments with ground-truth coordinates, the fragments are shuffled, and the shuffled patches pass through an encoder and a coordinate predictor that outputs a predicted row and column coordinate for each fragment. The predicted coordinates are used to reassemble the image, and a loss term compares predicted coordinates with true coordinates.
Figure 6. DiffAssemble architecture demonstrating the graph-based diffusion approach for 2D puzzle reassembly. Figure by the authors, created using Canva. Diagram of DiffAssemble. Puzzle pieces are embedded by a rotation-equivariant 2D encoder, each piece becomes a graph node combining its feature vector with position and rotation values at diffusion step t, and an attention-based graph neural network denoises the node attributes to step t minus 1. Below, four snapshots show the pieces moving from scattered at t equals T to a fully assembled image at t equals 0, with arrows marking forward diffusion during training and reverse denoising at inference.
Model
Rotation
Missing Pieces
Multi-size
Approach
JPDVT
No
Yes
No
Diffusion
FCViT
No
No
No
Regression
DiffAssemble
Yes
Yes
Yes
Graph-Diffusion
Table 5. Comparison of puzzle-solving model capabilities.
Model
Original Paper
Our Reproduction
Puzzle
Fragment
Puzzle
Fragment
JPDVT
68.7
83.3
67.89
80.02
FCViT
78.9
90.6
68.78
84.21
Table 6. Model reproduction results on ImageNet 3 × 3 puzzles. Puzzle and fragment accuracy (%) compared to original publications.
Figure 7. DiffAssemble reconstruction examples on CelebA-HQ ( Karras et al., 2018 ) across puzzle sizes (6 × 6 to 12 × 12). Left: successful reconstructions showing correct piece placement. Right: failure cases revealing common error patterns. Two side-by-side galleries of DiffAssemble reconstructions of CelebA-HQ face images at puzzle sizes 6 by 6, 8 by 8, 10 by 10 and 12 by 12. Each column shows the scrambled pieces, the model's reconstruction and the ground-truth image. On the left, success cases place every piece correctly: 36 of 36, 64 of 64, 100 of 100 and 144 of 144. On the right, failure cases degrade as the puzzle grows: 35 of 36 (97.2 percent), 62 of 64 (96.9 percent), 87 of 100 (87.0 percent) and 111 of 144 (77.1 percent).
Puzzle Size
Piece Accuracy (%)
Puzzle Accuracy (%)
6 × 6 (36 pieces)
99.32
91.93
8 × 8 (64 pieces)
98.80
83.39
10 × 10 (100 pieces)
98.57
75.53
12 × 12 (144 pieces)
98.05
68.07
Table 7. DiffAssemble performance across puzzle sizes on CelebA-HQ.
Figure 8. Baseline performance comparison of JPDVT, FCViT, and DiffAssemble on ImageNet 3 × 3 puzzles showing puzzle and fragment accuracy. Grouped bar chart of baseline accuracy on ImageNet 3 by 3 puzzles for three models, with paired bars for puzzle accuracy and fragment accuracy. JPDVT reaches 67.89 percent puzzle accuracy and 80.02 percent fragment accuracy, FCViT 68.78 and 84.21 percent, and DiffAssemble 56.61 and 84.03 percent.
Model
Puzzle Accuracy (%)
Fragment Accuracy (%)
JPDVT
32.92
55.89
FCViT
8.30
28.25
DiffAssemble
65.08
86.63
Table 8. Cross-domain evaluation: ImageNet-trained models tested on TEXMET 3 × 3 puzzles without fine-tuning.
Figure 9. Cross-dataset evaluation comparing model performance on ImageNet, TEXMET, and Oseberg fragments. DiffAssemble maintains performance while FCViT collapses, demonstrating the domain gap between natural images and historical textiles. Grouped bar chart of puzzle accuracy for three models on three datasets. JPDVT scores 67.89 percent on ImageNet, 32.92 percent on TEXMET and 20.00 percent on Oseberg fragments. FCViT scores 68.78, 8.30 and 24.44 percent. DiffAssemble scores 56.61, 65.08 and 26.67 percent. FCViT's TEXMET bar is by far the shortest, while DiffAssemble is the only model whose textile accuracy exceeds its ImageNet accuracy.
Model
Puzzle Accuracy (%)
JPDVT
20.00
FCViT
24.44
DiffAssemble
26.67
Table 9. Baseline performance on Oseberg fragments (3 × 3 puzzles, 45 fragments). All models trained on ImageNet without textile-specific fine-tuning.
Evaluation Dataset
Baseline (%)
Fine-tuned (%)
Improvement
TEXMET
65.08
85.89
+20.81
Oseberg
26.67
64.44
+37.77
Table 10. DiffAssemble performance before and after fine-tuning on TEXMET. The model was fine-tuned for 150 epochs on TEXMET training data.
Figure 10. Summary of DiffAssemble puzzle accuracy across the five experimental stages. Fine-tuning on TEXMET lifts accuracy in both evaluation domains: TEXMET rises from 65.08% to 85.89% and the held-out Oseberg fragments from 26.67% to 64.44%, indicating that the learned representations transfer to degraded archaeological materials. The drop between the third and fourth stages reflects the switch to the much harder Oseberg domain, not a decline in the model. Line chart of DiffAssemble puzzle accuracy across five experimental stages: ImageNet baseline 56.61 percent, TEXMET generalization 65.08 percent, TEXMET fine-tuned 85.89 percent, Oseberg generalization 26.67 percent and Oseberg fine-tuned 64.44 percent. The line climbs to a peak at the third stage, drops sharply at the fourth stage where the evaluation domain changes to Oseberg, then rises again. Two callout boxes note a gain of 20.8 points from fine-tuning on TEXMET and a gain of 37.8 points on Oseberg.
Dataset
Training
Standard
Inscribed
TEXMET
ImageNet baseline
65.08
25.79
TEXMET
Fine-tuned
85.89
27.08
Oseberg
ImageNet baseline
26.67
27.67
Oseberg
Fine-tuned
64.44
39.81
Table 11. Inscribed cropping analysis for DiffAssemble: puzzle accuracy (%) with standard and inscribed crops. Inscribed crops remove shape information from puzzle piece boundaries.
Condition
JPDVT
FCViT
DiffAssemble
No mask (baseline)
87.80
70.62
55.70
Regular mask
87.30
60.83
50.50
Irregular mask
79.10
46.15
54.70
Regular + inpainting
86.10
65.10
58.50
Irregular + inpainting
84.30
60.10
57.90
Table 12. Impact of masking and inpainting on 3 × 3 puzzle accuracy (%), evaluated on 1,000 ImageNet validation images per condition.
Figure 11. The masking and inpainting pipeline on ImageNet validation images. Top: regular centered-rectangle mask. Bottom: irregular mask derived from a single Oseberg fragment. Panels left to right: original image, mask, masked image, and ASUKA-inpainted result. The inpainted images restore visual plausibility to the masked regions. Two rows of four panels showing the masking and inpainting pipeline on ImageNet validation images. The top row uses a photograph of a construction crane with a regular centered rectangular mask covering 35.63 percent of the image. The bottom row uses a photograph of a graffiti-covered freight wagon with an irregular blob-shaped mask derived from an Oseberg fragment, covering 35.69 percent. In each row the panels are, from left to right, the original image, the binary mask, the masked image with the removed region shown in black, and the ASUKA-inpainted result, which restores a visually plausible scene.
Figure 12. Inpainting effectiveness: comparison of puzzle accuracy before (masked) and after inpainting for each model under both masking conditions. Two grouped bar charts comparing puzzle accuracy on masked images against inpainted images for each model, using 1,000 images per condition at 3 by 3. Under the regular mask, JPDVT moves from 87.3 to 86.1 percent (down 1.2 points), FCViT from 60.8 to 65.1 percent (up 4.3) and DiffAssemble from 50.5 to 58.5 percent (up 8.0). Under the irregular mask, JPDVT moves from 79.1 to 84.3 percent (up 5.2), FCViT from 46.1 to 60.1 percent (up 14.0) and DiffAssemble from 54.7 to 57.9 percent (up 3.2).
Figure 13. The ten Oseberg fragment images best reordered by an ImageNet-trained baseline model. Each fragment photograph is divided into a 3 × 3 grid and reassembled in isolation; nine of the ten are restored completely (9/9 patches) and one partially (7/9). Fragments with distinctive global structure survive severe degradation even under the baseline model, within this simplified setting. Grid of ten photographs of Oseberg textile fragments on black backgrounds, each overlaid with a 3 by 3 grid marking how it was split and reassembled. Nine are labeled 9 of 9 patches correct in green and the tenth is labeled 7 of 9 patches correct in orange. The fragments are irregular in outline and heavily degraded, with woven texture and surface loss clearly visible.
Dataset
3×3
4×4
Relative drop
ImageNet
67.89
56.62
−17%
TEXMET
32.92
9.54
−71%
Table 13. Grid-size ablation: JPDVT puzzle accuracy (%) as fragment count increases from 3×3 (9 fragments) to 4×4 (16 fragments). The 4×4 configurations use the JPDVT-T variant (patch size 64, 256 × 256 input) under the same evaluation protocol; TEXMET values are computed on the 1,865-image test split.
Category
n
Puzzle Acc. (%)
Fragment Acc. (%)
Tapestries
60
53.3
71.9
Embroidered
350
37.7
58.9
Costume
34
35.3
62.1
Rugs
36
33.3
56.8
Woven
883
29.4
53.0
Velvets
121
28.9
60.0
Table 14. Material-category ablation: ImageNet-trained JPDVT evaluated on TEXMET 3×3 (the cross-domain setting of Table 8 ), named material categories with n≥20 test images, MET curatorial classification used as a complexity proxy. The residual Other group is excluded here and reported in the supplementary material.
Figure 14. FCViT 3 × 3 puzzle-solving results on ImageNet ( Deng et al., 2009 ) . Top: successful reconstruction (9/9 pieces correct). Bottom: failure case showing the duplicate placement problem, where multiple fragments are assigned to the same coordinates (1/9 pieces correct). Two rows of three panels showing FCViT results on ImageNet 3 by 3 puzzles, with the original image, the scrambled version and the reconstruction in each row. In the top row a color photograph of an ambulance is reassembled correctly, with patch accuracy 1.00. In the bottom row a grayscale photograph of a cup on a newspaper is reconstructed with the same fragment repeated across the grid, giving patch accuracy 0.11, which illustrates the duplicate placement failure.
Jigsaw puzzle solving has been an increasingly popular task in the computer vision research community. Recent works have utilized cutting-edge architectures and computational approaches to reassemble groups of pieces into a coherent image, while achieving increasingly good results on well established datasets. However, most of these approaches share a common, restricting setting: operating solely on strictly square puzzle pieces. In this work, we introduce GAP, a set of novel jigsaw puzzles datasets containing synthetic, heavily eroded pieces of unrestricted shapes, generated by a learned distribution of real-world archaeological fragments. We also introduce PuzzleFlow, a novel ViT and Flow-Matching based framework for jigsaw puzzle solving, capable of handling complex puzzle pieces and demonstrating superior performance on GAP when compared to both classic and recent prominent works in this domain.
Ofir Itzhak Shahar, Gur Elkin, Ohad Ben-Shahar
Stein Faculty of Computer and Information Science Ben-Gurion University of the Negev, Israel
Unlike the clean and pixel-aligned complete pieces of standard jigsaw benchmarks, real-world puzzle fragments lose their boundary regions to erosion, destroying the signal that classical compatibility measures (CMs) rely on. We first re-establish, via oracle experiments, that the CM, not the solver, is the performance bottleneck: with a perfect CM, even a simple greedy solver achieves perfect reconstruction. This motivates STITCH, a self-supervised CM that sidesteps missing boundaries by comparing frozen ViT patch tokens from piece interiors via a lightweight cross-attention head (~249K trainable parameters), trained without manual annotation. Plugged into unchanged compatibility-based solvers, STITCH reaches .992/.954 Abs/Perf on JPwLEG-5 (+.629 Perf over the best prior method), nearly saturating the benchmark. On a new ImageNet 7x7 benchmark, it outperforms all alternatives, including inpaint-then-match pipelines built on modern generative models. Importantly, performance remains robust across erosion levels and generalizes to puzzles with over 40% missing pieces without retraining. Trained on the recent GAP datasets of irregular, heavily eroded fragments, the same CM recipe enables compatibility-based solvers to surpass recent learning-based solvers on GAP-3 and GAP-5. Since this leap in performance is achieved by a pluggable CM module, the takeaway is clear: fix the match, not the solver.
Automatic writer identification systems have progressed remarkably in recent years, yet their deployment in archival paleography remains limited by the scarcity of labeled training data, open scribe sets, and degraded image quality. We present TextileNet, a fully convolutional multi-task network trained exclusively on synthetic data to produce dense pixel-level texture embeddings, which we transfer zeroshot to historical manuscript analysis. As an original contribution to evaluation methodology, we designed a paleographic visual quiz of 80 pair and triplet questions and administered it to a range from lay participants to senior paleographers under strict anonymity, establishing to our knowledge for the first time a human baseline for script-style discrimination on late medieval text. We employ TextileNet embeddings to perform zero-shot retrieval on sub-word granularity for hand and gender identification. Our experimental results help in building the credibility of TextileNet in the paleographic domain, but more than that demonstrate in experimental terms that the question of gender in handwriting needs to be treated with caution.
Anguelos Nicolaou, Antonella Ambrosio, Desiree Di Donato +1
University of Graz, Graz, Austria · Università degli Studi di Napoli Federico II, Naples, Italy