cs.CVNov 3, 2026
SaveTEXMET: A Deep Learning Benchmark for Archaeological Textile Reconstruction
Organizations: Norwegian University of Science and Technology (NTNU), Department of Computer Science, The Norwegian Colour and Visual Computing Laboratory (Colourlab), Gjøvik, Norway
Abstract
The reconstruction of fragmented historical textiles presents a significant challenge in computational archaeology. The Oseberg ship burial, dating to 834 CE, yielded hundreds of degraded textile fragments that have resisted systematic computational analysis. While deep learning has revolutionized image-based puzzle solving, progress on archaeological textiles has been limited by the lack of domain-specific datasets and systematic evaluation of modern architectures. To address this gap, we present TEXMET, a curated dataset of 18,644 historical textile images from the Metropolitan Museum of Art's Open Access collection, representing the first textile-specific benchmark for puzzle-solving research. Using TEXMET, we conduct a systematic evaluation of three state-of-the-art puzzle-solving models: JPDVT, FCViT, and DiffAssemble. Our experiments reveal that the transformer-based models pre-trained on standard datasets fail to generalize to textiles, but domain-specific fine-tuning proves highly effective. The graph-based DiffAssemble model, when fine-tuned on TEXMET, achieves 85.89% puzzle accuracy on textile data. Most significantly, applying this fine-tuned model to 45 highly degraded Oseberg fragments yields a performance increase from 26.67% to 64.44% puzzle accuracy. These results establish TEXMET as a new benchmark for textile reconstruction and demonstrate that domain adaptation is essential for applying puzzle-solving methods to imagery of real archaeological materials.
Figures & tables
Figure 1. Examples of textile fragments from the Oseberg ship burial (834 CE). These degraded fragments present significant challenges for computational reconstruction. The fragments do not correspond to their true scale. Images by George Alexis Pantos, Museum of Cultural History, University of Oslo. Photographs of six archaeological textile fragments from the Oseberg ship burial, arranged on a black background. Each fragment is irregular in outline, torn at the edges and heavily faded, with woven figures, animals and geometric borders still partly visible in brown, gray and reddish tones.
| Metric | Count |
|---|---|
| Objects queried | 33,588 |
| Objects with images | 27,373 |
| Primary images downloaded | 27,360 |
| Additional images downloaded | 27,647 |
| Total images | 55,007 |
| Download success rate | 99.95% |
Table 1. TEXMET data collection summary from the MET archive.
Figure 2. UMAP projection of CLIP embeddings for the textile collection, colored by MET curatorial department, as displayed in the FiftyOne curation interface. Natural clusters emerge, enabling systematic identification of outliers and quality issues. Scatter plot of a two-dimensional UMAP projection of CLIP image embeddings on a dark background, where each point is one textile image colored by MET curatorial department. The points form one large dense cluster and one smaller separate cluster with scattered outliers around them. A legend lists fifteen departments, including Asian Art, Costume Institute, Islamic Art, Medieval Art and The Cloisters.
Figure 3. Overview of the final curated TEXMET dataset showing 18,644 high-quality textile images spanning diverse historical periods and cultural traditions. A dense grid of thumbnail images from the curated TEXMET dataset, each labeled with its object title. The thumbnails show tapestries, woven panels, rank badges, embroidered hangings and textile fragments from many periods and cultures, in both color and black and white.
| Property | Value |
|---|---|
| Total images | 18,644 |
| Train / Val / Test split | 14,915 / 1,864 / 1,865 |
| Average dimensions | 1,557 1,631 pixels |
| Average resolution | 3.3 MP |
| Color images (RGB) | 92.0% |
| Black and white images | 8.0% |
Table 2. TEXMET dataset statistics.
| Department | Samples | % |
|---|---|---|
| European Sculpture and Decorative Arts | 8,516 | 45.7 |
| Asian Art | 5,229 | 28.0 |
| Islamic Art | 3,029 | 16.2 |
| Other departments (11) | 1,870 | 10.0 |
| Total | 18,644 | 100.0 |
Table 3. Department distribution in TEXMET.
| Dataset | Domain | Size | Fragmented | License |
|---|---|---|---|---|
| ImageNet | Natural images | 1.28M | No | Research |
| CelebA-HQ | Faces | 30k | No | Research |
| TextileNet | Modern fabrics | 760k | No | Academic |
| RePAIR | Frescoes | 1k+ | Yes | Restricted |
| TEXMET | Historical textiles | 18,644 | Simulated | CC0 |
Table 4. Comparison of TEXMET with related datasets.
Figure 4. JPDVT architecture showing the Vision Transformer backbone integrated with the diffusion process for position prediction. Figure by the authors, created using Canva. Two-part pipeline diagram of JPDVT. On the left, the training path: image patches are shuffled, their position embeddings are noised by forward diffusion, and the patches pass through a linear projection of flattened patches and reverse diffusion transformer layers that output the predicted patch ordering. On the right, the testing path: visible patches, missing patches and Gaussian noise pass through the same transformer layers for T iterations, generating the missing patches and reassembling the complete textile image.
Figure 5. FCViT architecture illustrating the coordinate regression process where fragment positions are directly predicted. Figure by the authors, created using Canva. Flow diagram of FCViT. An input image is divided into a 3 by 3 grid of fragments with ground-truth coordinates, the fragments are shuffled, and the shuffled patches pass through an encoder and a coordinate predictor that outputs a predicted row and column coordinate for each fragment. The predicted coordinates are used to reassemble the image, and a loss term compares predicted coordinates with true coordinates.
Figure 6. DiffAssemble architecture demonstrating the graph-based diffusion approach for 2D puzzle reassembly. Figure by the authors, created using Canva. Diagram of DiffAssemble. Puzzle pieces are embedded by a rotation-equivariant 2D encoder, each piece becomes a graph node combining its feature vector with position and rotation values at diffusion step t, and an attention-based graph neural network denoises the node attributes to step t minus 1. Below, four snapshots show the pieces moving from scattered at t equals T to a fully assembled image at t equals 0, with arrows marking forward diffusion during training and reverse denoising at inference.
| Model | Rotation | Missing Pieces | Multi-size | Approach |
|---|---|---|---|---|
| JPDVT | No | Yes | No | Diffusion |
| FCViT | No | No | No | Regression |
| DiffAssemble | Yes | Yes | Yes | Graph-Diffusion |
Table 5. Comparison of puzzle-solving model capabilities.
| Model | Original Paper | Our Reproduction | ||
|---|---|---|---|---|
| Puzzle | Fragment | Puzzle | Fragment | |
| JPDVT | 68.7 | 83.3 | 67.89 | 80.02 |
| FCViT | 78.9 | 90.6 | 68.78 | 84.21 |
Table 6. Model reproduction results on ImageNet 3 3 puzzles. Puzzle and fragment accuracy (%) compared to original publications.
Figure 7. DiffAssemble reconstruction examples on CelebA-HQ ( Karras et al., 2018 ) across puzzle sizes (6 6 to 12 12). Left: successful reconstructions showing correct piece placement. Right: failure cases revealing common error patterns. Two side-by-side galleries of DiffAssemble reconstructions of CelebA-HQ face images at puzzle sizes 6 by 6, 8 by 8, 10 by 10 and 12 by 12. Each column shows the scrambled pieces, the model's reconstruction and the ground-truth image. On the left, success cases place every piece correctly: 36 of 36, 64 of 64, 100 of 100 and 144 of 144. On the right, failure cases degrade as the puzzle grows: 35 of 36 (97.2 percent), 62 of 64 (96.9 percent), 87 of 100 (87.0 percent) and 111 of 144 (77.1 percent).
| Puzzle Size | Piece Accuracy (%) | Puzzle Accuracy (%) |
|---|---|---|
| 6 6 (36 pieces) | 99.32 | 91.93 |
| 8 8 (64 pieces) | 98.80 | 83.39 |
| 10 10 (100 pieces) | 98.57 | 75.53 |
| 12 12 (144 pieces) | 98.05 | 68.07 |
Table 7. DiffAssemble performance across puzzle sizes on CelebA-HQ.
Figure 8. Baseline performance comparison of JPDVT, FCViT, and DiffAssemble on ImageNet 3 3 puzzles showing puzzle and fragment accuracy. Grouped bar chart of baseline accuracy on ImageNet 3 by 3 puzzles for three models, with paired bars for puzzle accuracy and fragment accuracy. JPDVT reaches 67.89 percent puzzle accuracy and 80.02 percent fragment accuracy, FCViT 68.78 and 84.21 percent, and DiffAssemble 56.61 and 84.03 percent.
| Model | Puzzle Accuracy (%) | Fragment Accuracy (%) |
|---|---|---|
| JPDVT | 32.92 | 55.89 |
| FCViT | 8.30 | 28.25 |
| DiffAssemble | 65.08 | 86.63 |
Table 8. Cross-domain evaluation: ImageNet-trained models tested on TEXMET 3 3 puzzles without fine-tuning.
Figure 9. Cross-dataset evaluation comparing model performance on ImageNet, TEXMET, and Oseberg fragments. DiffAssemble maintains performance while FCViT collapses, demonstrating the domain gap between natural images and historical textiles. Grouped bar chart of puzzle accuracy for three models on three datasets. JPDVT scores 67.89 percent on ImageNet, 32.92 percent on TEXMET and 20.00 percent on Oseberg fragments. FCViT scores 68.78, 8.30 and 24.44 percent. DiffAssemble scores 56.61, 65.08 and 26.67 percent. FCViT's TEXMET bar is by far the shortest, while DiffAssemble is the only model whose textile accuracy exceeds its ImageNet accuracy.
| Model | Puzzle Accuracy (%) |
|---|---|
| JPDVT | 20.00 |
| FCViT | 24.44 |
| DiffAssemble | 26.67 |
Table 9. Baseline performance on Oseberg fragments (3 3 puzzles, 45 fragments). All models trained on ImageNet without textile-specific fine-tuning.
| Evaluation Dataset | Baseline (%) | Fine-tuned (%) | Improvement |
|---|---|---|---|
| TEXMET | 65.08 | 85.89 | +20.81 |
| Oseberg | 26.67 | 64.44 | +37.77 |
Table 10. DiffAssemble performance before and after fine-tuning on TEXMET. The model was fine-tuned for 150 epochs on TEXMET training data.
Figure 10. Summary of DiffAssemble puzzle accuracy across the five experimental stages. Fine-tuning on TEXMET lifts accuracy in both evaluation domains: TEXMET rises from 65.08% to 85.89% and the held-out Oseberg fragments from 26.67% to 64.44%, indicating that the learned representations transfer to degraded archaeological materials. The drop between the third and fourth stages reflects the switch to the much harder Oseberg domain, not a decline in the model. Line chart of DiffAssemble puzzle accuracy across five experimental stages: ImageNet baseline 56.61 percent, TEXMET generalization 65.08 percent, TEXMET fine-tuned 85.89 percent, Oseberg generalization 26.67 percent and Oseberg fine-tuned 64.44 percent. The line climbs to a peak at the third stage, drops sharply at the fourth stage where the evaluation domain changes to Oseberg, then rises again. Two callout boxes note a gain of 20.8 points from fine-tuning on TEXMET and a gain of 37.8 points on Oseberg.
| Dataset | Training | Standard | Inscribed |
|---|---|---|---|
| TEXMET | ImageNet baseline | 65.08 | 25.79 |
| TEXMET | Fine-tuned | 85.89 | 27.08 |
| Oseberg | ImageNet baseline | 26.67 | 27.67 |
| Oseberg | Fine-tuned | 64.44 | 39.81 |
Table 11. Inscribed cropping analysis for DiffAssemble: puzzle accuracy (%) with standard and inscribed crops. Inscribed crops remove shape information from puzzle piece boundaries.
| Condition | JPDVT | FCViT | DiffAssemble |
|---|---|---|---|
| No mask (baseline) | 87.80 | 70.62 | 55.70 |
| Regular mask | 87.30 | 60.83 | 50.50 |
| Irregular mask | 79.10 | 46.15 | 54.70 |
| Regular + inpainting | 86.10 | 65.10 | 58.50 |
| Irregular + inpainting | 84.30 | 60.10 | 57.90 |
Table 12. Impact of masking and inpainting on 3 3 puzzle accuracy (%), evaluated on 1,000 ImageNet validation images per condition.
Figure 11. The masking and inpainting pipeline on ImageNet validation images. Top: regular centered-rectangle mask. Bottom: irregular mask derived from a single Oseberg fragment. Panels left to right: original image, mask, masked image, and ASUKA-inpainted result. The inpainted images restore visual plausibility to the masked regions. Two rows of four panels showing the masking and inpainting pipeline on ImageNet validation images. The top row uses a photograph of a construction crane with a regular centered rectangular mask covering 35.63 percent of the image. The bottom row uses a photograph of a graffiti-covered freight wagon with an irregular blob-shaped mask derived from an Oseberg fragment, covering 35.69 percent. In each row the panels are, from left to right, the original image, the binary mask, the masked image with the removed region shown in black, and the ASUKA-inpainted result, which restores a visually plausible scene.
Figure 12. Inpainting effectiveness: comparison of puzzle accuracy before (masked) and after inpainting for each model under both masking conditions. Two grouped bar charts comparing puzzle accuracy on masked images against inpainted images for each model, using 1,000 images per condition at 3 by 3. Under the regular mask, JPDVT moves from 87.3 to 86.1 percent (down 1.2 points), FCViT from 60.8 to 65.1 percent (up 4.3) and DiffAssemble from 50.5 to 58.5 percent (up 8.0). Under the irregular mask, JPDVT moves from 79.1 to 84.3 percent (up 5.2), FCViT from 46.1 to 60.1 percent (up 14.0) and DiffAssemble from 54.7 to 57.9 percent (up 3.2).
Figure 13. The ten Oseberg fragment images best reordered by an ImageNet-trained baseline model. Each fragment photograph is divided into a 3 3 grid and reassembled in isolation; nine of the ten are restored completely (9/9 patches) and one partially (7/9). Fragments with distinctive global structure survive severe degradation even under the baseline model, within this simplified setting. Grid of ten photographs of Oseberg textile fragments on black backgrounds, each overlaid with a 3 by 3 grid marking how it was split and reassembled. Nine are labeled 9 of 9 patches correct in green and the tenth is labeled 7 of 9 patches correct in orange. The fragments are irregular in outline and heavily degraded, with woven texture and surface loss clearly visible.
| Dataset | Relative drop | ||
|---|---|---|---|
| ImageNet | 67.89 | 56.62 | |
| TEXMET | 32.92 | 9.54 |
Table 13. Grid-size ablation: JPDVT puzzle accuracy (%) as fragment count increases from (9 fragments) to (16 fragments). The configurations use the JPDVT-T variant (patch size 64, 256 256 input) under the same evaluation protocol; TEXMET values are computed on the 1,865-image test split.
| Category | Puzzle Acc. (%) | Fragment Acc. (%) | |
|---|---|---|---|
| Tapestries | 60 | 53.3 | 71.9 |
| Embroidered | 350 | 37.7 | 58.9 |
| Costume | 34 | 35.3 | 62.1 |
| Rugs | 36 | 33.3 | 56.8 |
| Woven | 883 | 29.4 | 53.0 |
| Velvets | 121 | 28.9 | 60.0 |
Table 14. Material-category ablation: ImageNet-trained JPDVT evaluated on TEXMET (the cross-domain setting of Table 8 ), named material categories with test images, MET curatorial classification used as a complexity proxy. The residual Other group is excluded here and reported in the supplementary material.
Figure 14. FCViT 3 3 puzzle-solving results on ImageNet ( Deng et al., 2009 ) . Top: successful reconstruction (9/9 pieces correct). Bottom: failure case showing the duplicate placement problem, where multiple fragments are assigned to the same coordinates (1/9 pieces correct). Two rows of three panels showing FCViT results on ImageNet 3 by 3 puzzles, with the original image, the scrambled version and the reconstruction in each row. In the top row a color photograph of an ambulance is reassembled correctly, with patch accuracy 1.00. In the bottom row a grayscale photograph of a cup on a newspaper is reconstructed with the same fragment repeated across the grid, giving patch accuracy 0.11, which illustrates the duplicate placement failure.