cs.CVNov 6, 2026

GlobalPuzzle: Globally Informed Genetic Search for Eroded Jigsaw Puzzle Reassembly

Authors: Lunyan Wang, Chuanzhi Xu, Ziyuan Tao, Weidong Cai

Organizations: The University of Sydney · Macquarie University

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Jigsaw puzzle reassembly remains a highly challenging task, particularly when piece boundaries are irregularly eroded to simulate damage to cultural artifacts. On the GAP benchmark, existing methods achieve a perfect assembly accuracy of only 0.3% on 5×55 \times 5 puzzles, whereas humans can solve jigsaw puzzles with remarkably high accuracy. Humans typically approach puzzle solving by identifying potentially adjacent pieces, evaluating the current assembly, and iteratively performing complex operations such as adding pieces, replacing incorrectly placed pieces, and moving entire connected groups of pieces. However, existing methods face a fundamental limitation: approaches that leverage global visual information typically predict piece arrangements directly, making it difficult to explicitly perform such complex iterative operations, while search based methods can rearrange pieces but primarily rely on pairwise boundary compatibility, lacking global visual information to evaluate and guide the search process. To bridge this gap, we propose GlobalPuzzle, a jigsaw puzzle reassembly framework that integrates global canvas level evaluation, action guidance, and genetic search. By incorporating global visual information into the evaluation and optimization of candidate assemblies, GlobalPuzzle aims to resolve ambiguous adjacency relationships that are difficult to determine from irregularly eroded piece boundaries alone, enabling flexible, globally informed reconstruction.

Figures & tables

Figure 1: Fragment reconstruction: the top left shows randomly shuffled irregular pieces, the top right shows the image reconstructed using LaMa after correct assembly, the bottom left illustrates position query generation, and the bottom right shows query-based piece retrieval in GlobalPuzzle.
Figure 2: The proposed GlobalPuzzle framework. Offspring inherit common absolute placements (Absolute) and shared relative fragment relationships (Relative) from parent assemblies. The Policy Network generates position queries, which are matched with fragment features from the Piece Encoder to complete the assembly. The Evaluation Network then scores the complete canvas to guide genetic search.
Figure 3: The specific structure of the Policy Network and the Evaluation Network.
 GAP-3GAP-5
MethodPA ↑\uparrowAA ↑\uparrowPA ↑\uparrowAA ↑\uparrow
Classical
Greedy [ 20 ] (CVPR’11)0.011.60.04.1
GA [ 24 ] (CVPR’13)0.011.10.011.1
Deep Learning
JigsawGAN [ 11 ] (TIP’22)4.645.30.018.0
Table 1: Comparison on GAP-3 ( 3×33\times 3 ) and GAP-5 ( 5×55\times 5 ). PA and AA are reported as percentages.
Figure 4: Visual comparison of the restoration for each method. Codex and Claude performed puzzle reconstruction through their respective web interfaces, while PuzzleFlow [ 23 ] was evaluated using its officially released pretrained weights.
 GAP-3 (%)GAP-5 (%)
MethodPA ↑\uparrowAA ↑\uparrowPA ↑\uparrowAA ↑\uparrow
Random Evaluation Network1.925.20.09.6
Random Policy Network65.173.40.016.9
No Crossover60.075.410.442.5
No Swapping63.576.211.342.7
No Crossover and Swapping54.570.70.19.3
Table 2: Ablation study on GAP-3 and GAP-5. PA and AA are reported as percentages.
Figure 5: Evaluation Network: Mean predicted fitness versus the number of misplaced pieces on GAP-3 and GAP-5.
Figure 6: Policy Network: Top-1 accuracy versus the number of correctly placed pieces on GAP-3 and GAP-5.
Figure 7: Genetic search progression from earlier candidates to the final reconstruction. Red outlines mark misplaced fragments in each candidate assembly.
Figure 8: Reconstructing images from different numbers and sizes of fragments in daily life.
Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

 GAP-3GAP-5
SeedPAAAPAAA
4269.280.219.548.4
4368.582.819.149.6
202668.081.819.748.0
M ±\pm S68.6±0.668.6\pm 0.681.6±1.381.6\pm 1.319.4±0.319.4\pm 0.348.7±0.848.7\pm 0.8
Appendix
Table S1 : Robustness to random seeds on GAP-3 and GAP-5. Results are reported as percentages.
Table 12
Figure S1 : Additional compare reconstruction
Figure S2 : Additional compare reconstruction
Figure S3 : Additional compare reconstruction
Figure S4 : Additional compare reconstruction
Figure S5 : Additional compare reconstruction
Figure S6 : Additional examples of reconstruction live photos
Figure S7 : Additional examples of reconstruction live photos
Figure S8 : Additional examples of reconstruction live photos
More in this topic

Jigsaw Puzzle Solving

See all 10 papers

Explore similar work

CardsList
May 12, 2026

The Missing GAP: From Solving Square Jigsaw Puzzles to Handling Real World Archaeological Fragments

Jigsaw puzzle solving has been an increasingly popular task in the computer vision research community. Recent works have utilized cutting-edge architectures and computational approaches to reassemble groups of pieces into a coherent image, while achieving increasingly good results on well established datasets. However, most of these approaches share a common, restricting setting: operating solely on strictly square puzzle pieces. In this work, we introduce GAP, a set of novel jigsaw puzzles datasets containing synthetic, heavily eroded pieces of unrestricted shapes, generated by a learned distribution of real-world archaeological fragments. We also introduce PuzzleFlow, a novel ViT and Flow-Matching based framework for jigsaw puzzle solving, capable of handling complex puzzle pieces and demonstrating superior performance on GAP when compared to both classic and recent prominent works in this domain.

Ofir Itzhak Shahar, Gur Elkin, Ohad Ben-Shahar

Stein Faculty of Computer and Information Science Ben-Gurion University of the Negev, Israel

Jigsaw Puzzle Solving
AskPDF
Sep 16, 2026

JigSync: Gauge-Resolved Synchronization for Jigsaw Reassembly under Unknown Piece Orientation

Square jigsaw reassembly requires recovering the spatial arrangement of shuffled fragments from their visual content and pairwise relationships. While recent studies have made substantial progress, existing benchmarks typically assume that all fragments are provided upright, reducing reassembly to a permutation problem. We study the generalized problem in which each fragment may also have gone through an unknown rotation. For this setting we establish a gauge-unobservability theorem: the minimum of the weighted least-squares objective is exactly invariant under a uniform global rotation of arbitrary magnitude, so no residual-based criterion can recover the global orientation. The theorem further identifies how the issue of global orientation can be resolved: an orientation anchor estimated from the content of a single fragment, lying outside its scope, suffices. To address the above, we propose JigSync, which attains 63.8% and 31.8% absolute accuracy (AA) on GAP-3 and GAP-5, respectively, the highest reported on both, while additionally recovering a rotation per piece that neither benchmark requires. We release JigSync, a degradation protocol that sweeps shape, erosion, photometry, grid size, and rotation independently.

Soham Pahari, Antik Aich Roy, Ujjwal Bhattacharya

Indian Statistical Institute, Kolkata

Jigsaw Puzzle Solving
AskPDF
Mar 6, 2026

Solving Jigsaw Puzzles in the Wild: Human-Guided Reconstruction of Cultural Heritage Fragments

Reassembling real-world archaeological artifacts from fragmented pieces poses significant challenges due to erosion, missing regions, irregular shapes, and large-scale ambiguity. Traditional jigsaw puzzle solvers, often designed for clean synthetic scenarios, struggle under these conditions, especially when the number of fragments grows into the thousands, as in the RePAIR benchmark. In this paper, we propose a human-in-the-loop (HIL) puzzle solving framework designed to address the complexity and scale of real-world cultural heritage reconstruction. Our approach integrates an automatic relaxation-labeling solver with interactive human guidance, allowing users to iteratively lock verified placements, correct errors, and guide the system toward semantically and geometrically coherent assemblies. We introduce two complementary interaction strategies, Iterative Anchoring and Continuous Interactive Refinement, which support scalable reconstruction across varying levels of ambiguity and puzzle size. Experiments on several RePAIR groups demonstrate that our hybrid approach substantially outperforms both fully automatic and manual baselines in accuracy and efficiency, offering a practical solution for large-scale expert-in-the-loop artifact reassembly.

Omidreza Safaei, Sinem Aslan, Sebastiano Vascon +3

DAIS, Ca’ Foscari University of Venice, Venice, Italy · Department of Historical Studies, University of Milan, Milan, Italy · Zhejiang Normal University, Jinhua, Zhejiang Province, China

Jigsaw Puzzle SolvingHuman-in-the-Loop AI
AskPDF