GlobalPuzzle: Globally Informed Genetic Search for Eroded Jigsaw Puzzle Reassembly

Lunyan Wang; Chuanzhi Xu; Ziyuan Tao; Weidong Cai

cs.CVNov 6, 2026

GlobalPuzzle: Globally Informed Genetic Search for Eroded Jigsaw Puzzle Reassembly

Authors: Lunyan Wang, Chuanzhi Xu, Ziyuan Tao, Weidong Cai

Organizations: The University of Sydney · Macquarie University

Abstract

Jigsaw puzzle reassembly remains a highly challenging task, particularly when piece boundaries are irregularly eroded to simulate damage to cultural artifacts. On the GAP benchmark, existing methods achieve a perfect assembly accuracy of only 0.3% on $5 \times 5$ puzzles, whereas humans can solve jigsaw puzzles with remarkably high accuracy. Humans typically approach puzzle solving by identifying potentially adjacent pieces, evaluating the current assembly, and iteratively performing complex operations such as adding pieces, replacing incorrectly placed pieces, and moving entire connected groups of pieces. However, existing methods face a fundamental limitation: approaches that leverage global visual information typically predict piece arrangements directly, making it difficult to explicitly perform such complex iterative operations, while search based methods can rearrange pieces but primarily rely on pairwise boundary compatibility, lacking global visual information to evaluate and guide the search process. To bridge this gap, we propose GlobalPuzzle, a jigsaw puzzle reassembly framework that integrates global canvas level evaluation, action guidance, and genetic search. By incorporating global visual information into the evaluation and optimization of candidate assemblies, GlobalPuzzle aims to resolve ambiguous adjacency relationships that are difficult to determine from irregularly eroded piece boundaries alone, enabling flexible, globally informed reconstruction.

Figures & tables

Figure 1: Fragment reconstruction: the top left shows randomly shuffled irregular pieces, the top right shows the image reconstructed using LaMa after correct assembly, the bottom left illustrates position query generation, and the bottom right shows query-based piece retrieval in GlobalPuzzle.

Figure 2: The proposed GlobalPuzzle framework. Offspring inherit common absolute placements (Absolute) and shared relative fragment relationships (Relative) from parent assemblies. The Policy Network generates position queries, which are matched with fragment features from the Piece Encoder to complete the assembly. The Evaluation Network then scores the complete canvas to guide genetic search.

Figure 3: The specific structure of the Policy Network and the Evaluation Network.

Classical
	GAP-3		GAP-5
Method	PA $\uparrow$	AA $\uparrow$	PA $\uparrow$	AA $\uparrow$
Greedy [ 20 ] (CVPR’11)	0.0	11.6	0.0	4.1
GA [ 24 ] (CVPR’13)	0.0	11.1	0.0	11.1
Deep Learning
JigsawGAN [ 11 ] (TIP’22)	4.6	45.3	0.0	18.0

Table 1: Comparison on GAP-3 (

3\times 3

) and GAP-5 (

5\times 5

). PA and AA are reported as percentages.

Figure 4: Visual comparison of the restoration for each method. Codex and Claude performed puzzle reconstruction through their respective web interfaces, while PuzzleFlow [ 23 ] was evaluated using its officially released pretrained weights.

	GAP-3 (%)		GAP-5 (%)
Method	PA $\uparrow$	AA $\uparrow$	PA $\uparrow$	AA $\uparrow$
Random Evaluation Network	1.9	25.2	0.0	9.6
Random Policy Network	65.1	73.4	0.0	16.9
No Crossover	60.0	75.4	10.4	42.5
No Swapping	63.5	76.2	11.3	42.7
No Crossover and Swapping	54.5	70.7	0.1	9.3

Table 2: Ablation study on GAP-3 and GAP-5. PA and AA are reported as percentages.

Figure 5: Evaluation Network: Mean predicted fitness versus the number of misplaced pieces on GAP-3 and GAP-5.

Figure 6: Policy Network: Top-1 accuracy versus the number of correctly placed pieces on GAP-3 and GAP-5.

Figure 7: Genetic search progression from earlier candidates to the final reconstruction. Red outlines mark misplaced fragments in each candidate assembly.

Figure 8: Reconstructing images from different numbers and sizes of fragments in daily life.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

	GAP-3		GAP-5
Seed	PA	AA	PA	AA
42	69.2	80.2	19.5	48.4
43	68.5	82.8	19.1	49.6
2026	68.0	81.8	19.7	48.0
M $\pm$ S	$68.6\pm 0.6$	$81.6\pm 1.3$	$19.4\pm 0.3$	$48.7\pm 0.8$

Appendix

Table S1 : Robustness to random seeds on GAP-3 and GAP-5. Results are reported as percentages.

Table 12

Figure S1 : Additional compare reconstruction

Figure S2 : Additional compare reconstruction

Figure S3 : Additional compare reconstruction

Figure S4 : Additional compare reconstruction

Figure S5 : Additional compare reconstruction

Figure S6 : Additional examples of reconstruction live photos

Figure S7 : Additional examples of reconstruction live photos

Figure S8 : Additional examples of reconstruction live photos

Jigsaw Puzzle Solving

See all 10 papers

The Missing GAP: From Solving Square Jigsaw Puzzles to Handling Real World Archaeological Fragments
May 12, 2026Ofir Itzhak Shahar, Gur Elkin, Ohad Ben-ShaharJigsaw Puzzle Solving
Stein Faculty of Computer and Information Science Ben-Gurion University of the Negev, Israel
JigSync: Gauge-Resolved Synchronization for Jigsaw Reassembly under Unknown Piece Orientation
Sep 16, 2026Soham Pahari, Antik Aich Roy, Ujjwal BhattacharyaJigsaw Puzzle Solving
Indian Statistical Institute, Kolkata
Solving Jigsaw Puzzles in the Wild: Human-Guided Reconstruction of Cultural Heritage Fragments
Mar 6, 2026Omidreza Safaei, Sinem Aslan, Sebastiano Vascon +3Jigsaw Puzzle Solving Human-in-the-Loop AI
DAIS, Ca’ Foscari University of Venice, Venice, Italy · Department of Historical Studies, University of Milan, Milan, Italy · Zhejiang Normal University, Jinhua, Zhejiang Province, China

May 12, 2026J/K move · Enter open · S save

The Missing GAP: From Solving Square Jigsaw Puzzles to Handling Real World Archaeological Fragments

Ofir Itzhak Shahar, Gur Elkin, Ohad Ben-Shahar

Stein Faculty of Computer and Information Science Ben-Gurion University of the Negev, Israel

Jigsaw Puzzle Solving

Open paper AskPDF arXiv

Jigsaw puzzle solving has been an increasingly popular task in the computer vision research community. Recent works have utilized cutting-edge architectures and computational approaches to reassemble groups of pieces into a coherent image, while achieving increasingly good results on well established datasets. However, most of these approaches share a common, restricting setting: operating solely on strictly square puzzle pieces. In this work, we introduce GAP, a set of novel jigsaw puzzles datasets containing synthetic, heavily eroded pieces of unrestricted shapes, generated by a learned distribution of real-world archaeological fragments. We also introduce PuzzleFlow, a novel ViT and Flow-Matching based framework for jigsaw puzzle solving, capable of handling complex puzzle pieces and demonstrating superior performance on GAP when compared to both classic and recent prominent works in this domain.

Abstract

Jigsaw Puzzle Solving

Explore similar work

The Missing GAP: From Solving Square Jigsaw Puzzles to Handling Real World Archaeological Fragments

JigSync: Gauge-Resolved Synchronization for Jigsaw Reassembly under Unknown Piece Orientation

Solving Jigsaw Puzzles in the Wild: Human-Guided Reconstruction of Cultural Heritage Fragments