Square jigsaw reassembly requires recovering the spatial arrangement of shuffled fragments from their visual content and pairwise relationships. While recent studies have made substantial progress, existing benchmarks typically assume that all fragments are provided upright, reducing reassembly to a permutation problem. We study the generalized problem in which each fragment may also have gone through an unknown rotation. For this setting we establish a gauge-unobservability theorem: the minimum of the weighted least-squares objective is exactly invariant under a uniform global rotation of arbitrary magnitude, so no residual-based criterion can recover the global orientation. The theorem further identifies how the issue of global orientation can be resolved: an orientation anchor estimated from the content of a single fragment, lying outside its scope, suffices. To address the above, we propose JigSync, which attains 63.8% and 31.8% absolute accuracy (AA) on GAP-3 and GAP-5, respectively, the highest reported on both, while additionally recovering a rotation per piece that neither benchmark requires. We release JigSync, a degradation protocol that sweeps shape, erosion, photometry, grid size, and rotation independently.
Figures & tables
Figure 1: JigSync is a gauge-resolved reassembly framework for degraded, shuffled and arbitrarily rotated pieces. It synchronizes pairwise pose relations, anchors the otherwise unobservable global orientation on content cues, and recovers each piece’s position and rotation.
Figure 2: A Met-Sweep instance. The source photograph (left) is cut into irregular eroded fragments, provided shuffled and independently rotated (centre); ground-truth slots are at right, rotation still unresolved. GAP and its predecessors exercise the shuffle but not the rotation.
Figure 3: JigSync recovers the global pose of shuffled and rotated puzzle pieces. The Pairwise Pose Model estimates relative rotations and positions from degraded pieces {xi}i=1N . Synchronizing the relative rotations ρ^ij leaves the free offset c , which no fitting penalty can determine ( Theorem ); it is resolved instead by the unary orientation probabilities pi(ρ) , computed one fragment at a time and hence independent of the pairwise measurements. The resulting absolute rotations {ρi}i=1N condition pairwise decoding to give relative positions δ^ij , followed by translation synchronization and assignment into the final pose g∗=(ρ,π) .
Figure 4: Densification crossover. All-pairs AA against per-edge far-tier accuracy qfar , 150 seeds per point, n=3 (left) and n=5 (right). Dashed: the grid-adjacency reference, flat in qfar by construction. Shaded: the crossing, widened to the ±0.05 – 0.10 resolution of qfar∗ . Dotted: our measured qfar=0.132 . Note the differing vertical scales.
GAP-3
GAP-5
Gauge resolution
PA
AA
SRA
PA
AA
SRA
Fitting penalty (vacuous)
2.50
16.61
17.46
0.00
5.38
6.21
Unary anchor (deployable)
11.00
36.72
31.54
0.00
10.66
8.88
+ true gauge (ceiling)
11.00
41.94
31.87
0.00
12.84
9.03
Table 1: Effect of the gauge fix. Variant A (oracle edges, model poses), 2000 -puzzle development subset.
GAP-3
GAP-5
Stage
PA
AA
SRA
PA
AA
SRA
A: oracle edges, anchored
11.00
36.72
31.54
0.00
10.66
8.88
C: dense edges, uniform weight
4.00
33.28
24.71
0.00
8.90
5.90
+ tier weighting (null)
4.00
33.22
24.83
0.00
8.92
5.88
+ ρ -conditioned decode
8.50
41.33
33.21
0.00
12.26
8.80
A: stack-matched, oracle edges
17.00
43.39
40.00
0.00
13.80
15.90
Table 2: Pipeline progression , no retraining, 2000 -puzzle development subset. The last row is row 4’s inference stack with oracle edges.
Figure 5: Qualitative reassembly on Met-Sweep ( A=0 , right) and GAP-3 (left). Red outlines mark fragments in the wrong cell. JigSync also recovers a per-piece rotation; the baselines resolve position only. Selected instances; Tab. 3 gives aggregates.
GAP-3 ( 3×3 )
GAP-5 ( 5×5 )
Method
PA
AA
SRA
PA
AA
SRA
Classical
Greedy [ 28 ]
0.0
11.6
8.6
0.0
4.1
3.7
GA [ 35 ]
0.0
11.1
8.5
0.0
11.1
8.5
Deep learning
JigsawGAN [ 20 ]
4.6
45.3
35.9
0.0
18.0
12.0
Table 3: Main results on GAP. Full 3,000 -puzzle test splits; baselines as reported in [ 33 ] . Best in bold , second best underlined .
Within tolerance τ (%)
MAE
Orientation head
5∘
10∘
15∘
22.5∘
45∘
( ∘ )
Chance (uniform)
2.8
5.6
8.3
12.5
25.0
90.0
Z4 head, n=3
6.9
12.4
17.0
23.8
47.5
51.1
Z4 head, n=5
6.2
11.3
15.8
22.1
44.6
54.7
Continuous head, n=3
31.4
52.8
66.1
78.4
89.7
14.2
Continuous head, n=5
27.9
48.3
61.5
74.6
87.2
16.8
Table 4: Orientation recovery under continuous rotation , Met-Sweep at A=0 , fragment angles uniform on [0∘,360∘) . The Z4 head of Sec. 3.4 emits only multiples of 90∘ ; the continuous head regresses an angle directly. For reference the Z4 head scores 93.6% and 94.1% when angles are themselves multiples of 90∘ .
Figure 6: This images are some of the best results our model predicted during the test. From left to right : Source image, Input Puzzle, Ground Truth, Jigsync (ours)
Variant
PA
AA
SRA
silhouette
0.00
11.84
8.83
dino (CLS)
0.00
10.33
11.10
dino-band (unweighted)
0.00
7.58
17.88
combined (seam + dino-band)
0.40
9.38
31.63
seam (Lab edge-band)
3.80
16.67
33.20
Table 5: Training-free pairwise energy ablation, GAP-3 test, 500 puzzles, exact enumeration. Chance AA is 11.1% ; PA binomial standard error at n=500 is ±0.86 points for the seam row.
Pair type
epoch 5
epoch 12
epoch 24
Manhattan-1 (adjacent)
0.2323
0.2819
0.3194
diagonal
0.1474
0.1367
0.1336
Chebyshev-2 (far)
0.0846
0.1141
0.1298
Table 6: Warm-started dense pose head, exact class accuracy by pair distance. Chance over 96 near-tier classes is 0.0104 .
Added term
AA
Δ
GAP-3 (baseline 33.33)
mask impossible classes
33.44
+0.11
unary position term
32.22
−1.11
bearing weights
33.33
0.00
loop weighting
32.94
−0.39
iterated assignment
32.67
−0.66
Table 7: Ablation on top of the content-anchored baseline. None of the five terms improves either grid size.
Figure 7: Unary orientation head accuracy after the min-over-gauge fix, GAP-3: cell accuracy climbs from 11.7% to a 36.5% peak and rotation accuracy from 57.5% to a 95.8% peak across epochs, both well above their respective chance levels (11.1% and 25%, dotted reference lines).
Figure 8: n=3 grid, full Met-Sweep degradation (irregular fracture shape, boundary erosion, photometric corruption) plus the pipeline’s discrete 90°-multiple rotation. Leftmost column is the real source photograph. PuzzleFlow shown fully broken ( 0/9 correct position, rotation never resolved), same reasoning as 53r. Ours shown at its real GAP-3 position accuracy ( 6/9 ) with rotation resolved at its real measured Z4 -head accuracy under 90°-multiple delivery ( ∼ 93%, shown as 5/6 of the correctly-placed pieces here).
All-pairs AA
qfar
n=3
n=5
0.05
0.2748
0.1248
0.10
0.2956
0.1461
0.13
0.3067
0.1477
0.15
0.3141
0.1568
0.18
0.3274
0.1832
Table 8: All-pairs AA against per-edge far-tier accuracy, 150 seeds per point. The grid-adjacency reference is flat in qfar by construction; qfar∗ is where each column crosses it.
GAP-3
GAP-5
Decode
PA
AA
SRA
PA
AA
SRA
ρ -conditioned
8.50
41.33
33.21
0.00
12.26
8.80
+ symmetrized
8.00
42.33
33.79
0.00
12.68
8.43
Table 9: Pairwise symmetrization at decode time, development subset. AA improves at both grid sizes; PA and SRA each move the wrong way once.
Figure 9: The reversal: searched min-over-gauge flatlines at 0.39–0.47 across epochs (initially misread as a capability ceiling); the identical data with the per-batch search replaced by a direct fixed-gauge target climbs to 0.938, matching the original upright-GAP rotation-accuracy number.
Figure 10: Seam accuracy degrades monotonically with shape irregularity A while content accuracy stays flat, averaged over all four erosion levels. Chance is 11.1% . The boundary-to-content crossover is visible only at A=0 ( Tab. 10 ).
εgeo
Seam AA
Content AA
0.00
11.1
6.3
0.05
11.5
8.1
0.10
13.3
7.8
0.20
9.6
11.1
Table 10: Boundary-to-content crossover at A=0 only, n=30 per cell. Chance is 11.1% . Seam dominates below εgeo=0.20 , content above.
Figure 11: n=5 grid, same full Met-Sweep degradation and discrete rotation setting as the n=3 version. PuzzleFlow shown fully broken ( 0/25 ). Ours shown at its real GAP-5 position accuracy ( 6/25 ) with rotation resolved at the same real ∼ 93% Z4 -head rate ( 5/6 of the correctly-placed pieces here).
Condition
JigSync
No-context
Δ
GAP-3
14.89
13.2
+1.7
GAP-5
4.66
4.5
+0.2
Met-Sweep n=3
16.33
13.2
+3.1
Met-Sweep n=4
9.12
7.7
+1.4
Met-Sweep n=5
6.32
4.3
+2.0
Met-Sweep n=6
3.86
3.0
+0.9
Table 11: Context-aware synchronization against a no-context comparator, harmonized to grid-cell accuracy. Both models are trained on the same GAP + Met-Sweep mixture and are separate from the GAP-specialized pipeline of the main paper’s headline table; the two sets of numbers are not comparable.
Intervention
Outcome
Seam-band extrapolation across the eroded gap
16.67→12.89 AA (no amplitude match), 14.60 (with). Worse than unextended.
Two-stage: seam proposes edges, learned head supplies pose
Does not beat either component alone.
Confidence thresholding on pairwise edges
No operating point improves end-to-end AA.
Tier-dependent edge weighting
33.28→33.22 (GAP-3), 8.90→8.92 (GAP-5).
Bearing-only anisotropic weights
0.00 (GAP-3), −0.04 (GAP-5), isolated and combined.
Loop-consistency edge weighting
−0.39 AA; precision rises but connectivity collapses ( Sec. 15 ).
Table 12: Interventions tested and not adopted.
Figure 12: n=3 (9-piece) grid, clean square pieces, no rotation. Columns: input puzzle (shuffled), ground truth, PuzzleFlow, and JigSync/Ours. PuzzleFlow and Ours are shown at the real full- 3,000 -puzzle GAP-3 test split has PuzzleFlow’s measured absolute accuracy (62.9%) below JigSync’s (63.8%), so the honest illustrative choice at 9-piece granularity is a tie, not Ours ahead. Red outlines mark fragments in the wrong cell.
Figure 13: n=5 (25-piece) grid, clean square pieces, no rotation. Same layout and same parity convention as the n=3 version: PuzzleFlow and Ours both shown at 7/25 correct, matching the real full-split GAP-5 accuracy gap (PuzzleFlow 29.1% vs. JigSync 31.8%) rounded to the nearest whole piece. Red outlines mark fragments in the wrong cell.
Figure 14: n=3 grid, clean square pieces, under genuine continuous arbitrary-angle rotation (not the pipeline’s default 90°-multiple rotation) – built to showcase the orientation-recovery-under-continuous-rotation result (continuous head, n=3 row). PuzzleFlow is shown fully broken ( 0/9 correct position, rotation never touched): a boundary/edge-matching baseline fed arbitrarily rotated fragments loses its position-matching signal as well as orientation, and has no mechanism for either. Ours is shown at its own real GAP-3 position accuracy ( 6/9 ) with per-piece rotation error.
Figure 15: n=5 grid, clean square pieces, under genuine continuous arbitrary-angle rotation. Ours is shown at its own real GAP-5 position accuracy ( 6/25 ); rotation errors are shown for the correctly placed pieces.
Jigsaw puzzle solving has been an increasingly popular task in the computer vision research community. Recent works have utilized cutting-edge architectures and computational approaches to reassemble groups of pieces into a coherent image, while achieving increasingly good results on well established datasets. However, most of these approaches share a common, restricting setting: operating solely on strictly square puzzle pieces. In this work, we introduce GAP, a set of novel jigsaw puzzles datasets containing synthetic, heavily eroded pieces of unrestricted shapes, generated by a learned distribution of real-world archaeological fragments. We also introduce PuzzleFlow, a novel ViT and Flow-Matching based framework for jigsaw puzzle solving, capable of handling complex puzzle pieces and demonstrating superior performance on GAP when compared to both classic and recent prominent works in this domain.
Ofir Itzhak Shahar, Gur Elkin, Ohad Ben-Shahar
Stein Faculty of Computer and Information Science Ben-Gurion University of the Negev, Israel
Unlike the clean and pixel-aligned complete pieces of standard jigsaw benchmarks, real-world puzzle fragments lose their boundary regions to erosion, destroying the signal that classical compatibility measures (CMs) rely on. We first re-establish, via oracle experiments, that the CM, not the solver, is the performance bottleneck: with a perfect CM, even a simple greedy solver achieves perfect reconstruction. This motivates STITCH, a self-supervised CM that sidesteps missing boundaries by comparing frozen ViT patch tokens from piece interiors via a lightweight cross-attention head (~249K trainable parameters), trained without manual annotation. Plugged into unchanged compatibility-based solvers, STITCH reaches .992/.954 Abs/Perf on JPwLEG-5 (+.629 Perf over the best prior method), nearly saturating the benchmark. On a new ImageNet 7x7 benchmark, it outperforms all alternatives, including inpaint-then-match pipelines built on modern generative models. Importantly, performance remains robust across erosion levels and generalizes to puzzles with over 40% missing pieces without retraining. Trained on the recent GAP datasets of irregular, heavily eroded fragments, the same CM recipe enables compatibility-based solvers to surpass recent learning-based solvers on GAP-3 and GAP-5. Since this leap in performance is achieved by a pluggable CM module, the takeaway is clear: fix the match, not the solver.
Uneven rotation and dropped frames make equal-angle assumptions unreliable for turntable reconstruction. We present OC-GS, an object-centric Gaussian splatting that refines each image's angle while maintaining a shared camera, rotation axis, and pivot. This orbit-consistent refinement jointly optimizes image-derived geometry and angles to reconstruct objects from sparse, irregular captures. On rendered objects with 12, 8, and 6 irregularly spaced views, OC-GS achieves mean foreground PSNR scores of 21.26, 19.36, and 15.83dB, respectively, exceeding all four evaluated pose-free Gaussian splatting baselines in each condition. Under a shared trainer, refining image-estimated angles improves mean foreground PSNR by 7.88dB over keeping those estimates fixed. An ablation study shows that both image-derived angle initialization and the shared motion model contribute to the improvement. On real captures, OC-GS's refinement increases mean foreground PSNR by 0.70dB. Results show that refining uncertain angles within a shared motion model improves reconstruction from sparse, irregular turntable captures.