Can Image Generators Hold a Scene Together When Space Is Broken Apart?
Organizations: National Yang Ming Chiao Tung University
Abstract
Image generators are good at drawing scenes in which things that belong together appear next to each other. In almost every image they generate, and in almost every benchmark used to evaluate these images, "belonging to the same whole" and "adjacent to each other" go together. Therefore, we cannot know whether the model truly grasps the scene as a whole, or simply makes each part connect with its adjacent parts. We propose GJBench to disentangle the two. It requires the model to draw a scene as a scattered jigsaw puzzle: an image is cut into interlocking pieces, then moved apart and rotated. Thus, the spatial plausibility of the scene exists only in how these pieces collectively point to a whole. The model may have a reference image, a target layout, both, or neither. We separately evaluate three requirements: whether the scene is broken down into a specified number of separate pieces, whether the pieces interlock, and whether the pieces can reconstruct the reference scene. While every open and closed model we evaluated could draw these scenes well as a whole, none could keep them together as scattered puzzle pieces, and the three requirements categorized failures into different patterns. The models draw separate pieces but miscount them in different ways, and unless their outlines are provided, the pieces rarely interlock, and their content does not reconstruct the reference image. We also found that the target layout helps the puzzle pieces fit together, while the reference image improves neither their count nor their fit. These findings show that keeping a scene continuous across space that the canvas no longer shows remains an open problem for image generators.
Figures & tables
| Condition | Images | Text appended to the shared prompt |
|---|---|---|
| Text | none | Place and rotate the pieces at random across the gray background. |
| Ref | reference | Image 1 shows the scene exactly. + the Text clause |
| Layout | layout | Each piece takes the exact place of a different piece shape in Image 1, on Image 1’s background. |
| Ref+Layout | both | the Ref clause + the Layout clause, naming Image 2 |
| RQ2 | RQ3 | |||||
|---|---|---|---|---|---|---|
| Construction ( ) | 4 | 9 | 16 | 4 | 9 | 16 |
| Exact cut | .95 | .93 | .88 | .92 | .92 | .92 |
| One piece missing | .63 | .85 | .84 | .79 | .86 | .89 |
| Tabs shifted along their edge | .48 | .78 | .77 | as exact | ||
| Content turned in each piece | as exact | .67 | .75 | .80 | ||
| Content turned in each piece | as exact | .45 | .55 | .61 | ||
| Exact count (RQ1) | Interlock (RQ2) | Reconstruction (RQ3) | |||||||
|---|---|---|---|---|---|---|---|---|---|
| 4 | 9 | 16 | 4 | 9 | 16 | 4 | 9 | 16 | |
| GPT Image 2.6 | .98 | .98 | .94 | .74 | .73 | .74 | .38 | .45 | .49 |
| Nano Banana 2.1 | .95 | .83 | .45 | .63 | .63 | .57 | .38 | .45 | .48 |
| Muse Image 1.0 | .74 | .72 | .50 | .56 | .64 | .65 | .33 | .40 | .44 |
| Qwen-Image-2.1 | .79 | .76 | .65 | .51 | .51 | .50 | .33 | .33 | .37 |
| FLUX 3 Image Klein | .73 | .67 | .39 | .28 | .34 | .33 | .31 | .35 | .39 |
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
| Exact count (RQ1) | Interlock (RQ2) | Reconstruction (RQ3) | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| System | Reading | 4 | 9 | 16 | 4 | 9 | 16 | 4 | 9 | 16 |
| GPT Image 2.6 | A | .95 | .89 | .81 | .67 | .64 | .62 | .36 | .43 | .47 |
| B | .98 | .90 | .82 | .70 | .66 | .66 | .38 | .45 | .49 | |
| C | .87 | .77 | .65 | .68 | .66 | .65 | .35 | .42 | .45 | |
| D | .80 | .69 | .60 | .71 | .71 | .71 | .36 | .43 | .47 | |
| Nano Banana 2.1 | A | .95 | .80 | .41 | .54 | .53 | .44 | .35 | .41 | .44 |
| System | opt. | mean gap | RQ2 found | RQ2 bound |
|---|---|---|---|---|
| GPT Image 2.6 | .63 | 0.015 | .74 | .74 |
| Nano Banana 2.1 | .62 | 0.011 | .57 | .60 |
| Muse Image 1.0 | .51 | 0.024 | .65 | .66 |
| Qwen-Image-2.1 | .72 | 0.035 | .50 | .50 |
| FLUX 3 Image Klein | .56 | 0.029 | .33 | .35 |
| FLUX.2 [klein] | .86 | 0.103 | .04 | .07 |
| System | opt. | gap | opt. | gap | opt. | gap |
|---|---|---|---|---|---|---|
| GPT Image 2.6 | 1.00 | – | 1.00 | – | .63 | 0.012 |
| Nano Banana 2.1 | 1.00 | – | 1.00 | – | .62 | 0.008 |
| Muse Image 1.0 | 1.00 | – | 1.00 | – | .51 | 0.021 |
| Qwen-Image-2.1 | 1.00 | – | 1.00 | – | .72 | 0.028 |
| FLUX 3 Image Klein | 1.00 | – | 1.00 | – | .56 | 0.022 |
| Constant | Value | Source |
| Background color | median of the outermost ring | task: plain background |
| Foreground threshold | triangle method | [ 78 ] |
| Speck depth | 2 pixels | too thin to be a piece |
| Median filter (readings B, D) | smallest symmetric window | |
| Merged pieces | area , rounded, minus one | plain reading of the count |
| Interlock score | Dice | [ 14 ] |
| RQ2 | RQ3 | Precision | Recall | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Construction ( ) | 4 | 9 | 16 | 4 | 9 | 16 | 4 | 9 | 16 | 4 | 9 | 16 |
| Exact cut | .95 | .93 | .88 | .92 | .92 | .92 | .92 | .92 | .92 | .92 | .92 | .92 |
| One piece missing | .63 | .85 | .84 | .79 | .86 | .89 | .92 | .92 | .92 | .69 | .82 | .86 |
| Three pieces missing | .00 | .77 | .79 | .38 | .73 | .82 | .88 | .91 | .90 | .24 | .61 | .76 |
| One piece duplicated | .95 | .93 | .88 | .81 | .87 | .89 | .81 | .87 | .89 | .81 | .87 | .89 |
| Half the pieces duplicated | .95 | .93 | .88 | .70 | .77 | .77 | .70 | .77 | .77 | .70 | .77 | .77 |
| System | Exact count | Interlock | Reconstruction | |
|---|---|---|---|---|
| GPT Image 2.6 | 4 | .98 [.96, 1.00] | .74 [.71, .76] | .38 [.34, .42] |
| 9 | .98 [.95, 1.00] | .73 [.71, .74] | .45 [.41, .49] | |
| 16 | .94 [.90, .97] | .74 [.73, .75] | .49 [.46, .53] | |
| Nano Banana 2.1 | 4 | .95 [.91, .98] | .63 [.60, .66] | .38 [.34, .41] |
| 9 | .83 [.76, .89] | .63 [.61, .64] | .45 [.42, .49] | |
| 16 | .45 [.37, .52] | .57 [.55, .58] | .48 [.45, .51] |
| Adding a layout | Adding a reference | ||||
|---|---|---|---|---|---|
| System | Exact count | Interlock | Reconstruction | Exact count | Interlock |
| GPT Image 2.6 | +.01 [ .02, .03] | +.14 [.12, .16] | +.04 [.03, .05] | .02 [ .05, .02] | +.01 [ .01, .03] |
| Nano Banana 2.1 | +.13 [.06, .21] | +.08 [.06, .10] | +.02 [.01, .03] | .04 [ .11, .04] | .02 [ .05, .01] |
| Muse Image 1.0 | +.01 [ .09, .10] | +.16 [.13, .18] | .01 [ .02, .00] | +.04 [ .05, .14] | .03 [ .05, .01] |
| Qwen-Image-2.1 | +.31 [.23, .39] | +.48 [.46, .51] | +.05 [.02, .08] | .21 [ .29, .14] | .06 [ .09, .04] |
| FLUX 3 Image Klein | .02 [ .10, .07] | +.04 [ .01, .08] | +.01 [ .02, .03] | .08 [ .16, .01] | .10 [ .13, .06] |
| Scatter (RQ1) | Interlock | Reconstruction (RQ3) | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| System | Condition | exact | too few | too many | merged | border | RQ2 | RQ3 | P | R | thumbnail | |
| GPT Image 2.6 | Text | 4 | 1.00 | .00 | .00 | 0.00 | .00 | .63 | – | – | – | – |
| 9 | .97 | .03 | .00 | 0.03 | .01 | .62 | – | – | – | – | ||
| 16 | .94 | .06 | .00 | 0.09 | .01 | .69 | – | – | – | – | ||
| Layout | 4 | 1.00 | .00 | .00 | 0.00 | .00 | .85 | – | – | – | – | |
| 9 | .97 | .00 | .03 | 0.00 | .00 | .81 | – | – | – | – | ||
| Condition | Captions | Rows or grid | Turns kept | “Random” kept |
|---|---|---|---|---|
| Text | 91 | 85% | 81% | 41% |
| Layout | 96 | 45% | 31% | – |
| Ref | 96 | 42% | 91% | 51% |
| Ref+Layout | 96 | 54% | 46% | – |
| Exact count (RQ1) | Reconstruction (RQ3) | Change | ||||||||
| System | Jigsaw | Scattered | In place | Jigsaw | Scattered | In place | Whole | Position, scattered jigsaw | RQ3, in place scattered | RQ3, whole in place |
| GPT Image 2.6 | .97 | 1.00 | .47 | .43 | .37 | .39 | .84 | +.16 | +.02 | +.44 |
| Nano Banana 2.1 | .75 | .88 | .88 | .44 | .37 | .51 | .63 | +.12 | +.14 | +.12 |
| Muse Image 1.0 | .81 | .78 | .97 | .40 | .33 | .36 | .85 | +.15 | +.03 | +.49 |
| Qwen-Image-2.1 | .53 | .94 | .91 | .29 | .27 | .27 | .61 | +.30 | +.00 | +.34 |
| FLUX 3 Image Klein | .78 | .88 | .66 | .34 | .30 | .32 | .85 | +.23 | +.02 | +.52 |
| RQ3, best pairing | RQ3, one assembly | Connectors present | Connectors fitted | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 4 | 9 | 16 | 4 | 9 | 16 | 4 | 9 | 16 | 4 | 9 | 16 | |
| GPT Image 2.6 | .37 | .44 | .49 | .33 | .38 | .43 | 1.00 | 1.00 | 1.00 | .89 | .95 | .94 |
| Nano Banana 2.1 | .36 | .44 | .47 | .32 | .37 | .40 | 1.00 | 1.00 | 1.00 | .82 | .80 | .75 |
| Muse Image 1.0 | .32 | .39 | .43 | .30 | .35 | .38 | 1.00 | 1.00 | .99 | .87 | .94 | .92 |
| Qwen-Image-2.1 | .31 | .31 | .37 | .29 | .29 | .33 | .91 | .90 | .85 | .62 | .61 | .59 |
| FLUX 3 Image Klein | .28 | .31 | .37 | .27 | .28 | .34 | .84 | .87 | .84 | .44 | .48 | .44 |
| System | Identifier | Setting | Scattered | Whole | Latency | Cost (USD) |
|---|---|---|---|---|---|---|
| FLUX 3 Image Klein [ 5 ] | black-forest-labs/flux-3-image-klein , revision 09561231 | bfloat16, 16 steps, no guidance, prompt upsampler | 15.6 s | GPU | ||
| FLUX.2 [klein] 4B [ 4 ] | black-forest-labs/FLUX.2-klein-4B , revision e7b7dc27 | bfloat16, 4 steps, guidance 1.0 | 1.8 s | GPU | ||
| Qwen-Image-2.1 [ 52 ] | Qwen/Qwen-Image-2.1 , revision d26bb612 | bfloat16, 40 steps, true CFG 1.0, key-value cache | 17.7 s | GPU | ||
| GPT Image 2.6 [ 48 ] | gpt-image-2.6-sunburst , Images API | quality max, complexity high, moderation low | 92.2 s | 45.03 / 51.78 | ||
| Gemini Nano Banana 2.1 [ 24 ] | gemini-nano-banana-2.1 , Gemini API | image size 1K, aspect 3:2, thinking level high | 34.6 s | 14.41 / 15.83 | ||
| Muse Image 1.0 [ 46 ] | muse-image-1.0 , Meta model API | reasoning strength high, tools off | 23.3 s | 2.56 / 2.88 |