Text-to-image diffusion and flow models yield a preview, their clean-image estimate, at every denoising step. When they miscount, drop or misplace objects, test-time search is in a bind: its verifiers are reward models built for finished images, so judging early misreads blurry previews, and judging late, as best-of-N does, pays to fully denoise doomed candidates. We find that the image is not the problem: the outcome is settled as denoising starts, by the seed or by a stochastic sampler's first steps, and already shows in early previews; what fails on them is the judge, as reward models read them near chance on several kinds of prompts. We introduce Keystone, a training-free judge of previews that compiles a prompt into typed questions about its objects, counts, colours and positions and has an off-the-shelf multimodal decision model answer them. Asked this way, the judge reads counts and positions from early previews better than reward models read them from finished images. Searching at these key steps on GenEval with SD3.5-Medium, Keystone's adaptive search and pruning significantly outperform best-of-8 with every other verifier at under half its generator compute, and as a drop-in verifier there, the Keystone score significantly improves best-of-N, progressive seed pruning and FK steering; without retuning, both searches stay significantly above the strongest other verifier's best-of-N on average over four base models and three benchmarks. Keystone thus moves test-time judgment to the key steps where the outcome already shows.
Figures & tables
Figure 1: The outcome is visible long before denoising ends. Four of eight candidates for a development prompt on SD3.5-Medium. Keystone asks whether there are exactly four bananas and scores each candidate’s preview at step 8 of 40 (top); of the four shown, its pruning keeps the two it rates highest (framed) and need not finish the other two, which the official evaluator marks wrong (bottom, finished for reference). Example chosen by a fixed rule ( Sec. D.16.1 ).
Figure 2: Keystone compiles the prompt, reads previews at the key steps and searches there. (a) The prompt becomes typed questions about its slots, and an image’s score is its smallest slot score ( Sec. 3.2 ). (b) The outcome is decided in the first steps and read at a key step, here step 8 of the example of Fig. 1 ( Secs. 3.3 and 4.1 ). (c) Adaptive search and pruning spend compute by the preview scores ( Sec. 3.4 , Alg. 1 ).
Figure 3: Keystone’s searches lie above best-of- N with every judge at the same cost (SD3.5-Medium, official test prompts). Lines: best-of- N on finished images with each judge; points: Keystone’s adaptive search (filled) and pruning (open) with the same judge, each significantly more accurate with the Keystone score; numbers: paired gains over best-of- N with the Keystone score (ours) at a higher cost, both significant.
Figure 4: Compositional outcomes are decided in the first steps and can be read well before the end (SD3.5-Medium, development prompts). Left: fraction of the 40 sibling groups whose GenEval verdicts differ when only the noise of one step of a 10-step stochastic sampler (Flow-GRPO’s [ 40 ] ) changes. Right: fraction of candidate pairs with different verdicts that each judge orders correctly from the preview at each step of the 40-step schedule, counting and two-object prompts pooled. Left, error bars: 95% Wilson intervals; right, band and bars: 95% bootstrap intervals over prompts.
Figure 5: Case study: Keystone ranks the candidate that ends correct above the other at every step (the first and last step-8 candidates of Fig. 1 ). Top: previews and final image with their Keystone score. Bottom: each judge’s preference between the two, the difference of its two scores over the spread of its scores at that step.
Figure 6: On Sana-0.6B, an open judge answering Keystone’s questions reads previews best (GenEval counting and two-object; six candidates per prompt that branch after step 2 of 20). Pairs of candidates ordered correctly from the step-8 preview (left) and from the final image (right), with Keystone’s questions answered by Qwen3-VL-8B. Every judge and reading: Sec. D.7 .
Figure 7: Keystone’s pruning ends correct where ImageReward’s ends wrong (SD3.5-Medium, official test prompts). Per prompt, an image’s eight step-8 candidates with Keystone scores (framed: the four kept) and the image each pruning ends on (correct/wrong: verdicts of GenEval’s official evaluator, as in Fig. 1 ). Examples follow a fixed rule ( Sec. D.16.2 ); this happens for 198 images in these categories.
Preview at step 8
Final image
Judge
Two obj.
Count.
Colours
Pos.
Col. attr.
Count.
Colours
Pos.
Keystone score (ours)
0.73
0.86
0.62
0.87
0.57
0.93
0.68
0.97
VQAScore
0.57
0.74
0.56
0.57
0.51
0.82
0.64
0.79
HPSv2.1
0.58
0.66
0.50
0.52
0.51
0.77
0.64
0.52
ImageReward
0.61
0.52
0.52
0.57
0.48
0.72
0.63
0.66
Table 1: Keystone reads counts, objects and positions from step-8 previews best (SD3.5-Medium, development prompts). Fraction of candidate pairs with different official verdicts that each judge orders correctly, from the step-8 preview and, for three categories, the final image; bold: best (step-8 intervals and test prompts: Sec. D.6 ).
Method
NFE
Count.
Position
Overall
One sample
40
0.56
0.26
0.68
Best-of-2, VQAScore
80
0.66
0.32
0.73
Best-of-2, Keystone score (ours)
80
0.70
0.37
0.74
Adaptive search ‡ , VQAScore
71
0.71
0.39
0.75
Keystone, adaptive (ours)
73
0.79
0.53
0.80
Best-of-4, VQAScore
160
0.72
0.37
0.76
Table 2: Keystone’s searches are the most accurate at each budget (SD3.5-Medium, official test prompts, four images per prompt, each a search over eight candidates). Each block: methods that cost no more than its best-of- N , against the strongest verifiers (every verifier: Sec. D.1 ); bold: best overall; ‡ : Keystone’s search with another judge. Each row with the Keystone score is significantly above its procedure with every other verifier. Pruning keeps four of eight at step 8 and two at step 16; PSP [ 19 ] and FK steering [ 47 ] use ImageReward. Gains are computed before rounding.
Figure 10Figure 11Figure 12
Appendix figures & tables24 assets
Supplementary material from the paper’s appendix.
Appendix
Minus
Adaptive search (Keystone)
Pruning (Keystone)
Best-of-2 / best-of-4 (the budgets of Tab. 2 )
Keystone score
+0.054 [+0.043, +0.066]
+0.028 [+0.020, +0.036]
VQAScore
+0.071 [+0.058, +0.083]
+0.061 [+0.050, +0.072]
HPSv2.1
+0.079 [+0.066, +0.091]
+0.076 [+0.062, +0.088]
ImageReward
+0.074 [+0.061, +0.087]
+0.068 [+0.055, +0.081]
PickScore
+0.079 [+0.066, +0.092]
+0.072 [+0.059, +0.085]
Appendix
Table 5: Keystone’s searches are significantly more accurate than every best-of- N and search in their blocks of Tab. 2 , and than every other verifier’s best-of-8 at under half its cost (SD3.5-Medium, official test prompts; the searches of Tab. 2 at 73 and 144 NFE; PSP with its own schedule; FK steering with four particles, 160 NFE). Overall GenEval accuracy of Keystone’s search minus that of the row’s method; mean difference with its 95% paired bootstrap interval over prompts resampled within category.
Method
Compute
GenEval
Gain
SD3.5-Large, as reported by PSP [ 19 ]
One sample
1.0 ×
0.713
–
Best-of- N
4.0 ×
0.747
+0.034
FK steering
4.0 ×
0.742
+0.029
DSearch
4.0 ×
0.704
−0.009
NTS
4.0 ×
0.699
−0.014
Appendix
Table 6: Against published test-time methods: Keystone’s searches gain more than best-of- N at lower compute, which none of the seven published methods above does . Upper block: as reported by PSP [ 19 ] on SD3.5-Large (1024 px, 32 steps). Lower block: Keystone on SD3.5-Medium (512 px, 40 steps). Compute is a multiple of one sample; as models and samplers differ, only comparisons within each block (gains over one sample, and against best-of- N ) are meaningful. Bold: Keystone’s gains.
Figure 14: With the Keystone score as its verifier, best-of- N is more accurate than with any other verifier at every N from 2 to 8, and its lead widens from N=2 to N=8 (SD3.5-Medium, official test prompts, the candidates of Tab. 2 ; correctness from GenEval’s official evaluator). (a) GenEval overall accuracy of best-of- N on finished images with each verifier, from N=1 , one sample, to N=8 (40 N NFE per image). (b) Best-of- N with the Keystone score minus best-of- N with each other verifier at the same N and cost: mean difference (line) and its 95% paired bootstrap interval over prompts resampled within category (band); ∗ : interval above zero, as for all 42 comparisons (smallest lower bound +0.013). Values: (a) Tab. 7 ; (b) Tabs. 7 and 17 .
Both read the step-8 preview
Category
Keystone score minus
Difference
Two objects
VQAScore
+0.16 [+0.08, +0.24]
HPSv2.1
+0.15 [+0.06, +0.24]
ImageReward
+0.12 [+0.05, +0.19]
Counting
VQAScore
+0.12 [+0.06, +0.18]
HPSv2.1
+0.20 [+0.14, +0.25]
Appendix
Table 8: From the step-8 preview, the Keystone score orders pairs significantly better than the other judge in every comparison of Tab. 1 stated in Sec. 4.2 (SD3.5-Medium development prompts): pairs ordered correctly by the Keystone score minus those ordered by the other judge, with 95% intervals from a paired bootstrap over prompts (2,000 draws); bold where the interval lies above zero. Left: both judges read the step-8 preview; right: the Keystone score reads the step-8 preview and the other judge the finished image.
Preview at step 8
Judge
Single
Two obj.
Count.
Colours
Position
Col. attr.
Keystone score (ours)
0.59
0.69
0.84
0.67
0.79
0.60
HPSv2.1
0.49 †
0.57 †
0.73 †
0.57 †
0.53 †
0.53 †
ImageReward
0.45 †
0.61 †
0.61 †
0.60 †
0.54 †
0.55 †
Appendix
Table 9: On the official test prompts, the Keystone score orders pairs from the same step-8 previews significantly better than both reward models, in every category (SD3.5-Medium, 553 prompts × 4 images × 8 candidates; pairs within one image’s candidates). Bold: best in the column. † : the Keystone score’s step-8 advantage over that judge has a 95% paired bootstrap interval above zero.
Judge
Pairs ordered
HPSv2.1
ImageReward
PickScore
CLIP score
Qwen3-VL-8B
Step-8 preview
+0.06 [+0.02, +0.10] ⋆
+0.05 [+0.01, +0.10] ⋆
+0.08 [+0.04, +0.11] ⋆
+0.12 [+0.07, +0.16] ⋆
Qwen3-VL-8B
Final image
+0.06 [+0.01, +0.10] ⋆
+0.06 [+0.01, +0.11] ⋆
+0.08 [+0.03, +0.12] ⋆
+0.14 [+0.10, +0.18] ⋆
Jev
Final image
+0.06 [+0.02, +0.11] ⋆
+0.07 [+0.02, +0.11] ⋆
+0.08 [+0.04, +0.12] ⋆
+0.14 [+0.10, +0.19] ⋆
Appendix
Table 10: Keystone’s questions answered by Qwen3-VL-8B order pairs significantly better than every preference model, from the step-8 preview and from the final image (Sana-0.6B, GenEval counting and two-object, 179 prompts with four seeds each; the readings of Fig. 6 ). Pairs ordered correctly by Keystone’s questions, answered by Qwen3-VL-8B or by Jev, minus those ordered by each other verifier reading the same image, with 95% intervals from a paired bootstrap over prompts (2,000 draws); ⋆ : interval above zero.
Preview, step 8 of 20
Final image
Judge
Pairs ordered
Key-step pick
Pairs ordered
Best-of-6
Keystone, Jev (ours)
0.682
0.742
0.723
0.745
Keystone, Qwen3-VL-8B (ours)
0.701
0.744
0.720
0.751
HPSv2.1
0.641
0.735
0.661
0.728
ImageReward
0.647
0.723
0.658
0.725
PickScore
0.625
0.725
0.645
0.723
Appendix
Table 11: Fig. 6 as a table: Keystone’s questions, answered by Jev or by Qwen3-VL-8B, are the two best of the six judges in every column (Sana-0.6B; GenEval counting and two-object, 179 prompts × 4 seeds, six candidates that share the first two of 20 steps and branch there). Pairs ordered: fraction of candidate pairs with different official verdicts that the judge orders correctly, pooled over all pairs. Key-step pick and best-of-6: GenEval accuracy of the candidate the judge ranks first at step 8 and on final images. Bold: best; underlined: second best. The GenEval detector applied to the preview, single sample and oracle are references, not methods.
Method
NFE
Count.
Position
Overall
One sample
40
0.56
0.26
0.678
Best-of-2, CLIP score
80
0.63
0.28
0.708
Best-of-2, PickScore
80
0.67
0.28
0.719
Best-of-2, VQAScore
80
0.66
0.32
0.727
Best-of-2, ImageReward
80
0.64
0.30
0.724
Best-of-2, HPSv2.1
80
0.66
0.28
0.719
Appendix
Table 12: Keystone’s searches with Qwen3-VL-8B as the judge: in every block, Keystone with either judge is more accurate overall than every other verifier (SD3.5-Medium, GenEval). Rows are best-of- N with each verifier and the two Keystone searches driven by each score, all on the candidates of Tab. 2 . NFE is network evaluations per image; overall is GenEval’s score, the mean of its six category scores, given to three decimals, counting and position to two; bold: best overall in the block.
Keystone with Qwen3-VL-8B minus
Adaptive search
Pruning
Best-of- N at the same NFE, Keystone, Qwen3-VL-8B
+0.058 [+0.046, +0.070]
+0.030 [+0.020, +0.039]
Best-of- N at the same NFE, Keystone, Jev
+0.056 [+0.044, +0.067]
+0.024 [+0.013, +0.034]
Best-of- N at the same NFE, HPSv2.1
+0.074 [+0.062, +0.087]
+0.067 [+0.055, +0.079]
Best-of- N at the same NFE, ImageReward
+0.071 [+0.059, +0.084]
+0.060 [+0.048, +0.073]
Best-of- N at the same NFE, VQAScore
+0.068 [+0.057, +0.081]
+0.054 [+0.042, +0.066]
Best-of- N at the same NFE, PickScore
+0.075 [+0.063, +0.087]
+0.065 [+0.053, +0.077]
Appendix
Table 13: With Qwen3-VL-8B as the judge, both Keystone searches are significantly more accurate than best-of- N at the same NFE with every verifier, and than the same search with VQAScore, HPSv2.1 or ImageReward . Overall GenEval accuracy on SD3.5-Medium; mean difference with its 95% paired bootstrap interval over prompts resampled within category. Best-of- N is taken at the same NFE, interpolated linearly between integer N .
One
Adaptive budget
Pruning budget
Benchmark
sample
BoN- k
Keystone
BoN-4
Keystone
GenEval (4 models)
0.63
0.69
0.76 †
0.71
0.77 †
T2I-CompBench (4 models)
0.64
0.70
0.73 †
0.71
0.74 †
Mean of the 8 pairs
0.64
0.69
0.74 †
0.71
0.76 †
GenAI-Bench (4 models)
0.76
0.78
0.79 †
0.78
0.80 †
Mean of the 12 pairs
0.68
0.72
0.76 †
0.73
0.77 †
Appendix
Table 14: Fig. 13 as a table, with GenAI-Bench: on every benchmark, averaged over the four base models, both of Keystone’s searches are significantly above best-of- N with the strongest other verifier (benchmark score, the mean over SD3.5-Medium, FLUX.1-dev, SDXL and Sana-0.6B; GenAI-Bench is scored by VQAScore, so its strongest other verifier is among the other four; one search over eight candidates per prompt; 40 steps; settings fixed on SD3.5-Medium). BoN- k : best-of- k with the strongest other verifier at the smallest k that costs at least as much as Keystone’s adaptive search; BoN-4 (160 NFE) likewise, against pruning (144 NFE). Adaptive search costs 71–103 NFE on GenEval and T2I-CompBench and 105–114 on GenAI-Bench ( k=2 for SD3.5-Medium and FLUX.1-dev on GenEval, 3 otherwise), and the strongest other verifier is VQAScore on GenEval and T2I-CompBench, except ImageReward for Sana-0.6B on GenEval, and ImageReward on GenAI-Bench. Bold: the better of the two in each budget; † : above that best-of- N , pooled paired 95% interval above zero.
Pairs ordered
Pruning, GenEval
Judge
at step 8
Overall
Position
ImageReward
0.56
0.75
0.33
HPSv2.1
0.57
0.76
0.30
One holistic question
0.67
0.81
0.51
Keystone score (ours)
0.70
0.82
0.57
Appendix
Table 16: Fig. 11 as a table: Keystone’s compiled questions lead every column (SD3.5-Medium, official test prompts). The same judge asked either Keystone’s compiled questions or one holistic question, next to both reward models, all on the same candidates. Pairs ordered correctly at step 8: share of candidate pairs for the same image, one correct and one not, that the step-8 preview’s score puts in the right order (ties count half), averaged over categories. Pruning: the schedule of Tab. 2 with each score.
N
NFE
HPSv3
Keystone score
Difference
2
80
0.725
0.744
+0.019 [+0.015, +0.023]
3
120
0.744
0.773
+0.029 [+0.023, +0.036]
4
160
0.755
0.791
+0.036 [+0.028, +0.044]
5
200
0.762
0.802
+0.040 [+0.031, +0.051]
6
240
0.766
0.810
+0.044 [+0.033, +0.055]
7
280
0.770
0.816
+0.046 [+0.033, +0.059]
Appendix
Table 17: The Keystone score is a significantly better verifier of best-of- N than HPSv3 at every N from 2 to 8 (SD3.5-Medium, official test prompts, the candidates of Tab. 2 ). Overall GenEval accuracy of best-of- N with each verifier at the same N and cost, and the difference, Keystone score minus HPSv3, with its 95% paired bootstrap interval over prompts resampled within category; bold: higher.
Row of Tab. 2
minus the same procedure with
Overall
Count.
Position
Best-of-2, Keystone score
VQAScore
+0.017 [+0.013, +0.020]
+0.031 [+0.019, +0.043]
+0.049 [+0.037, +0.061]
HPSv2.1
+0.025 [+0.020, +0.029]
+0.035 [+0.023, +0.046]
+0.089 [+0.073, +0.106]
ImageReward
+0.020 [+0.016, +0.024]
+0.054 [+0.041, +0.068]
+0.071 [+0.057, +0.087]
PickScore
+0.025 [+0.020, +0.029]
+0.029 [+0.019, +0.040]
+0.092 [+0.076, +0.109]
CLIP score
+0.036 [+0.031, +0.041]
+0.067 [+0.054, +0.080]
+0.087 [+0.070, +0.102]
Keystone, adaptive
adaptive search, VQAScore
+0.049 [+0.035, +0.062]
+0.081 [+0.033, +0.133]
+0.140 [+0.100, +0.180]
Appendix
Table 18: Every row of Tab. 2 with the Keystone score is significantly above its procedure with every other verifier, and pruning also above PSP and FK steering, overall and on counting and position (SD3.5-Medium, official test prompts). Mean difference with its 95% paired bootstrap interval over prompts; bold: above zero.
Keystone’s pruning from
minus best-of- K0 with each verifier
VQAScore
HPSv2.1
ImageReward
PickScore
CLIP score
K0=8
+0.039 [+0.026, +0.052]
+0.065 [+0.048, +0.081]
+0.052 [+0.038, +0.066]
+0.055 [+0.039, +0.070]
+0.079 [+0.063, +0.094]
K0=16
+0.048 [+0.029, +0.069]
+0.075 [+0.054, +0.096]
+0.068 [+0.047, +0.090]
+0.081 [+0.058, +0.103]
+0.095 [+0.074, +0.117]
K0=32
+0.048 [+0.019, +0.078]
+0.063 [+0.032, +0.092]
+0.069 [+0.038, +0.101]
+0.089 [+0.057, +0.120]
+0.090 [+0.061, +0.122]
Appendix
Table 19: Keystone’s pruning from K0 candidates beats best-of- K0 with every other verifier (SD3.5-Medium, official test prompts, the candidates of Fig. 9 ). Overall GenEval accuracy, pruning minus best-of- K0 , with its 95% paired bootstrap interval over prompts; bold: interval above zero.
Figure 17: From step 4 on, the Keystone score orders candidates better than VQAScore and both reward models reading the same image, and from step 12 its previews order them better than every other judge orders the finished images (SD3.5-Medium, counting and two-object prompts). (a) Development prompts, the same preview; (b) official test prompts, against the finished image. Band: 95% paired bootstrap interval over prompts; ∗ : interval above zero, as for all 39 comparisons. Values: the bottom block of Tab. 20 .
Tab. 1 , step-8 reading
Keystone score minus
Development prompts
Test prompts
Counting, preview against the final image
VQAScore
+0.034 [−0.019, +0.092]
+0.055 [+0.023, +0.087] ⋆
Colours
VQAScore
+0.059 [−0.048, +0.168]
+0.092 [+0.033, +0.143] ⋆
Colour attribution
HPSv2.1
+0.055 [−0.021, +0.133]
+0.069 [+0.034, +0.106] ⋆
Appendix
Table 20: The Keystone score leads in every comparison across categories and read steps, with every interval above zero on the official test prompts and in every comparison of Fig. 4 (SD3.5-Medium; paired bootstrap over prompts). Top: step-8 readings of Tab. 1 . Bottom: the values of Fig. 17 . Difference: the Keystone score minus the other judge, with its 95% interval; ⋆ : above zero.
Figure 18: In each example, Keystone’s score ranks the candidate that ends correct above the one that ends wrong at every preview from step 4 on (SD3.5-Medium, development prompts). Each pair of rows: two candidates of one prompt, their previews with the Keystone score under each and the final image with the verdict of GenEval’s official evaluator; indigo heads: pruning’s read steps. Selection: per category, the first development prompt whose eight candidates have mixed verdicts, with its highest-scoring correct and lowest-scoring wrong candidate at step 8 (of 41, 56 and 73 such prompts).
Figure 19: Here, a new noise draw at the first step leaves three of the six siblings correct and three wrong, and so does one at step 2; a new draw at any of steps 3–7 leaves all six correct (SD3.5-Medium, development prompt, the 10-step stochastic sampler of Fig. 4 ). Each row is one sibling group of Fig. 4 (left): six siblings share the starting noise and every noise draw except the one at the row’s step. Left: the preview they share just before that step; right: their final images with the verdict of GenEval’s official evaluator; shaded: the verdicts differ. Over all 40 prompt–seed pairs of Fig. 4 (left), the verdicts differ in 29 and 26 of the 40 groups at steps 1 and 2 and in at most 17 at each of steps 3–7. Selection: 9 of the 40 pairs split three to three at step 1; shown is the first of them in prompt order.
Figure 20: The same step-8 previews read by three judges: Keystone’s questions prefer the candidate that ends correct, ImageReward and HPSv2.1 the one that ends wrong (SD3.5-Medium, test prompts). Each row: two candidates of one image, with step-8 previews (framed: Keystone’s preference), each judge’s scores (bold: its preference) and final images with GenEval’s official verdict. Selection: of the 364, 647 and 578 candidate pairs (counting, position, colour attribution) with a correct and a wrong candidate of one image that the Keystone score at step 8 orders correctly and both reward models the other way, strictly, the first two prompts per category whose verdicts GPT-5.4 confirms ( Sec. C.3 ); per prompt, its first such image and pair.
Figure 21: Keystone’s pruning ends on a correct image where the same pruning driven by ImageReward ends on a wrong one (SD3.5-Medium, official GenEval test prompts, the candidates of Tab. 2 ). Each row is one image: its eight step-8 previews with their Keystone scores (framed: the four kept), the step-16 previews of those four with their scores (framed: the two kept), and the final image each pruning ends on, Keystone’s and that of ImageReward under the same schedule (correct/wrong: verdicts of GenEval’s official evaluator); equal scores go to the candidate ranked higher at the previous step, then to the earlier one. Selection: an image qualifies when, under every order of tied scores at every cut, Keystone’s pruning ends on a correct image and ImageReward’s on a wrong one; qualifying images per category: counting 47 (in 30 prompts), position 104 (in 62), colour attribution 39 (in 31), two objects 9 (in 7), colours 13 (in 13), single object 1 (in 1); with ties broken in the fixed order of Fig. 7 , the first three categories give 198. An image is shown only if GPT-5.4 ( Sec. C.3 ) agrees: it calls Keystone’s final image a match for the prompt and ImageReward’s not. Each prompt contributes its first such image, the prompts of Fig. 7 and Fig. 16 are left out, and the rows are the first eight prompts of each category by prompt index (or fewer where fewer remain). Continued in Fig. 22 .
Figure 22: Keystone’s pruning ends on a correct image where the same pruning driven by ImageReward ends on a wrong one, continued (position, colour attribution, two objects and colours; setting, layout and selection rule as in Fig. 21 ).
Figure 23: Examples on FLUX.1-dev and SDXL where Keystone’s pruning ends on a correct image and best-of-4 picks a wrong one (GenEval, official verdicts). Pruning (144 NFE) against best-of-4 (160 NFE) with the strongest other verifier, VQAScore on both models. Qualifying prompts: 33 for FLUX.1-dev and 58 for SDXL; shown: the first three of each that GPT-5.4 confirms. Thumbnails: finished images, labelled with the Keystone scores of their step-8 and step-16 previews (layout and rule: Sec. D.9 ).
Figure 24: Examples on Sana-0.6B and SD3.5-Medium where Keystone’s pruning ends on the image the benchmark scores higher . Pruning (144 NFE) against best-of-4 (160 NFE) with the strongest other verifier. Sana-0.6B on T2I-CompBench: metric BLIP-VQA or UniDet by category, verifier VQAScore, 31 prompts qualify. SD3.5-Medium on GenAI-Bench: metric VQAScore, verifier ImageReward, 21 qualify. Shown: the first three of each that GPT-5.4 confirms; thumbnails (finished images), layout and rule as in Fig. 23 .
Decision model
Overall
Single
Two obj.
Count.
Colours
Pos.
Col. attr.
$ / 1k calls
Jev
0.71
0.66
0.73
0.88
0.62
0.77
0.58
0.065
GPT-5.4
0.67
0.57
0.69
0.81
0.62
0.77
0.56
1.43
Qwen3-VL-8B
0.67
0.64
0.69
0.82
0.62
0.66
0.57
local
Claude Sonnet 4.6
0.66 ∗
0.52
0.70
0.84
0.62
0.71
0.55
1.83
Qwen3-VL-235B-A22B
0.64 ∗
0.61
0.67
0.77
0.59
0.69
0.53
0.27
Jev, typed questions (as in the paper)
0.71
0.66
0.73
0.88
0.62
0.81
0.56
0.065
Appendix
Table 21: Asked the same questions, Jev orders step-8 previews best, at a fraction of the cost per call (SD3.5-Medium, official test prompts, the first of the four images per prompt: 553 images of eight candidates each, so the numbers differ from Tab. 2 ). Pairs ordered correctly at step 8: the share of candidate pairs with different official verdicts that the score orders correctly, ties counted half, per category and averaged over the six. Every model answers the yes/no questions; the last row is Jev with Keystone’s typed questions, as in the paper. Each API model answers all questions about an image in one call, as whole-number probabilities from 0 to 100 (5–20% of its pairs tie); GPT-5.4 is run at its default sampling; Claude Sonnet 4.6 adds free-text reasoning before its answers in 5% of replies. Bold: best among the yes/no rows; ∗ : below Jev, paired 95% bootstrap interval of the difference above zero. Cost per 1,000 calls: measured tokens at Bedrock’s list price for the API models, the billed cost for Jev; Qwen3-VL-8B runs locally.
Diffusion models can produce striking images and videos, but they still struggle with the compositional details that make a generation faithful to a prompt, such as object counts, attribute binding, spatial relations, and temporally grounded actions. A common way to improve prompt satisfaction is to spend more compute at test time through Best-of-N sampling, but final-sample selection is fixed. Best-of-N can only choose among completed outputs and cannot repair a promising trajectory before it fails. We introduce PreviewDiff, a training-free test-time search method that turns diffusion sampling from scalar search into a multimodal critic-guided search over intermediate latents. At selected denoising checkpoints, PreviewDiff decodes a partial preview, asks a multimodal judge to score and critique it, and uses the resulting natural-language feedback to branch over semantic prompt edits and locally re-noised latent continuations. These branches are then scored and selectively rolled forward, allowing verifier compute to guide generation while the sample is still editable. Across image and video generation benchmarks, PreviewDiff consistently improves over budget-matched Best-of-N selection and strong scalar-search baselines. Ablations show that earlier interventions and increased search width provide the largest gains, while deeper search and additional semantic variants offer complementary improvements. PreviewDiff demonstrates that multimodal feedback is most useful not only as a final verifier, but as an active controller inside the denoising process.
Recent text-to-image (T2I) diffusion and flow-matching models can produce highly realistic images from natural language prompts. In practical scenarios, T2I systems are often run in a ``generate--then--select'' mode: many seeds are sampled and only a few images are kept for use. However, this pipeline is highly resource-intensive since each candidate requires tens to hundreds of denoising steps, and evaluation metrics such as CLIPScore and ImageReward are post-hoc. In this work, we address this inefficiency by introducing Probe-Select, a plug-in module that enables efficient evaluation of image quality within the generation process. We observe that certain intermediate denoiser activations, even at early timesteps, encode a stable coarse structure, object layout and spatial arrangement--that strongly correlates with final image fidelity. Probe-Select exploits this property by predicting final quality scores directly from early activations, allowing unpromising seeds to be terminated early. Across diffusion and flow-matching backbones, our experiments show that early evaluation at only 20% of the trajectory accurately ranks candidate seeds and enables selective continuation. This strategy reduces sampling cost by over 60% while improving the quality of the retained images, demonstrating that early structural signals can effectively guide selective generation without altering the underlying generative model. Code is available at https://github.com/Guhuary/ProbeSelect.
Huanlei Guo, Hongxin Wei, Bingyi Jing
Southern University of Science and Technology · The Chinese University of Hong Kong, Shenzhen · Shenzhen Loop Area Institute
Diffusion and flow-matching models dominate conditional image generation, yet inference-time scaling for these models is far less developed than for autoregressive language models. Because final quality is highly sensitive to the initial noise seed, many approaches spend extra compute on seed search or resampling under a black-box reward, but typically maintaining a constant memory footprint throughout inference. We show that relaxing this constraint enables an underexplored inference-time scaling axis: by front-loading exploration, evaluating many seeds early, and pruning aggressively, we can use a fixed compute budget more effectively. Progressive Seed Pruning (\PSP) scores intermediate denoised estimates and progressively narrows the candidate set so that only promising trajectories are fully denoised, while keeping the total number of model evaluations fixed. Across diffusion and flow-matching backbones, \PSP \ consistently improves reward-guided selection and achieves higher GenEval scores (automated) and better human evaluation on prompt-alignment than best-of-N, importance-sampling, and tree-search baselines at matched compute. Project page: https://www.vision.caltech.edu/psp. Code: https://github.com/rogerioagjr/psp.
Rogerio Guimaraes, Pietro Perona
California Institute of Technology, Pasadena CA 91125, USA