Tiny Inference-Time Scaling with Latent Verifiers

Davide Bucciarelli; Evelyn Turri; Lorenzo Baraldi; Marcella Cornia; Lorenzo Baraldi; Rita Cucchiara

cs.CVMar 23, 2026

Tiny Inference-Time Scaling with Latent Verifiers

Authors: Davide Bucciarelli, Evelyn Turri, Lorenzo Baraldi, Marcella Cornia, Lorenzo Baraldi, Rita Cucchiara

Organizations: University of Modena and Reggio Emilia, Italy · University of Pisa, Italy

Abstract

Inference-time scaling has emerged as an effective way to improve generative models at test time by using a verifier to score and select candidate outputs. A common choice is to employ Multimodal Large Language Models (MLLMs) as verifiers, which can improve performance but introduce substantial inference-time cost. Indeed, diffusion pipelines operate in an autoencoder latent space to reduce computation, yet MLLM verifiers still require decoding candidates to pixel space and re-encoding them into the visual embedding space, leading to redundant and costly operations. In this work, we propose Verifier on Hidden States (VHS), a verifier that operates directly on intermediate hidden representations of Diffusion Transformer (DiT) single-step generators. VHS analyzes generator features without decoding to pixel space, thereby reducing the per-candidate verification cost while improving or matching the performance of MLLM-based competitors. We show that, under tiny inference budgets with only a small number of candidates per prompt, VHS enables more efficient inference-time scaling reducing joint generation-and-verification time by 63.3%, compute FLOPs by 51% and VRAM usage by 14.5% with respect to a standard MLLM verifier, achieving a +2.7% improvement on GenEval at the same inference-time budget.

Figures & tables

Figure 1: (A) Comparison between standard inference-time scaling and VHS. VHS skips part of the generation pipeline and avoids the decoding and re-encoding steps. (B) VHS achieves a comparable quality score on GenEval [ 10 ] in just 57% of the compute time.

Figure 2: Comparison between a standard generation-verification pipeline (top) and VHS (bottom). VHS consumes visual features directly from the hidden states of the generator, bypassing subsequent DiT layers, autoencoder (AE) decoding, and CLIP-based re-encoding, significantly reducing sampling and verification overhead.

	Inference Time (ms)					TFLOPs					Peak VRAM Usage (GB)
	Saved (%)	$\mathsf{Bo1}$	$\mathsf{Bo2}$	$\mathsf{Bo4}$	$\mathsf{Bo6}$	Saved (%)	$\mathsf{Bo1}$	$\mathsf{Bo2}$	$\mathsf{Bo4}$	$\mathsf{Bo6}$	Saved (%)	$\mathsf{Bo1}$	$\mathsf{Bo2}$	$\mathsf{Bo4}$	$\mathsf{Bo6}$
MLLM w/ CLIP	-	277	554	1108	1662	-	15.1	28.5	55.1	81.8	-	13.8	15.5	18.8	22.2
MLLM w/ AE	50.2%	138	401	677	953	51.0%	7.4	14.8	29.5	44.3	14.5%	11.8	11.9	12.3	12.6
VHS on $h_{7}$	63.3%	102	363	565	767	62.9%	5.6	11.3	22.5	33.8	14.5%	11.8	11.9	12.3	12.6

Table 1: End-to-end inference time, FLOPs and VRAM usage for Best-of-

N

generation with SANA-Sprint [ 6 ] under different computational budgets, along with the relative savings (%) compared to the standard verifier.

Budget	Generator	Steps	Verifier	Best-of-N	Single	Two	Counting	Color	Position	Attribution	Overall
200ms	SANA-Sprint	1	-	Best-of-1	99.3	88.1	56.0	87.6	54.1	47.8	71.6
550ms	SANA-1.5	4	-	Best-of-1	98.8	78.2	66.5	71.1	50.6	20.8	63.0
	SANA-Sprint	8	-	Best-of-1	99.5	91.9	59.3	86.0	57.8	52.4	74.0
	SANA-Sprint	1	MLLM w/ CLIP	Best-of-2	100.0	91.3	59.5	88.0	61.0	55.4	75.4
	SANA-Sprint	1	MLLM w/ AE	Best-of-3	100.0	90.9	59.0	89.6	55.8	50.6	73.1
	SANA-Sprint	1	VHS (Ours)	Best-of-4	100.0	93.9	61.5	90.6	66.2	58.4	78.1

Table 2: Accuracy (%) on the GenEval benchmark [ 10 ] across computational budgets, generator backbones, and verifier configurations (on LLM Qwen2.5-0.5B). Results compare SANA-1.5 and SANA-Sprint [ 6 ] under matched wall-clock budgets (milliseconds), with each verifier operating under the same time constraint via adaptive Best-of-

N

VHS
Verifier	Single	Two	Counting	Color	Position	Attribution	Overall
w/ $h_{1}$ Weighted XE Loss	99.7	88.1	56.8	87.8	51.8	48.2	71.3
w/ $h_{5}$ Weighted XE Loss	100.0	92.3	58.5	90.0	66.6	59.4	77.7
w/ $h_{9}$ Weighted XE Loss	100.0	93.1	65.5	88.9	65.0	59.8	78.3
w/ $h_{19}$ Weighted XE Loss	99.7	88.1	61.7	88.5	65.6	57.8	76.5
w/ $h_{7}$ XE Loss	100.0	90.9	59.5	88.1	61.6	60.4	76.3

Table 3: Accuracy (%) of SANA-Sprint [ 6 ] on the GenEval benchmark [ 10 ] across varying hidden layers, training losses, training data, and reference LLMs on a time budget of 1100 ms.

Figure 3: Visual comparison of the best pick images by different verifiers for GenEval-generated images.

	Verification Time		GenEval Overall (%)
	t (ms)	t savings (%)	$\mathsf{Bo2}$	$\mathsf{Bo3}$	$\mathsf{Bo4}$	$\mathsf{Bo5}$	$\mathsf{Bo6}$
MLLM w/ CLIP	145	-	43.7	44.7	45.1	45.6	46.9
MLLM w/ AE	165	$-$ 14.0	41.0	41.0	41.9	42.3	41.6
VHS on $h_{13}$	76	48.0	43.0	45.2	45.5	46.1	46.4

Table 4: Verification latency and GenEval [ 10 ] accuracy scores for Best-of-

N

generation with PixArt-

\alpha

[ 5 , 50 ] under equivalent computational budgets defined by MLLM w/ CLIP.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

VHS
Verifier	Single	Two	Counting	Color	Position	Attribution	Overall
w/ $h_{7}$ No Scoring	99.3	90.7	57.5	87.5	63.2	52.6	74.7
w/ $h_{7}$ Token Probability Scoring (Ours)	100.0	95.7	66.5	88.9	69.8	63.8	80.5
MLLM w/ CLIP
w/ No Scoring	99.5	91.1	58.3	88.3	62.0	54.8	75.3
w/ Token Probability Scoring	100.0	92.7	66.0	88.9	65.9	61.6	78.8

Appendix

Table 5: Accuracy (%) of SANA-Sprint [ 6 ] on the GenEval benchmark [ 10 ] . Ablation on score methodology on a time budget of 1100 ms.

Budget	Generator	Steps	Verifier	$\mathsf{BoN}$	Single	Two	Count	Color	Pos	Attr	All
550ms	SD3.5-Turbo	1	MLLM w/ CLIP	$\mathsf{Bo1}$	83.0	29.9	35.5	61.1	11.4	13.6	37.4
	Flux-Schnell	1	MLLM w/ CLIP	$\mathsf{Bo1}$	99.0	90.3	60.5	82.8	28.0	58.4	68.9
	Sana-Sprint	1	VHS (Ours)	$\mathsf{Bo4}$	100.0	93.9	61.5	90.6	66.2	58.4	78.1
1100ms	SD3.5-Turbo	1	MLLM w/ CLIP	$\mathsf{Bo2}$	90.1	35.6	37.8	63.8	13.8	16.6	41.2
	Flux-Schnell	1	MLLM w/ CLIP	$\mathsf{Bo2}$	99.8	94.3	65.8	84.7	34.4	65.0	73.2
	Flux-Schnell	2	MLLM w/ CLIP	$\mathsf{Bo2}$	99.0	91.7	61.3	80.9	28.4	57.4	68.9

Appendix

Table 6: GenEval benchmark [ 10 ] results with compute-equivalent baselines.

Figure 4: Efficient Best-of-N pipeline with VHS.

No LoRA Fine-Tuning
Verifier	Single	Two	Counting	Color	Position	Attribution	Overall	$\Delta$
MLLM w/ CLIP	100.0	94.7	59.3	91.9	69.4	60.2	79.1	$+$ 1.4
VHS	100.0	95.7	66.5	88.9	69.8	63.8	80.5	$+$ 1.4
50% LoRA Fine-Tuning
MLLM w/ CLIP	100.0	92.9	60.8	90.6	63.4	58.2	77.3	$+$ 1.8
VHS	100.0	92.1	61.5	89.8	70.4	62.0	79.1	$+$ 1.8

Appendix

Table 7: Accuracy (%) of SANA-Sprint [ 6 ] on the GenEval benchmark [ 10 ] , varying levels of LoRA Fine-Tuning on a time budget of 1100 ms for MLLM w/ CLIP and VHS.

Figure 5: Overall accuracy (%) of SANA-Sprint [ 6 ] on GenEval [ 10 ] across time (seconds) TFLOPs, and VRAM usage (GB).

Figure 6: Score distribution comparison between MLLM w/ CLIP and VHS. Histograms display the frequency of scores within 5% bins, with smoothed density curves overlaid in gray (MLLM w/ CLIP) and pink (VHS). Box plots above show the quartile ranges and distributional characteristics of each verifier.

Figure 7: Qualitative results of captioning produced by VHS and MLLM w/ CLIP after the alignment training phase.

Figure 8: Cases where judgments from VHS is not aligned with the GenEval verifier.

Figure 9: Visual comparison of the best pick images by different verifiers for images generated by SANA-Sprint [ 6 ] on GenEval [ 10 ] prompts. $\checkmark$ and $\times$ express the alignment with the GenEval verifier.

Inference-Time Scaling

Inference-time scaling studies how extra computation during deployment can improve model performance through search, sampling, verification, or adaptive reasoning.

See all 115 papers

Imperfect Verifiers, Exact Targets: Inference-Time Scaling for Continuous Generation
Oct 11, 2026Kijung Jeon, Molei TaoInference-Time Guidance Diffusion Sampling
Georgia Institute of Technology
ES-dLLM: Efficient Inference for Diffusion Large Language Models by Early-Skipping
Mar 10, 2026Zijian Zhu, Fei Ren, Zhanhong Tan +1Diffusion Language Model Inference Diffusion Language Models
IIIS, Tsinghua University · Polar Bear Tech.
Inference-Time Scaling in Diffusion Models through Iterative Partial Refinement
May 19, 2026Taegu Kang, Jaesik Yoon, Sungjin AhnInference-Time Scaling Diffusion Models
KAIST · KAIST & SAP · KAIST & NYU

Oct 11, 2026J/K move · Enter open · S save

Imperfect Verifiers, Exact Targets: Inference-Time Scaling for Continuous Generation

Kijung Jeon, Molei Tao

Georgia Institute of Technology

Inference-Time Guidance Diffusion Sampling

Open paper AskPDF arXiv

Verifier-guided inference-time scaling uses additional computation to score intermediate states and steer generation toward better outputs. However, imperfect intermediate verifiers can distort sampling, and repeated refinement alone does not ensure recovery of the desired reward-tilted distribution. We introduce Continuous-State Verifier-Guided Backtracking (CS-VGB), an inference-time scaling method for sampling from reward-tilted distributions with continuous generative models through repeated denoising and renoising. With correctly paired denoising and renoising transitions, the stationary distribution of CS-VGB at the clean checkpoint equals the desired reward-tilted target, even when intermediate verifiers are inaccurate. Under stated regularity conditions, CS-VGB converges exponentially to the target despite verifier error, and the expected number of updates needed to reach a clean sample grows only linearly with the number of checkpoints. In analytically tractable settings, we empirically demonstrate that CS-VGB recovers the desired reward-tilted distribution despite verifier error, whereas the compared verifier-based sampling methods fail to recover the target under the same verifier errors. We also instantiate the framework with diffusion models and one-step stochastic posterior maps. Across prompt-guided ImageNet generation, facial-attribute generation on CelebA, and multi-robot planning, recurrent CS-VGB search improves reward and task success as inference budgets grow, with image-generation rewards continuing beyond the observed plateaus of denoising-only baselines.

Abstract

Inference-Time Scaling

Explore similar work

Imperfect Verifiers, Exact Targets: Inference-Time Scaling for Continuous Generation

ES-dLLM: Efficient Inference for Diffusion Large Language Models by Early-Skipping

Inference-Time Scaling in Diffusion Models through Iterative Partial Refinement