FoMo: Forking Moment in Generative Trajectory as a Perceptual Distance

Jaihyun Lew; Mingi Jung; Minjun Park; Wooseok Song; Sungroh Yoon

cs.CVSep 22, 2026

FoMo: Forking Moment in Generative Trajectory as a Perceptual Distance

Authors: Jaihyun Lew, Mingi Jung, Minjun Park, Wooseok Song, Sungroh Yoon

Organizations: Interdisciplinary Program in AI, Seoul National University · Department of Electrical and Computer Engineering, Seoul National University · AIIS, ASRI, INMC, and ISRC, Seoul National University

Abstract

Reference-based image quality assessment (IQA) metrics aim to reflect how humans perceive the perceptual distance between a pair of images. To learn how the human visual system (HVS) operates, recent reference-based IQA metrics heavily rely on human-annotated data. Mean opinion score (MOS)-based pointwise scoring, which assigns a scalar quality value per image, is preferable for annotation but is prohibitively expensive to collect at scale and is known to be noisy due to inconsistent human judgments. As an alternative, two-alternative forced choice (2AFC) pairwise labels have gained popularity due to their reliability and efficiency, but they capture only relative comparisons between pairs. In this paper, we propose a fully automated data generation pipeline that generates pointwise perceptual distance labels between image pairs without any human annotation. Our approach exploits the generative dynamics of diffusion models as a perceptual distance proxy, where the coarse structure of an image is generated in the early timesteps and the fine details are generated in the later timesteps. Images that fork early in the generation process share only coarse structure and are perceptually far apart; images that fork late differ only in fine detail. We demonstrate that the diffusion trajectory aligns well with the human visual system, and use this forking moment, FoMo, as a reference-grounded distance label to supervise the training of a reference-based IQA metric. The pointwise labels, which support universal comparison between arbitrary image pairs, enable an information-rich training objective. Extensive experiments across diverse backbone architectures confirm the effectiveness of our generation pipeline, outperforming human-annotated datasets in multiple benchmarks.

Figures & tables

Panel of Figure 1 — Figure 1: (a) Single-reference study: participants rank synthesized variants to verify whether the divergence timestep in diffusion generative trajectory reflects the degree of perceptual similarity to the reference image. (b) Cross-reference study: participants are asked which of two pairs, built from different references, holds the two more similar images. Further examples in the Appendix.

Figure 2: Visualization of our overall framework. (Left) Data generation trajectories, where the divergence point of the denoising trajectory (solid arrows) serves as a perceptual distance proxy. (Right) Unlike the traditional 2AFC framework, limited to within-anchor comparisons, our pointwise scoring system enables inter-anchor comparisons.

Figure 3: Data sample image pairs and their labels from our constructed dataset. $s$ denote the forking step (smaller-the-further), or the distance label of the image with respect to the reference image.

Train Data	Human Annot.	Objective	CNN-based			Transformer-based				Avg.
Train Data	Human Annot.	Objective	LPIPS-Alex	LPIPS-VGG	DISTS	DINOv3	CLIP	MAE	DreamSim †	Avg.
Results on PIPAL [ 18 ]
BAPPS [ 43 ]	✓	2AFC	0.622	0.634	0.489	0.287	0.151	0.338	0.760	0.469
PieAPP [ 31 ]	✓	2AFC	0.602	0.635	0.564	0.325	0.258	0.294	0.711	0.484
NIGHTS [ 13 ]	✓	2AFC	0.577	0.545	-0.294	0.234	0.277	0.234	0.662	0.319
KADID-10K [ 23 ]	✓	L1	0.577	0.590	0.518	0.298	0.312	0.253	0.617	0.452

Table 1: Comparison of training datasets and their objectives on four reference-based IQA benchmarks, across CNN- and Transformer-based backbones. Mean SROCC over five random seeds; standard deviations are given in Table 6 . † Evaluated under 224 resolution.

	FoMo (Ours)						KADID-10K [ 23 ]
Label	$t$			$s$			DMOS
Objective	Rank	2AFC	L1	Rank	2AFC	L1	Rank	2AFC	L1
LPIPS-Alex [ 43 ]	0.733	0.592	0.440	0.733	0.592	0.390	0.625	0.611	0.577
DINOv3 [ 35 ]	0.699	0.622	0.694	0.697	0.614	0.695	0.309	0.145	0.298

Table 2: Comparative experiment on the objective function and label type. Ranked binary cross-entropy loss (Rank) consistently shows stronger results, compared to triplet-based paired cross-entropy loss (2AFC) and L1 regression loss on the label (L1). Our label has shown to be more helpful in training, compared to the large-scale human annotated KADID-10k. Mean over five random seeds; full numbers with standard deviations can be found in Table 9 .

Backbone	Batch size
Backbone	16	32	64	128	256
LPIPS-Alex [ 43 ]	0.719 $\pm$ 0.006	0.729 $\pm$ 0.006	0.733 $\pm$ 0.006	0.728 $\pm$ 0.006	0.712 $\pm$ 0.006
DINOv3 [ 35 ]	0.479 $\pm$ 0.080	0.650 $\pm$ 0.018	0.696 $\pm$ 0.007	0.705 $\pm$ 0.003	0.707 $\pm$ 0.005

Table 3: Effect of batch size on training performance. The Transformer-based backbone improves monotonically with batch size, while the CNN-based backbone saturates at 64 and degrades slightly beyond it. Mean

\pm

standard deviation over five random seeds.

Backbone	Data Generator (10k pairs)
Backbone	SD-1.5 [ 33 ]	SD-XL [ 29 ]	SD-3 [ 11 ]	FLUX.1 [ 20 ]
LPIPS-Alex [ 43 ]	0.687 $\pm$ 0.008	0.662 $\pm$ 0.008	0.695 $\pm$ 0.007	0.741 $\pm$ 0.002
DINOv3 [ 35 ]	0.574 $\pm$ 0.078	0.418 $\pm$ 0.044	0.517 $\pm$ 0.017	0.672 $\pm$ 0.016

Table 4: Experimental results using diverse diffusion models for data generation. Our pipeline is not specific to FLUX, and every generator yields a working metric, although FLUX.1 is the strongest choice. Cells report mean

\pm

standard deviation over five train runs, each using disjoint 10k subsets from the generator.

Backbone	All	Sampling Range of Forking Timesteps
Backbone	$[0,50]$	$[0,25]$	$[0,35]$	$[13,37]$	$[15,50]$	$[25,50]$
LPIPS-Alex [ 43 ]	0.733 $\pm$ 0.006	0.748 $\pm$ 0.008	0.757 $\pm$ 0.007	0.729 $\pm$ 0.008	0.731 $\pm$ 0.007	0.720 $\pm$ 0.006
LPIPS-VGG [ 43 ]	0.683 $\pm$ 0.008	0.688 $\pm$ 0.005	0.697 $\pm$ 0.006	0.685 $\pm$ 0.004	0.666 $\pm$ 0.006	0.626 $\pm$ 0.005
DISTS [ 9 ]	0.615 $\pm$ 0.001	0.464 $\pm$ 0.002	0.583 $\pm$ 0.001	0.569 $\pm$ 0.002	0.556 $\pm$ 0.001	0.485 $\pm$ 0.003
DINOv3 [ 35 ]	0.703 $\pm$ 0.006	0.620 $\pm$ 0.015	0.696 $\pm$ 0.015	0.700 $\pm$ 0.009	0.702 $\pm$ 0.015	0.680 $\pm$ 0.014

Table 5: Experiment on different sampling ranges of forking timesteps. At a matched window width the earlier window is the most useful one, and the full schedule overall provides the most stable result. Mean

\pm

standard deviation over five random seeds.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Train Data	CNN-based			Transformer-based				Avg.
Train Data	LPIPS-Alex	LPIPS-VGG	DISTS	DINOv3	CLIP	MAE	DreamSim †	Avg.
Results on PIPAL [ 18 ]
BAPPS [ 43 ]	0.622 $\pm$ 0.005	0.634 $\pm$ 0.018	0.489 $\pm$ 0.003	0.287 $\pm$ 0.020	0.151 $\pm$ 0.040	0.338 $\pm$ 0.013	0.760 $\pm$ 0.007	0.469 $\pm$ 0.008
PieAPP [ 31 ]	0.602 $\pm$ 0.008	0.635 $\pm$ 0.016	0.564 $\pm$ 0.003	0.325 $\pm$ 0.017	0.258 $\pm$ 0.037	0.294 $\pm$ 0.023	0.711 $\pm$ 0.007	0.484 $\pm$ 0.007
NIGHTS [ 13 ]	0.577 $\pm$ 0.006	0.545 $\pm$ 0.011	-0.294 $\pm$ 0.003	0.234 $\pm$ 0.024	0.277 $\pm$ 0.025	0.234 $\pm$ 0.029	0.662 $\pm$ 0.019	0.319 $\pm$ 0.005
KADID-10K [ 23 ]	0.577 $\pm$ 0.025	0.590 $\pm$ 0.034	0.518 $\pm$ 0.002	0.298 $\pm$ 0.010	0.312 $\pm$ 0.018	0.253 $\pm$ 0.041	0.617 $\pm$ 0.003	0.452 $\pm$ 0.006

Appendix

Table 6: Comparison of training datasets and their objectives on four reference-based IQA benchmarks, across CNN- and Transformer-based backbones. Mean

\pm

standard deviation of SROCC over five random seeds; full-statistics version of Table 1 . † Evaluated under 224 resolution.

Train Data	CNN-based			Transformer-based				Avg.
Train Data	LPIPS-Alex	LPIPS-VGG	DISTS	DINOv3	CLIP	MAE	DreamSim †	Avg.
Results on PIPAL [ 18 ]
BAPPS [ 43 ]	0.440 $\pm$ 0.004	0.455 $\pm$ 0.014	0.337 $\pm$ 0.002	0.194 $\pm$ 0.014	0.101 $\pm$ 0.027	0.229 $\pm$ 0.009	0.564 $\pm$ 0.007	0.332 $\pm$ 0.006
PieAPP [ 31 ]	0.420 $\pm$ 0.007	0.450 $\pm$ 0.013	0.389 $\pm$ 0.002	0.222 $\pm$ 0.013	0.175 $\pm$ 0.026	0.197 $\pm$ 0.016	0.511 $\pm$ 0.006	0.338 $\pm$ 0.005
NIGHTS [ 13 ]	0.405 $\pm$ 0.005	0.385 $\pm$ 0.008	-0.204 $\pm$ 0.002	0.159 $\pm$ 0.017	0.186 $\pm$ 0.017	0.157 $\pm$ 0.020	0.471 $\pm$ 0.018	0.223 $\pm$ 0.004
KADID-10K [ 23 ]	0.403 $\pm$ 0.020	0.419 $\pm$ 0.027	0.360 $\pm$ 0.001	0.204 $\pm$ 0.007	0.212 $\pm$ 0.012	0.170 $\pm$ 0.028	0.440 $\pm$ 0.003	0.315 $\pm$ 0.005

Appendix

Table 7: Comparison of training datasets and their objectives on four reference-based IQA benchmarks, across CNN- and Transformer-based backbones. Mean

\pm

standard deviation of KROCC over five random seeds. † Evaluated under 224 resolution.

Train Data	CNN-based			Transformer-based				Avg.
Train Data	LPIPS-Alex	LPIPS-VGG	DISTS	DINOv3	CLIP	MAE	DreamSim †	Avg.
Results on PIPAL [ 18 ]
BAPPS [ 43 ]	0.659 $\pm$ 0.005	0.676 $\pm$ 0.014	0.495 $\pm$ 0.002	0.314 $\pm$ 0.015	0.186 $\pm$ 0.043	0.348 $\pm$ 0.018	0.779 $\pm$ 0.007	0.494 $\pm$ 0.010
PieAPP [ 31 ]	0.607 $\pm$ 0.010	0.637 $\pm$ 0.014	0.558 $\pm$ 0.003	0.364 $\pm$ 0.014	0.284 $\pm$ 0.029	0.338 $\pm$ 0.025	0.707 $\pm$ 0.005	0.499 $\pm$ 0.003
NIGHTS [ 13 ]	0.634 $\pm$ 0.006	0.605 $\pm$ 0.007	0.361 $\pm$ 0.002	0.296 $\pm$ 0.028	0.324 $\pm$ 0.030	0.259 $\pm$ 0.037	0.669 $\pm$ 0.022	0.450 $\pm$ 0.008
KADID-10K [ 23 ]	0.599 $\pm$ 0.022	0.623 $\pm$ 0.034	0.562 $\pm$ 0.001	0.351 $\pm$ 0.021	0.346 $\pm$ 0.019	0.274 $\pm$ 0.034	0.619 $\pm$ 0.007	0.482 $\pm$ 0.008

Appendix

Table 8: Comparison of training datasets and their objectives on four reference-based IQA benchmarks, across CNN- and Transformer-based backbones. Mean

\pm

standard deviation of PLCC over five random seeds. † Evaluated under 224 resolution.

	FoMo (Ours)						KADID-10K [ 23 ]
Label	$t$			$s$			DMOS
Objective	Rank	2AFC	L1	Rank	2AFC	L1	Rank	2AFC	L1
LPIPS-Alex [ 43 ]	0.733 $\pm$ 0.006	0.592 $\pm$ 0.002	0.440 $\pm$ 0.059	0.733 $\pm$ 0.006	0.592 $\pm$ 0.002	0.390 $\pm$ 0.049	0.625 $\pm$ 0.004	0.611 $\pm$ 0.003	0.577 $\pm$ 0.025
DINOv3 [ 35 ]	0.699 $\pm$ 0.006	0.622 $\pm$ 0.020	0.694 $\pm$ 0.008	0.697 $\pm$ 0.006	0.614 $\pm$ 0.009	0.695 $\pm$ 0.013	0.309 $\pm$ 0.030	0.145 $\pm$ 0.010	0.298 $\pm$ 0.010

Appendix

Table 9: Comparative experiment on the objective function and label type. Ranked binary cross-entropy loss (Rank) consistently shows stronger results, compared to triplet-based paired cross-entropy loss (2AFC) and L1 regression loss on the label. Our label has shown to be more helpful in training, compared to the large-scale human annotated KADID-10k [ 23 ] . PIPAL SROCC, mean

\pm

standard deviation over five random seeds.

Figure 4: Screenshot of the human annotation website for experiments in Sec. 3

Figure 5: Screenshot of the human annotation website for experiments in Sec. 3

Figure 6: Examples from cross-reference human study, with each samples’ forked moments, the gap of forking moments and labels from FoMo provided.

Figure 7: Examples from cross-reference human study, with each samples’ forked moments, the gap of forking moments and labels from FoMo provided.

		LPIPS-Alex [ 43 ]		LPIPS-VGG [ 43 ]		DISTS [ 9 ]		DreamSim [ 13 ]
		Released	FoMo	Released	FoMo	Released	FoMo	Released	FoMo
PIPAL [ 18 ]	SROCC	0.620	0.733	0.612	0.683	0.672	0.615	0.759	0.776
	KROCC	0.435	0.536	0.438	0.497	0.482	0.435	0.563	0.571
	PLCC	0.623	0.767	0.668	0.712	0.685	0.654	0.780	0.762
TID2013 [ 30 ]	SROCC	0.745	0.785	0.670	0.663	0.708	0.691	0.812	0.801
	KROCC	0.548	0.586	0.497	0.489	0.521	0.512	0.614	0.605

Appendix

Table 10: Released off-the-shelf metrics compared with the same architecture trained with FoMo supervision. Better of each pair in bold . Released checkpoints are single deterministic models whereas FoMo columns are 5-seed means.

Forking step $s$	LPIPS-Alex			DISTS
Forking step $s$	mean $d$	std	CoV	mean $d$	std	CoV
5	0.692	0.043	6.4%	0.391	0.034	8.7%
15	0.518	0.046	9.1%	0.306	0.031	10.1%
25	0.371	0.030	7.9%	0.225	0.021	9.2%
35	0.233	0.011	4.7%	0.150	0.011	6.9%
45	0.090	0.002	2.7%	0.067	0.004	5.6%

Appendix

Table 11: Spread of the measured distance across generation seeds. 120 references

\times

5 forking steps

\times

K=8

seeds. For each reference we take the eight distances obtained at one forking step and compute their mean and standard deviation; the table reports these averaged over the 120 references, with CoV their ratio. A larger

s

denotes a later fork, and hence a variant closer to the reference.

Benchmark	types	LPIPS-Alex			DINOv3
Benchmark	types	best baseline	FoMo	led	best baseline	FoMo	led
PIPAL	7	0.611	0.681	7/7	0.259	0.525	7/7
TID2013	24	0.794	0.783	10/24	0.592	0.710	22/24
CSIQ	6	0.945	0.936	1/6	0.613	0.808	6/6
LIVE	5	0.952	0.942	2/5	0.748	0.896	5/5

Appendix

Table 12: Per-distortion-type summary on all four benchmarks. “Best baseline” is the strongest of the four human-annotated recipes for that backbone on that benchmark, selected independently per benchmark. “Full” is the score over the whole benchmark and “led” the number of distortion types on which FoMo is ahead. PIPAL uses its training split, the only one that exposes distortion types. Symmetrized native-resolution protocol, seed-0 checkpoints.

Distortion type	LPIPS-Alex		DINOv3
Distortion type	best baseline	FoMo ( Ours )	best baseline	FoMo ( Ours )
PIPAL [ 18 ] (training split)
SR (traditional)	0.577	0.669	0.098	0.618
SR (PSNR-oriented)	0.707	0.785	0.315	0.721
SR (kernel mismatch)	0.560	0.650	0.349	0.537
SR (GAN-based)	0.501	0.565	0.172	0.471

Appendix

Table 13: Every distortion type of all four benchmarks, against the strongest human-annotated recipe for the same backbone on the same benchmark. Bold marks the better of each pair. FoMo supervision leads on 20 of the 42 types with the CNN backbone and on 40 of the 42 with the Transformer backbone.

Explore similar work

Q-DeepSight: Incentivizing Thinking with Images for Image Quality Assessment and Refinement
Apr 18, 2026Xudong Li, Jiaxi Tan, Ziyin Zhou +6Tool Use in VLMs Multimodal CoT Reasoning
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, 361005, P.R. China · Beijing Institute of Technology
Evaluating Generative Models via One-Dimensional Code Distributions
Mar 9, 2026Zexi Jia, Pengcheng Luo, Yijia Zhong +2Image Quality Assessment Image Generation Evaluation
WeChat AI, Tencent Inc., China · School of Intelligence Science and Technology, Peking University · College of Computer Science and Artificial Intelligence, Fudan University
TIQA: Human-Aligned Perceptual Text Quality Assessment in Generated Images
Mar 7, 2026Kirill Koltsov, Aleksandr Gushchin, Anastasia Antsiferova +1Image Quality Assessment T2I Generation Evaluation
Lomonosov Moscow State University Moscow, Russia · ISP RAS Research Center for Trusted Artificial Intelligence Lomonosov Moscow State University Moscow, Russia · ISP RAS Research Center for Trusted Artificial Intelligence MSU Institute for Artificial Intelligence Moscow, Russia +1

Apr 18, 2026J/K move · Enter open · S save

Q-DeepSight: Incentivizing Thinking with Images for Image Quality Assessment and Refinement

Xudong Li, Jiaxi Tan, Ziyin Zhou, Yan Zhong+5

Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, 361005, P.R. China · Beijing Institute of Technology

Tool Use in VLMs Multimodal CoT Reasoning Image Quality Assessment RL for VLMs

Open paper AskPDF arXiv

Image Quality Assessment (IQA) models are increasingly deployed as perceptual critics to guide generative models and image restoration. This role demands not only accurate scores but also actionable, localized feedback. However, current MLLM-based methods adopt a single-look, language-only paradigm, which departs from human evidence-seeking judgment and yields weakly grounded rationales, limiting their reliability for in-the-loop refinement. We propose Q-DeepSight, a think-with-image framework that emulates this human-like process. It performs interleaved Multimodal Chain-of-Thought (iMCoT) with tool-augmented evidence acquisition (e.g., crop-and-zoom) to explicitly determine where quality degrades and why. To train these long iMCoT trajectories via reinforcement learning, we introduce two techniques: Perceptual Curriculum Reward (PCR) to mitigate reward sparsity and Evidence Gradient Filtering (EGF) to improve credit assignment for visually-grounded reasoning. Q-DeepSight achieves state-of-the-art performance across diverse benchmarks, including natural, restored, and AI-generated content. Furthermore, we demonstrate its practical value with Perceptual-in-Generation (PiG), a training-free framework where Q-DeepSight's diagnoses guide iterative image enhancement, effectively closing the loop between assessment and refinement.

Abstract

Explore similar work

Q-DeepSight: Incentivizing Thinking with Images for Image Quality Assessment and Refinement

Evaluating Generative Models via One-Dimensional Code Distributions

TIQA: Human-Aligned Perceptual Text Quality Assessment in Generated Images