Do Generative Priors Align with Human Naturalness Perception?

Taiki Fukiage

cs.CVOct 7, 2026

Do Generative Priors Align with Human Naturalness Perception?

Authors: Taiki Fukiage

Organizations: Communication Science Laboratories, NTT, Inc.

Abstract

Visual generative models are trained to capture the probability distributions of natural images, yet whether their native priors reflect the regularities governing human perception of image naturalness remains an open question. Here, we probe these priors through native prediction errors across 25 open image and video generators. Because raw single-image losses are dominated by scene content and visual complexity, we evaluate directional loss differences using content-preserving, paired relational interventions that selectively disrupt facial configurations or physical illumination consistency while limiting changes in low-level image statistics. Across both domains, these loss differences reproduce human-like selective sensitivities and tolerances, capturing the classic Thatcher effect on faces and shape-dependent responses to illumination inconsistencies. Notably, these loss differences reliably track continuous gradations of human naturalness judgments across individual stimulus pairs (peaking at $r = .84$ on faces and $.64$ on physical scenes) and retain unique human-aligned signals even after controlling for feature distances from frozen vision encoders and standard image quality metrics. We also find that while overall sensitivity to these violations broadly covaries with human alignment across models, the two systematically decouple along denoising schedules, with alignment peaking earlier than sensitivity, revealing that human-like naturalness judgments dissociate from generic violation detection. Together, these findings demonstrate that learning visual distributions yields generative loss landscapes that capture distinct aspects of human naturalness perception.

Figures & tables

Figure 1: Overview of experimental paradigm and paired response profiles. (A) Measurement pipelines. Left: human naturalness ratings are averaged across observers to define the paired perceptual effect $h$ . Right: native prediction errors from frozen models are averaged over matched timesteps and noise to obtain image losses and the paired directional loss difference $u_{m}$ (schematic shows a noise-predicting diffusion model; prediction targets vary across architectures). (B) Controlled stimulus pairs and representative response profiles. Content-preserving relational interventions flip local elements (eyes/mouth, floor, or central objects; red boxes), leaving overall scene composition intact. Thatcher : both humans and generative models penalize facial distortions far more strongly in upright than in inverted faces. Illumination : factor-level contrasts across cast shadows, light directions, and reflections; generative models mirror human sensitivities to shadows and lighting. † Reflection color is a representative divergence, where humans remain sensitive to small hue mismatches while generative responses decrease. Markers indicate human means with 95% bootstrap intervals (Appendix E ), top-five generative models (median and individuals), and top-five vision encoders (median). Complete profiles: Appendix J . Face images depict a synthetic person for illustration and were not used in experiments.

Figure 2: Spatial loss changes. Mean paired loss maps for two high-alignment models per task. Top-row pixel differences identify the modified regions. Complete 25-model maps: Appendix M .

Figure 3: Item-level human alignment and comparison with baselines. (A) Model–human Pearson correlations on individual stimulus pairs. Saturated points mark top-five Generative and Encoder models and all IQA metrics; diamonds show medians. Dotted lines and bands show mean human split-half reliability and its 95% interval. (B) Bidirectional partial correlations ( $r(h,G\mid X)$ , purple circles; $r(h,X\mid G)$ , green circles). Faint points show all pairs; saturated points show the selected 5 $\times$ 5 model pairs; stars mark the rank-1 pair. Large markers and bars in all panels show medians and 95% bootstrap CIs (Section 3.4 , Appendix E.3 ).

Figure 4: Cross-model association between sensitivity and human alignment. (A) Violation sensitivity versus human alignment (

r

) for 25 generative models on Thatcher and Illumination. Solid lines show linear fits; black diamonds and dashed crosshairs mark human sensitivity and split-half reliability. Marker colors indicate model types (pixel, latent, video); sizes scale with parameter count (bottom legend). (B) Cross-model Pearson correlations (

r

) between sensitivity and alignment for each task as a whole (All) and within individual conditions. Error bars denote 95% bootstrap CIs.

Figure 5: Dissociation of sensitivity and human alignment along denoising schedules. (A) Representative schedule dynamics (Wan2.1 T2V 14B) on Thatcher and Illumination: human alignment

r

(blue, solid) and sensitivity (orange, dashed) across normalized schedule positions (

0\to 1

); solid dots mark peak positions. (B) Peak locations for all 25 models on Thatcher and Illumination across normalized schedules (

[0,1]

); the dashed diagonal marks equal timing. Dots are colored by model category following Figure 4 ; open black rings identify Wan2.1 T2V 14B.

Appendix figures & tables39 assets

Supplementary material from the paper’s appendix.

Appendix

Figure S1: Thatcher stimulus construction. (A) The 68 facial landmarks detected on an AI-generated fictional face (eyes: blue; mouth: orange; remaining points: gray). (B) Selected regions for feature inversion. Dashed rectangles indicate padded landmark bounds; solid ellipses delimit vertically inverted regions feathered along their boundaries. (C) Original (left) and Thatcherized (right) versions in upright (top) and inverted (bottom; exact $180^{\circ}$ rotation) orientations. The face is used only to illustrate the procedure and was not included in the human experiments or model analyses.

Figure S2: Illumination stimulus construction. Original (left) and modified (right) images for the three operations. The floor region is horizontally flipped in Shadow and Reflection, and the central object in Light direction. Orange outlines indicate the affected regions.

Figure S3: Shadow: shape pairs and height. Each example pairs the full original image (left, blue rule) with its modification (right, orange rule). S1 exchanges shadows between distinct object shapes to introduce salient, categorical geometric violations. S3 compares both object heights for the cube–sphere pair, testing whether elevating objects above the floor reduces the salience of shadow inconsistencies.

Figure S4: Shadow: rotations and shadow softness. S2 shows all six orientation pairs, with angles indicating object orientations. In modified images, exchanging shadows alters directional correspondence: this inconsistency is subtle for shapes lacking distinct landmarks (e.g., torus knot) or pairs with near-symmetric 2D silhouettes despite 3D depth asymmetry (e.g., Spot $-45^{\circ}\leftrightarrow 135^{\circ}$ ), requiring observers to scrutinize depth relationships, whereas prominent directional features (e.g., teapot spout) make it immediately obvious. S4 compares soft and hard shadows, testing whether blurred boundaries increase tolerance to discrepancies.

Figure S5: Reflection: shape pairs and rotations. R1 shows all four shape pairs and R2 all four orientation pairs, with height and floor roughness held at the baseline settings. R1 introduces categorical shape discrepancies between objects and their reflections, whereas R2 tests sensitivity to mirror-reflection symmetry across 3D orientations. In R2, inconsistencies are subtle and difficult to detect for shapes lacking distinctive landmarks (e.g., torus knot) or pairs appearing near-symmetric in 2D (e.g., Spot $-45^{\circ}\leftrightarrow 135^{\circ}$ ), whereas salient directional asymmetry (e.g., Spot $45^{\circ}\leftrightarrow 90^{\circ}$ ) makes violations apparent.

Figure S6: Reflection: color, height, and floor roughness. R3 introduces chromatic mismatches without altering geometry, testing sensitivity to color differences. R4 and R5 compare heights and surface roughness, examining how spatial proximity and blur modulate the salience of reflection anomalies.

Figure S7: Light direction: shapes. L1 shows every shape at shared baseline settings. Horizontally mirroring only the central object reverses its shading relative to the flanking objects and unchanged cast shadows. This tests how object geometry affects lighting consistency, contrasting simple shapes (e.g., sphere) where illumination direction is unambiguous with complex shapes where lighting discrepancies are easily overlooked.

Figure S8: Light direction: material, height, and light angle. Each factor is illustrated at both levels using spheres, holding other factors fixed. L2 tests how surface material properties modulate lighting perception (diffuse shading gradients vs. specular highlights). L3 examines whether the presence of a consistent cast shadow strengthens the perceived lighting conflict compared to floating objects without shadows. L4 tests the effect of illumination inclination ( $15^{\circ}$ vs. $30^{\circ}$ ).

Figure S9: Participant response consistency and task-specific screening cutoffs. Distributions of leave-one-out Pearson correlations

r

for (A) Thatcher (

n=162

) and (B) Illumination (

n=165

), after duplicate submissions and repeated presentations were reduced by retaining the first occurrence. Each participant’s ratings are correlated with the image-wise means of the other participants. Red and blue bars denote excluded and retained participants, respectively; histogram bins have width

.04

. Solid curves show Gaussian kernel density estimates with Silverman’s bandwidths (

b=.03836

and

.11157

), scaled to histogram counts. Dashed lines mark the task-specific cutoffs (

r=.2481

and

.3236

). Undefined correlations (2 for Thatcher, 1 for Illumination) are excluded from the plots and counted among excluded participants.

Family	Model	Identifier / Source Repository	Architecture	Evaluated Resolution	Steps ( $\|T_{m}\|$ )	Noise ( $\|E_{m}\|$ )
Pixel-space image models
JiT	JiT-B/32	LTH14/JiT::jit-b-32/checkpoint-last.pth	Pixel DiT (ImageNet)	$512\times 512$	100	20
JiT	JiT-L/32	LTH14/JiT::jit-l-32/checkpoint-last.pth	Pixel DiT (ImageNet)	$512\times 512$	100	20
JiT	JiT-H/32	LTH14/JiT::jit-h-32/checkpoint-last.pth	Pixel DiT (ImageNet)	$512\times 512$	100	20
PixelGen	PixelGen 512	zehongma/PixelGen::PixelGen_XXL_T2I.ckpt	Pixel DiT / Flow	$512\times 512$	100	20
HiDream	HiDream-O1-Image	HiDream-ai/HiDream-O1-Image	Pixel-space UiT / Flow	$512\times 512$	100	20

Appendix

Table S1: Evaluated open generative models. Specifications of the 25 generative models across 11 families, grouped by architectural domain: pixel-space image models, latent-space image models, and spatiotemporal video models. Superscripts T and I mark the top five generative models selected by overall human alignment in Thatcher and Illumination, respectively; these selections are held fixed across conditions (Section I ).

Encoder	Thatcher	Illumination
ResNet-50	$7/16$	$16/16$ (output)
CLIP ViT-B/16	$12/12$	$9/12$
DINOv2-Base	$2/12$	$12/12$ (output)
DINOv2-g/14	$10/40$	$8/40$
DINOv3-H+/16	$32/32$ (output)	$1/32$
DINOv3-7B/16	$36/40$	$5/40$

Appendix

Table S2: Selected encoder layers. Entries indicate the selected layer index and total model depth for each task. Entries marked “(output)” use the model’s default final embedding rather than intermediate block readouts (Section D.1 ). For Qwen, indices denote language-decoder blocks. Selected layers remain fixed across all condition-specific comparisons within each task.

Pool	Predictor	Identifier / checkpoint	Training objective	Params. (B)	Readout / score
Encoder	ResNet-50 (ImageNet-1K, torchvision V2)	torchvision::IMAGENET1K_V2	ImageNet supervision	.024	Block spatial mean; cosine distance
Encoder	CLIP ViT-B/16 T,I	openai/clip-vit-base-patch16	Image–text contrastive	.086	Unprojected block CLS; cosine distance
Encoder	DINOv2-Base	facebook/dinov2-base	Vision-only self-supervision	.087	Corresponding patch-token distance
Encoder	SigLIP 2 Base/16 I (224)	google/siglip2-base-patch16-224	Image–text sigmoid contrastive	.093	LayerNorm, attention pooling; cosine distance
Encoder	PE-Core B/16 I (224)	facebook/PE-Core-B16-224	Vision–language representation	.094	Block spatial mean; cosine distance
Encoder	DINOv2-g/14 spatial	facebook/dinov2-giant	Vision-only self-supervision	1.136	Corresponding patch-token distance

Appendix

Table S3: Evaluated non-generative baseline suite. Specifications of the 12 frozen vision encoders alongside five full-reference (FR-IQA) and five no-reference (NR-IQA) metrics. Parameter counts (B) report total network parameters in billions. Superscripts T and I mark the top five encoders selected by overall human alignment on Thatcher and Illumination, respectively; these selections are held fixed across conditions (Section I ).

Figure S10: Impact of low-level image statistics on native generative loss. (A) Illustrative CSIQ reference image alongside corrupted examples at maximum severity across six distortion types. (B) Paired loss difference ( $L_{\mathrm{distorted}}-L_{\mathrm{reference}}$ )–DMOS Pearson correlations within each CSIQ distortion type across five pixel-space diffusion models. (C) Example synthetic scenes: original baseline, one-shadow deletion, and shadow swap (relocation). (D) Mean loss changes ( $L_{\mathrm{modified}}-L_{\mathrm{original}}$ ), divided by their sample standard deviation over the 96 pairs of each manipulation, for one-shadow deletion (filled circles) versus shadow swap (open circles). Error bars denote percentile 95% intervals from 10,000 resamples of the 48 base scenes.

Figure S11: Single-image losses and paired loss changes in relation to human judgments. Item-level Pearson correlations for all 25 generative models, pooled across conditions within (A) Thatcher and (B) Illumination. Original- and modified-image losses are correlated with the corresponding image’s mean unnaturalness,

5-R(x)

; paired loss changes are correlated with

R(x^{\mathrm{orig}})-R(x^{\mathrm{mod}})

. Faint lines and small points show individual models, with each line connecting estimates from the same model. Thick lines, diamonds, and numeric labels indicate the median across all 25 models.

Figure S12: Model-wise sensitivity and human alignment across conditions. (A) Violation sensitivity, $S_{m,c}=\bar{u}_{m,c}/s_{m,\mathrm{task}}$ : the condition mean loss difference divided by the model’s sample SD over the full task. (B) Human alignment: Pearson correlation between model loss differences and human paired rating differences within each condition. The top rows show human references and the median across all 25 models; human alignment is the mean raw split-half correlation across 10,000 disjoint participant splits.

Figure S13: Sensitivity and human alignment across stimulus conditions. Violation sensitivity versus human alignment (

r

) for 25 generative models across individual conditions: upright Thatcher (A), inverted Thatcher (B), cast shadow (C), surface reflection (D), and light direction (E). Error bars and shaded bands indicate 95% bootstrap intervals (Appendix E ). Black diamonds and dashed crosshairs mark human sensitivity and mean split-half reliability. Solid gray lines show linear fits across models.

Figure S14: Condition-level alignment and partial correlation with baseline models. Panels show Thatcher ( A , upright; B , inverted) and Illumination ( C , shadow; D , reflection; E , light direction), evaluated with the top-five models and encoder layers selected over each full task. Top panels show human alignment (faint points: all candidates; saturated points: selected models; diamonds and error bars: selected median and 95% bootstrap CI; dotted lines and bands: mean human split-half reliability and its 95% interval). Bottom panels show bidirectional partial correlations (faint points: all pairs; saturated points: selected $5\times 5$ pairs; stars: rank-1 pair; diamonds and bars: selected median and 95% CI). Purple circles show $r(h,G\mid X)$ and green circles show $r(h,X\mid G)$ . Bootstrap intervals account for task-level layer and model reselection (Appendix E.3 ).

Figure S15: Factor-profile errors across model categories. Columns separate categories and rows separate factors. Dots show individual models (five per category: generative models, visual encoders, FR-IQA, and NR-IQA); diamonds indicate category medians. Models and encoder layers are fixed by task-pooled alignment. The spread describes model variation, not confidence intervals. Overall displays the aggregate MSE across all 13 factor profiles (

\mathrm{MSE}_{m}

, Eq. 7 ) for each model. Factor codes correspond to the detailed profiles in Figures S16 – S18 .

Figure S16: Shadow: factor profiles and model error. Columns separate the four model categories (generative models, visual encoders, FR-IQA, and NR-IQA). Black open circles and horizontal bars show human means and 95% bootstrap intervals (Appendix E ), repeated across columns. Colored dots show individual models, faint lines connect the same model across factor levels, and filled diamonds denote category medians. Gray strips show corresponding factor MSEs. Responses are scaled by task standard deviation without centering (encoder distances are unsigned), with model sets and encoder layers held constant across factors.

Figure S17: Reflection: factor profiles and model error. Plot conventions match Figure S16 .

Figure S18: Light direction: factor profiles and model error. Plot conventions match Figure S16 .

Figure S19: Model size, sensitivity, and human alignment. ( A, B ) Violation sensitivity and ( C, D ) human alignment (

r

) versus scoring parameter count for 25 generators and 12 encoders. Solid and dashed lines show linear fits against log parameter count for generators and encoders, respectively; annotations report Pearson

r

and Spearman

\rho

. Error bars indicate 95% bootstrap intervals (Appendix E ). Colors and markers match Figure 4 and the bottom legend.

Figure S20: Model size and condition-specific metrics in the Thatcher task. ( A, B ) Violation sensitivity and ( C, D ) human alignment (

r

) versus scoring parameter count for upright and inverted faces across 25 generators. Solid lines show linear fits against log parameter count; annotations report Pearson

r

and Spearman

\rho

. Error bars indicate 95% bootstrap intervals (Appendix E ). Colors and markers match Figure 4 and the bottom legend.

Figure S21: Model size and condition-specific metrics in the Illumination task. ( A–C ) Violation sensitivity and ( D–F ) human alignment (

r

) versus scoring parameter count for cast shadow, surface reflection, and light direction across 25 generators. Solid lines show linear fits against log parameter count; annotations report Pearson

r

and Spearman

\rho

. Error bars indicate 95% bootstrap intervals (Appendix E ). Colors and markers match Figure S20 and the bottom legend.

ID	Evaluated model	B	Published entry	Score
Image models: Artificial Analysis human-preference Elo
P05	HiDream-O1-Image	8.805	HiDream-O1-Image	979
L01	Stable Diffusion v1.5	0.86	Stable Diffusion 1.5	468
L02	SDXL Base 1.0	2.6	Stable Diffusion XL 1.0	688
L03	Stable Diffusion 3 Medium	2	Stable Diffusion 3 Medium	726
L04	FLUX.1 dev	12	FLUX.1 [dev]	842

Appendix

Table S4: Models matched to public generation evaluations. Parameters refer to the scoring backbone in billions. The same IDs, colors, and markers identify models in Figures S22 and S23 . Published names do not guarantee identical checkpoint revisions or generation pipelines.

Models ( $n$ )	Task	$\rho(Q,A)$	$\rho(Q,A\mid S)$	$\rho(Q,S\mid A)$
Image (11)	Thatcher	$.191$	$.182$	$.201$
	Illumination	$.827$	$.670$	$-.045$
Video (7)	Thatcher	$.929$	$.900$	$.122$
	Illumination	$.821$	$.778$	$.218$

Appendix

Table S5: Generation evaluations and human alignment. Spearman associations of generation evaluations

Q

, alignment

A

, and sensitivity

S

; variables after

\mid

are controlled. Scatter plots are in Figures S22 – S24 .

Figure S22: Published generation evaluations and human alignment before adjustment. Image models (A, B;

n=11

) and video models (C, D;

n=7

), with Thatcher on the left and Illumination on the right. Each point represents one model; IDs, colors, and markers match Figure S23 . The vertical axis is human alignment (

r

). Vertical bars show 95% participant-and-scene bootstrap CIs for each model; horizontal bars in A and B show published Elo intervals.

Figure S23: Generation evaluations and human alignment after controlling for violation sensitivity. The same models and panel arrangement as Figure S22 . Both axes show residuals from regressing the corresponding model ranks on sensitivity rank, divided by their own sample standard deviations. Zero denotes agreement with the sensitivity-based regression prediction. Gray lines show least-squares fits, whose slope equals the annotated partial Spearman correlation. These are the sensitivity-adjusted values in Table S5 .

Figure S24: Generation evaluations and sensitivity after controlling for human alignment. Same format, models, and conventions as Figure S23 , showing standardized rank residuals of generation evaluations (horizontal) and violation sensitivity (vertical) after regressing out human alignment rank. These are the alignment-adjusted values in Table S5 .

Models	Task	$n$	$S$	$S,P$
			Partial Spearman
Image	Thatcher	11	$.182$	$.222$
	Illumination	11	$.670$	$.783$
Video	Thatcher	7	$.900$	$.913$
	Illumination	7	$.778$	$.914$

Appendix

Table S6: Generation-evaluation associations with sensitivity and size adjustment. All coefficients relate generation evaluation

Q

to human alignment

A

S

is violation sensitivity and

P

is scoring-backbone parameter count. Partial Spearman coefficients control ranks of the indicated covariates.

Figure S25: Mean loss-change maps for image models (Part 1 of 2).

Figure S26: Mean loss-change maps for image models (Part 2 of 2).

Figure S27: Mean loss-change maps for video models.

Figure S28: Condition-specific alignment and sensitivity peak positions. Each panel displays all 25 models (dots colored by model type: pixel, latent, and video, following Figure 4 ). Horizontal and vertical axes show condition-specific alignment and sensitivity peak positions, respectively, on each model’s normalized schedule (

[0,1]

). Points above the diagonal indicate a later sensitivity peak (counts printed below each panel).

Figure S29: Condition-specific Thatcher timestep profiles for all 25 models. Panels show individual models (top bar colored as in Figure 4 ). Blue and orange curves denote upright and inverted faces; solid curves show item-level Pearson alignment (left axis) and dashed curves show task-scaled sensitivity (right axis). Filled and open circles mark alignment and sensitivity maxima, respectively. The horizontal axis indicates normalized schedule position (

[0,1]

Figure S30: Condition-specific Illumination timestep profiles for all 25 models. Panels show individual models (top bar colored as in Figure 4 ). Blue, purple, and green curves denote shadow, reflection, and light-direction interventions; solid curves show item-level Pearson alignment (left axis) and dashed curves show task-scaled sensitivity (right axis). Filled and open circles mark alignment and sensitivity maxima, respectively. The horizontal axis indicates normalized schedule position (

[0,1]

Condition	Image models	Video models
Thatcher (upright)	A close-up frontal photograph of a person’s face, shown upright.	A static camera shot showing a close-up frontal view of a person’s face, shown upright.
Thatcher (inverted)	An upside-down close-up frontal photograph of a person’s face.	A static camera shot showing an upside-down close-up frontal view of a person’s face.
Illumination (all conditions)	A computer-generated 3D scene of multiple geometric objects on a flat tabletop against a dark background.	A static camera shot of a computer-generated 3D scene with multiple geometric objects on a flat tabletop against a dark background.

Appendix

Table S7: Fixed descriptive prompts for text conditioning. Prompts used in the conditional evaluation. Each prompt was identical for the original and modified images within each pair and did not mention the visual manipulation or judge naturalness. The baseline condition used an empty string.

Model	Empty	Desc.	$\Delta r$ [95% CI]	Empty	Desc.	$\Delta r$ [95% CI]
	Thatcher			Illumination
SD v1.5	$.825$	$.822$	$-.003\;[-.006,.000]$	$.234$	$.268$	$+.033\;[.017,.050]$
SDXL	$.660$	$.662$	$+.002\;[-.028,.033]$	$.226$	$.233$	$+.007\;[-.011,.025]$
SD3	$.730$	$.744$	$+.014\;[-.002,.031]$	$.263$	$.267$	$+.004\;[-.034,.044]$
FLUX.1 dev	$.751$	$.752$	$+.001\;[-.007,.009]$	$.353$	$.364$	$+.011\;[-.012,.031]$
PixelGen	$.485$	$.493$	$+.008\;[-.044,.060]$	$.101$	$.117$	$+.016\;[-.067,.091]$

Appendix

Table S8: Paired text-conditioning control. Human alignment (Pearson

r

) under empty and descriptive (Desc.) prompts, with

\Delta r=r_{\mathrm{Desc.}}-r_{\mathrm{Empty}}

. Brackets give paired percentile 95% bootstrap confidence intervals for

\Delta r

from 10,000 participant-and-scene draws (140 face identities in Thatcher or the 48 base configurations within each operation in Illumination), shared between empty and descriptive prompts.

Task	Same-half	Cross-half	$\Delta r$
Thatcher	$.6962$	$.6971$	$+.0009$
Illumination	$.9242$	$.9238$	$-.0005$

Appendix

Table S9: Shared sampling noise control across 19 models. Cross-model correlations between violation sensitivity and human alignment under same-half and cross-half draw splits, with

\Delta r=r_{\mathrm{Cross}}-r_{\mathrm{Same}}

Explore similar work

May 22, 2026cs.CV

Not Too Generative, Not Too Discriminative: The Human Alignment Sweet Spot

A central question in computational vision is whether human-like visual representations are better explained by discriminative or generative learning. Existing comparisons, however, often confound the learning objective with architecture, scale, and training data, leaving open whether the objective itself drives alignment. We address this confound using Joint Energy-Based Models (JEMs), which interpolate continuously between discriminative and generative training within a fixed architecture. By varying a single mixing coefficient, we isolate the effect of the learning objective and evaluate the resulting models across six human-alignment benchmarks spanning perceptual similarity, gloss perception, human response uncertainty, robustness, shape-texture cue conflict, and diagnostic feature attribution. Across this diverse suite, human alignment is consistently maximized at intermediate points of the generative-discriminative continuum, rather than at either endpoint. Hybrid JEMs combine the categorical structure induced by discriminative learning with the sensitivity to input structure induced by generative learning, yielding more human-like behavior across multiple levels of vision. These results suggest that the generative-discriminative dichotomy is the wrong axis for understanding human-aligned vision: alignment emerges not from choosing one objective over the other, but from balancing both.

Jorge Chang Ortega, Bastien Le Lan, Thomas Serre +1

ANITI · Brown University, ANITI · CNRS, ANITI

Discriminative Generative Models Texture

Ask PDF

May 12, 2026q-bio.NC

Human face perception reflects inverse-generative and naturalistic discriminative objectives

The perceptual representations supporting our ability to recognize faces remain a computational mystery. Deep neural networks offer mechanistic hypotheses for human face perception, but theoretically distinct models often make indistinguishable representational predictions for randomly sampled faces. To expose diagnostic differences among these hypotheses, we compared six neural network models sharing an architecture but trained on distinct tasks, using face pairs optimized to elicit contrasting model predictions ("controversial" pairs) alongside randomly sampled pairs. We tested model predictions against face-dissimilarity judgments from 864 human participants across stimulus sets differing in realism and pose variation. Models prioritizing high-level, invariant structures (trained via inverse rendering, face identification, or object classification) most robustly matched human judgments. Furthermore, models trained on natural images typically outperformed synthetic-trained counterparts. Together, these findings suggest that human face perception is shaped by mechanisms that infer latent causes of facial appearance, discount nuisance variation, and are tuned by natural image statistics.

Wenxuan Guo, Heiko H. Schütt, Kamila Maria Jozwik +3

Department of Psychology, Columbia University, New York, NY, USA. · Department of Behavioural and Cognitive Sciences, Université du Luxembourg, Esch-sur-Alzette, Luxembourg. · MRC Cognition and Brain Sciences Unit, University of Cambridge, Cambridge, England. +4

Face Recognition Model Perception Objectives

Ask PDF

May 6, 2026cs.LG

Threshold-Guided Optimization for Visual Generative Models

Aligning large visual generative models with human feedback is often performed through pairwise preference optimization. While such approaches are conceptually simple, they fundamentally rely on annotated pairs, limiting scalability in settings where feedback is collected as independent scalar ratings. In this work, we revisit the KL-regularized alignment objective and show that the optimal policy implicitly compares each sample's reward to an instance-specific baseline that is generally intractable. We propose a threshold-guided alignment framework that replaces this oracle baseline with a data-driven global threshold estimated from empirical score statistics. This formulation turns alignment into a binary decision task on unpaired data, enabling effective optimization directly from scalar feedback. We also incorporate a confidence weighting term to emphasize samples whose scores deviate strongly from the threshold, improving sample efficiency. Experiments across both diffusion and masked generative paradigms, spanning three test sets and five reward models, show that our method consistently improves preference alignment over previous methods. These results position our threshold-guided framework as a simple yet principled alternative for aligning visual generative models without paired comparisons.

Jinbin Bai, Yu Lei, Qingyu Shi +6

National University of Singapore · Collov Labs · Peking University +2

Preference Alignment Generative Models Feedback

Ask PDF