cs.CVOct 7, 2026
SaveDo Generative Priors Align with Human Naturalness Perception?
Organizations: Communication Science Laboratories, NTT, Inc.
Abstract
Visual generative models are trained to capture the probability distributions of natural images, yet whether their native priors reflect the regularities governing human perception of image naturalness remains an open question. Here, we probe these priors through native prediction errors across 25 open image and video generators. Because raw single-image losses are dominated by scene content and visual complexity, we evaluate directional loss differences using content-preserving, paired relational interventions that selectively disrupt facial configurations or physical illumination consistency while limiting changes in low-level image statistics. Across both domains, these loss differences reproduce human-like selective sensitivities and tolerances, capturing the classic Thatcher effect on faces and shape-dependent responses to illumination inconsistencies. Notably, these loss differences reliably track continuous gradations of human naturalness judgments across individual stimulus pairs (peaking at on faces and on physical scenes) and retain unique human-aligned signals even after controlling for feature distances from frozen vision encoders and standard image quality metrics. We also find that while overall sensitivity to these violations broadly covaries with human alignment across models, the two systematically decouple along denoising schedules, with alignment peaking earlier than sensitivity, revealing that human-like naturalness judgments dissociate from generic violation detection. Together, these findings demonstrate that learning visual distributions yields generative loss landscapes that capture distinct aspects of human naturalness perception.
Figures & tables
Figure 1: Overview of experimental paradigm and paired response profiles. (A) Measurement pipelines. Left: human naturalness ratings are averaged across observers to define the paired perceptual effect . Right: native prediction errors from frozen models are averaged over matched timesteps and noise to obtain image losses and the paired directional loss difference (schematic shows a noise-predicting diffusion model; prediction targets vary across architectures). (B) Controlled stimulus pairs and representative response profiles. Content-preserving relational interventions flip local elements (eyes/mouth, floor, or central objects; red boxes), leaving overall scene composition intact. Thatcher : both humans and generative models penalize facial distortions far more strongly in upright than in inverted faces. Illumination : factor-level contrasts across cast shadows, light directions, and reflections; generative models mirror human sensitivities to shadows and lighting. † Reflection color is a representative divergence, where humans remain sensitive to small hue mismatches while generative responses decrease. Markers indicate human means with 95% bootstrap intervals (Appendix E ), top-five generative models (median and individuals), and top-five vision encoders (median). Complete profiles: Appendix J . Face images depict a synthetic person for illustration and were not used in experiments.
Figure 2: Spatial loss changes. Mean paired loss maps for two high-alignment models per task. Top-row pixel differences identify the modified regions. Complete 25-model maps: Appendix M .
Figure 3: Item-level human alignment and comparison with baselines. (A) Model–human Pearson correlations on individual stimulus pairs. Saturated points mark top-five Generative and Encoder models and all IQA metrics; diamonds show medians. Dotted lines and bands show mean human split-half reliability and its 95% interval. (B) Bidirectional partial correlations ( , purple circles; , green circles). Faint points show all pairs; saturated points show the selected 5 5 model pairs; stars mark the rank-1 pair. Large markers and bars in all panels show medians and 95% bootstrap CIs (Section 3.4 , Appendix E.3 ).
Figure 4: Cross-model association between sensitivity and human alignment. (A) Violation sensitivity versus human alignment ( ) for 25 generative models on Thatcher and Illumination. Solid lines show linear fits; black diamonds and dashed crosshairs mark human sensitivity and split-half reliability. Marker colors indicate model types (pixel, latent, video); sizes scale with parameter count (bottom legend). (B) Cross-model Pearson correlations ( ) between sensitivity and alignment for each task as a whole (All) and within individual conditions. Error bars denote 95% bootstrap CIs.
Figure 5: Dissociation of sensitivity and human alignment along denoising schedules. (A) Representative schedule dynamics (Wan2.1 T2V 14B) on Thatcher and Illumination: human alignment (blue, solid) and sensitivity (orange, dashed) across normalized schedule positions ( ); solid dots mark peak positions. (B) Peak locations for all 25 models on Thatcher and Illumination across normalized schedules ( ); the dashed diagonal marks equal timing. Dots are colored by model category following Figure 4 ; open black rings identify Wan2.1 T2V 14B.
Appendix figures & tables39 assets
Supplementary material from the paper’s appendix.
Appendix
Figure S1: Thatcher stimulus construction. (A) The 68 facial landmarks detected on an AI-generated fictional face (eyes: blue; mouth: orange; remaining points: gray). (B) Selected regions for feature inversion. Dashed rectangles indicate padded landmark bounds; solid ellipses delimit vertically inverted regions feathered along their boundaries. (C) Original (left) and Thatcherized (right) versions in upright (top) and inverted (bottom; exact rotation) orientations. The face is used only to illustrate the procedure and was not included in the human experiments or model analyses.
Figure S2: Illumination stimulus construction. Original (left) and modified (right) images for the three operations. The floor region is horizontally flipped in Shadow and Reflection, and the central object in Light direction. Orange outlines indicate the affected regions.
Figure S3: Shadow: shape pairs and height. Each example pairs the full original image (left, blue rule) with its modification (right, orange rule). S1 exchanges shadows between distinct object shapes to introduce salient, categorical geometric violations. S3 compares both object heights for the cube–sphere pair, testing whether elevating objects above the floor reduces the salience of shadow inconsistencies.
Figure S4: Shadow: rotations and shadow softness. S2 shows all six orientation pairs, with angles indicating object orientations. In modified images, exchanging shadows alters directional correspondence: this inconsistency is subtle for shapes lacking distinct landmarks (e.g., torus knot) or pairs with near-symmetric 2D silhouettes despite 3D depth asymmetry (e.g., Spot ), requiring observers to scrutinize depth relationships, whereas prominent directional features (e.g., teapot spout) make it immediately obvious. S4 compares soft and hard shadows, testing whether blurred boundaries increase tolerance to discrepancies.
Figure S5: Reflection: shape pairs and rotations. R1 shows all four shape pairs and R2 all four orientation pairs, with height and floor roughness held at the baseline settings. R1 introduces categorical shape discrepancies between objects and their reflections, whereas R2 tests sensitivity to mirror-reflection symmetry across 3D orientations. In R2, inconsistencies are subtle and difficult to detect for shapes lacking distinctive landmarks (e.g., torus knot) or pairs appearing near-symmetric in 2D (e.g., Spot ), whereas salient directional asymmetry (e.g., Spot ) makes violations apparent.
Figure S6: Reflection: color, height, and floor roughness. R3 introduces chromatic mismatches without altering geometry, testing sensitivity to color differences. R4 and R5 compare heights and surface roughness, examining how spatial proximity and blur modulate the salience of reflection anomalies.
Figure S7: Light direction: shapes. L1 shows every shape at shared baseline settings. Horizontally mirroring only the central object reverses its shading relative to the flanking objects and unchanged cast shadows. This tests how object geometry affects lighting consistency, contrasting simple shapes (e.g., sphere) where illumination direction is unambiguous with complex shapes where lighting discrepancies are easily overlooked.
Figure S8: Light direction: material, height, and light angle. Each factor is illustrated at both levels using spheres, holding other factors fixed. L2 tests how surface material properties modulate lighting perception (diffuse shading gradients vs. specular highlights). L3 examines whether the presence of a consistent cast shadow strengthens the perceived lighting conflict compared to floating objects without shadows. L4 tests the effect of illumination inclination ( vs. ).
Figure S9: Participant response consistency and task-specific screening cutoffs. Distributions of leave-one-out Pearson correlations for (A) Thatcher ( ) and (B) Illumination ( ), after duplicate submissions and repeated presentations were reduced by retaining the first occurrence. Each participant’s ratings are correlated with the image-wise means of the other participants. Red and blue bars denote excluded and retained participants, respectively; histogram bins have width . Solid curves show Gaussian kernel density estimates with Silverman’s bandwidths ( and ), scaled to histogram counts. Dashed lines mark the task-specific cutoffs ( and ). Undefined correlations (2 for Thatcher, 1 for Illumination) are excluded from the plots and counted among excluded participants.
| Family | Model | Identifier / Source Repository | Architecture | Evaluated Resolution | Steps ( ) | Noise ( ) |
| Pixel-space image models | ||||||
| JiT | JiT-B/32 | LTH14/JiT::jit-b-32/checkpoint-last.pth | Pixel DiT (ImageNet) | 100 | 20 | |
| JiT | JiT-L/32 | LTH14/JiT::jit-l-32/checkpoint-last.pth | Pixel DiT (ImageNet) | 100 | 20 | |
| JiT | JiT-H/32 | LTH14/JiT::jit-h-32/checkpoint-last.pth | Pixel DiT (ImageNet) | 100 | 20 | |
| PixelGen | PixelGen 512 | zehongma/PixelGen::PixelGen_XXL_T2I.ckpt | Pixel DiT / Flow | 100 | 20 | |
| HiDream | HiDream-O1-Image | HiDream-ai/HiDream-O1-Image | Pixel-space UiT / Flow | 100 | 20 | |
Table S1: Evaluated open generative models. Specifications of the 25 generative models across 11 families, grouped by architectural domain: pixel-space image models, latent-space image models, and spatiotemporal video models. Superscripts T and I mark the top five generative models selected by overall human alignment in Thatcher and Illumination, respectively; these selections are held fixed across conditions (Section I ).
| Encoder | Thatcher | Illumination |
|---|---|---|
| ResNet-50 | (output) | |
| CLIP ViT-B/16 | ||
| DINOv2-Base | (output) | |
| DINOv2-g/14 | ||
| DINOv3-H+/16 | (output) | |
| DINOv3-7B/16 |
Table S2: Selected encoder layers. Entries indicate the selected layer index and total model depth for each task. Entries marked “(output)” use the model’s default final embedding rather than intermediate block readouts (Section D.1 ). For Qwen, indices denote language-decoder blocks. Selected layers remain fixed across all condition-specific comparisons within each task.
| Pool | Predictor | Identifier / checkpoint | Training objective | Params. (B) | Readout / score |
| Encoder | ResNet-50 (ImageNet-1K, torchvision V2) | torchvision::IMAGENET1K_V2 | ImageNet supervision | .024 | Block spatial mean; cosine distance |
| Encoder | CLIP ViT-B/16 T,I | openai/clip-vit-base-patch16 | Image–text contrastive | .086 | Unprojected block CLS; cosine distance |
| Encoder | DINOv2-Base | facebook/dinov2-base | Vision-only self-supervision | .087 | Corresponding patch-token distance |
| Encoder | SigLIP 2 Base/16 I (224) | google/siglip2-base-patch16-224 | Image–text sigmoid contrastive | .093 | LayerNorm, attention pooling; cosine distance |
| Encoder | PE-Core B/16 I (224) | facebook/PE-Core-B16-224 | Vision–language representation | .094 | Block spatial mean; cosine distance |
| Encoder | DINOv2-g/14 spatial | facebook/dinov2-giant | Vision-only self-supervision | 1.136 | Corresponding patch-token distance |
Table S3: Evaluated non-generative baseline suite. Specifications of the 12 frozen vision encoders alongside five full-reference (FR-IQA) and five no-reference (NR-IQA) metrics. Parameter counts (B) report total network parameters in billions. Superscripts T and I mark the top five encoders selected by overall human alignment on Thatcher and Illumination, respectively; these selections are held fixed across conditions (Section I ).
Figure S10: Impact of low-level image statistics on native generative loss. (A) Illustrative CSIQ reference image alongside corrupted examples at maximum severity across six distortion types. (B) Paired loss difference ( )–DMOS Pearson correlations within each CSIQ distortion type across five pixel-space diffusion models. (C) Example synthetic scenes: original baseline, one-shadow deletion, and shadow swap (relocation). (D) Mean loss changes ( ), divided by their sample standard deviation over the 96 pairs of each manipulation, for one-shadow deletion (filled circles) versus shadow swap (open circles). Error bars denote percentile 95% intervals from 10,000 resamples of the 48 base scenes.
Figure S11: Single-image losses and paired loss changes in relation to human judgments. Item-level Pearson correlations for all 25 generative models, pooled across conditions within (A) Thatcher and (B) Illumination. Original- and modified-image losses are correlated with the corresponding image’s mean unnaturalness, ; paired loss changes are correlated with . Faint lines and small points show individual models, with each line connecting estimates from the same model. Thick lines, diamonds, and numeric labels indicate the median across all 25 models.
Figure S12: Model-wise sensitivity and human alignment across conditions. (A) Violation sensitivity, : the condition mean loss difference divided by the model’s sample SD over the full task. (B) Human alignment: Pearson correlation between model loss differences and human paired rating differences within each condition. The top rows show human references and the median across all 25 models; human alignment is the mean raw split-half correlation across 10,000 disjoint participant splits.
Figure S13: Sensitivity and human alignment across stimulus conditions. Violation sensitivity versus human alignment ( ) for 25 generative models across individual conditions: upright Thatcher (A), inverted Thatcher (B), cast shadow (C), surface reflection (D), and light direction (E). Error bars and shaded bands indicate 95% bootstrap intervals (Appendix E ). Black diamonds and dashed crosshairs mark human sensitivity and mean split-half reliability. Solid gray lines show linear fits across models.
Figure S14: Condition-level alignment and partial correlation with baseline models. Panels show Thatcher ( A , upright; B , inverted) and Illumination ( C , shadow; D , reflection; E , light direction), evaluated with the top-five models and encoder layers selected over each full task. Top panels show human alignment (faint points: all candidates; saturated points: selected models; diamonds and error bars: selected median and 95% bootstrap CI; dotted lines and bands: mean human split-half reliability and its 95% interval). Bottom panels show bidirectional partial correlations (faint points: all pairs; saturated points: selected pairs; stars: rank-1 pair; diamonds and bars: selected median and 95% CI). Purple circles show and green circles show . Bootstrap intervals account for task-level layer and model reselection (Appendix E.3 ).
Figure S15: Factor-profile errors across model categories. Columns separate categories and rows separate factors. Dots show individual models (five per category: generative models, visual encoders, FR-IQA, and NR-IQA); diamonds indicate category medians. Models and encoder layers are fixed by task-pooled alignment. The spread describes model variation, not confidence intervals. Overall displays the aggregate MSE across all 13 factor profiles ( , Eq. 7 ) for each model. Factor codes correspond to the detailed profiles in Figures S16 – S18 .
Figure S16: Shadow: factor profiles and model error. Columns separate the four model categories (generative models, visual encoders, FR-IQA, and NR-IQA). Black open circles and horizontal bars show human means and 95% bootstrap intervals (Appendix E ), repeated across columns. Colored dots show individual models, faint lines connect the same model across factor levels, and filled diamonds denote category medians. Gray strips show corresponding factor MSEs. Responses are scaled by task standard deviation without centering (encoder distances are unsigned), with model sets and encoder layers held constant across factors.
Figure S17: Reflection: factor profiles and model error. Plot conventions match Figure S16 .
Figure S18: Light direction: factor profiles and model error. Plot conventions match Figure S16 .
Figure S19: Model size, sensitivity, and human alignment. ( A, B ) Violation sensitivity and ( C, D ) human alignment ( ) versus scoring parameter count for 25 generators and 12 encoders. Solid and dashed lines show linear fits against log parameter count for generators and encoders, respectively; annotations report Pearson and Spearman . Error bars indicate 95% bootstrap intervals (Appendix E ). Colors and markers match Figure 4 and the bottom legend.
Figure S20: Model size and condition-specific metrics in the Thatcher task. ( A, B ) Violation sensitivity and ( C, D ) human alignment ( ) versus scoring parameter count for upright and inverted faces across 25 generators. Solid lines show linear fits against log parameter count; annotations report Pearson and Spearman . Error bars indicate 95% bootstrap intervals (Appendix E ). Colors and markers match Figure 4 and the bottom legend.
Figure S21: Model size and condition-specific metrics in the Illumination task. ( A–C ) Violation sensitivity and ( D–F ) human alignment ( ) versus scoring parameter count for cast shadow, surface reflection, and light direction across 25 generators. Solid lines show linear fits against log parameter count; annotations report Pearson and Spearman . Error bars indicate 95% bootstrap intervals (Appendix E ). Colors and markers match Figure S20 and the bottom legend.
| ID | Evaluated model | B | Published entry | Score |
| Image models: Artificial Analysis human-preference Elo | ||||
| P05 | HiDream-O1-Image | 8.805 | HiDream-O1-Image | 979 |
| L01 | Stable Diffusion v1.5 | 0.86 | Stable Diffusion 1.5 | 468 |
| L02 | SDXL Base 1.0 | 2.6 | Stable Diffusion XL 1.0 | 688 |
| L03 | Stable Diffusion 3 Medium | 2 | Stable Diffusion 3 Medium | 726 |
| L04 | FLUX.1 dev | 12 | FLUX.1 [dev] | 842 |
Table S4: Models matched to public generation evaluations. Parameters refer to the scoring backbone in billions. The same IDs, colors, and markers identify models in Figures S22 and S23 . Published names do not guarantee identical checkpoint revisions or generation pipelines.
| Models ( ) | Task | |||
|---|---|---|---|---|
| Image (11) | Thatcher | |||
| Illumination | ||||
| Video (7) | Thatcher | |||
| Illumination |
Table S5: Generation evaluations and human alignment. Spearman associations of generation evaluations , alignment , and sensitivity ; variables after are controlled. Scatter plots are in Figures S22 – S24 .
Figure S22: Published generation evaluations and human alignment before adjustment. Image models (A, B; ) and video models (C, D; ), with Thatcher on the left and Illumination on the right. Each point represents one model; IDs, colors, and markers match Figure S23 . The vertical axis is human alignment ( ). Vertical bars show 95% participant-and-scene bootstrap CIs for each model; horizontal bars in A and B show published Elo intervals.
Figure S23: Generation evaluations and human alignment after controlling for violation sensitivity. The same models and panel arrangement as Figure S22 . Both axes show residuals from regressing the corresponding model ranks on sensitivity rank, divided by their own sample standard deviations. Zero denotes agreement with the sensitivity-based regression prediction. Gray lines show least-squares fits, whose slope equals the annotated partial Spearman correlation. These are the sensitivity-adjusted values in Table S5 .
Figure S24: Generation evaluations and sensitivity after controlling for human alignment. Same format, models, and conventions as Figure S23 , showing standardized rank residuals of generation evaluations (horizontal) and violation sensitivity (vertical) after regressing out human alignment rank. These are the alignment-adjusted values in Table S5 .
| Partial Spearman | ||||
|---|---|---|---|---|
| Models | Task | |||
| Image | Thatcher | 11 | ||
| Illumination | 11 | |||
| Video | Thatcher | 7 | ||
| Illumination | 7 | |||
Table S6: Generation-evaluation associations with sensitivity and size adjustment. All coefficients relate generation evaluation to human alignment . is violation sensitivity and is scoring-backbone parameter count. Partial Spearman coefficients control ranks of the indicated covariates.
Figure S25: Mean loss-change maps for image models (Part 1 of 2).
Figure S26: Mean loss-change maps for image models (Part 2 of 2).
Figure S27: Mean loss-change maps for video models.
Figure S28: Condition-specific alignment and sensitivity peak positions. Each panel displays all 25 models (dots colored by model type: pixel, latent, and video, following Figure 4 ). Horizontal and vertical axes show condition-specific alignment and sensitivity peak positions, respectively, on each model’s normalized schedule ( ). Points above the diagonal indicate a later sensitivity peak (counts printed below each panel).
Figure S29: Condition-specific Thatcher timestep profiles for all 25 models. Panels show individual models (top bar colored as in Figure 4 ). Blue and orange curves denote upright and inverted faces; solid curves show item-level Pearson alignment (left axis) and dashed curves show task-scaled sensitivity (right axis). Filled and open circles mark alignment and sensitivity maxima, respectively. The horizontal axis indicates normalized schedule position ( ).
Figure S30: Condition-specific Illumination timestep profiles for all 25 models. Panels show individual models (top bar colored as in Figure 4 ). Blue, purple, and green curves denote shadow, reflection, and light-direction interventions; solid curves show item-level Pearson alignment (left axis) and dashed curves show task-scaled sensitivity (right axis). Filled and open circles mark alignment and sensitivity maxima, respectively. The horizontal axis indicates normalized schedule position ( ).
| Condition | Image models | Video models |
|---|---|---|
| Thatcher (upright) | A close-up frontal photograph of a person’s face, shown upright. | A static camera shot showing a close-up frontal view of a person’s face, shown upright. |
| Thatcher (inverted) | An upside-down close-up frontal photograph of a person’s face. | A static camera shot showing an upside-down close-up frontal view of a person’s face. |
| Illumination (all conditions) | A computer-generated 3D scene of multiple geometric objects on a flat tabletop against a dark background. | A static camera shot of a computer-generated 3D scene with multiple geometric objects on a flat tabletop against a dark background. |
Table S7: Fixed descriptive prompts for text conditioning. Prompts used in the conditional evaluation. Each prompt was identical for the original and modified images within each pair and did not mention the visual manipulation or judge naturalness. The baseline condition used an empty string.
| Thatcher | Illumination | |||||
|---|---|---|---|---|---|---|
| Model | Empty | Desc. | [95% CI] | Empty | Desc. | [95% CI] |
| SD v1.5 | ||||||
| SDXL | ||||||
| SD3 | ||||||
| FLUX.1 dev | ||||||
| PixelGen | ||||||
Table S8: Paired text-conditioning control. Human alignment (Pearson ) under empty and descriptive (Desc.) prompts, with . Brackets give paired percentile 95% bootstrap confidence intervals for from 10,000 participant-and-scene draws (140 face identities in Thatcher or the 48 base configurations within each operation in Illumination), shared between empty and descriptive prompts.
| Task | Same-half | Cross-half | |
|---|---|---|---|
| Thatcher | |||
| Illumination |
Table S9: Shared sampling noise control across 19 models. Cross-model correlations between violation sensitivity and human alignment under same-half and cross-half draw splits, with .