TRIG: Toward Measuring Texture Realism in Image Generation

Zhiwei Xu; Jiali Cui; Zhiqiang Lao; Heather Yu; Liang Peng

cs.CVOct 10, 2026

TRIG: Toward Measuring Texture Realism in Image Generation

Authors: Zhiwei Xu, Jiali Cui, Zhiqiang Lao, Heather Yu, Liang Peng

Organizations: Northwestern University · Futurewei Technologies

Abstract

This paper studies the backbone used by metrics for evaluating the quality of generated images. Although generative models have advanced rapidly, some outputs still exhibit unrealistic texture characteristics, and these differences are difficult to measure. Widely used evaluation metrics provide limited insight into texture realism, since they often rely on a backbone trained for another objective, such as image classification or image-text alignment. Such objectives typically encourage invariance to low-level variations that carry texture, so the resulting features primarily encode semantic and global structure. As a result, texture-specific discrepancies may not be explicitly captured. We propose TRIG, a metric directed at the texture realism of generated images. Its backbone is instead the discriminator of a generative adversarial network pre-trained on a texture dataset, which provides a learned prior for the local statistics of natural textures. A plausibility head is fine-tuned to identify local texture degradations, while preserving natural defocus and sensor noise as positive cases. On the features of this backbone, we compute a standard distributional distance between a reference set of real images and a set of generated ones. We apply TRIG to several generative models and observe closer agreement with human realness ratings than baseline metrics.

Figures & tables

Panel of Figure 2 — Figure 2: Backbone sensitivity to high-frequency texture degradation. Under the same high-frequency suppression, semantic backbones remain relatively texture-invariant, whereas our texture-oriented backbone exhibits the largest feature change and strongest spatial activation redistribution.

Panel of Figure 3 — Figure 3: Examples of texture images in ( Lin et al. 2024 ) .

Figure 4: Overview of TRIG. A texture GAN discriminator provides a frozen texture prior. A feature pyramid and plausibility head are fine-tuned on localized synthetic degradations of real images. At inference, the network produces a texture-realism map for per-image scoring and a feature map whose spatial mean is used to compute the set-level distance. The feature pyramid fuses backbone features from coarse to fine.

Model	FID $\downarrow$		KID $\downarrow$ $(\times 10^{-3})$	CMMD $\downarrow$ $(\times 10^{-4})$	Gram-MMD 1 $\downarrow$ $(\times 10^{-2})$	Gram-MMD 2 $\downarrow$ $(\times 10^{-2})$	TRIG $\downarrow$
Model	Paper	Reproduced	KID $\downarrow$ $(\times 10^{-3})$	CMMD $\downarrow$ $(\times 10^{-4})$	Gram-MMD 1 $\downarrow$ $(\times 10^{-2})$	Gram-MMD 2 $\downarrow$ $(\times 10^{-2})$	TRIG $\downarrow$
ImageNet-eval	—	1.491	$0.145{\pm}0.039$	$0.048$	$1.512{\pm}0.035$	$1.561{\pm}0.069$	$0.105{\pm}0.001$
SiT-XL/2 (SDE) ( Ma et al. 2024 )	2.06	2.017	$1.860{\pm}0.087$	$2.785$	$\mathbf{7.679{\pm}0.028}$	$6.923{\pm}0.053$	$0.485{\pm}0.008$
SiT-XL/2 (ODE) ( Ma et al. 2024 )	2.15	2.024	$1.386{\pm}0.116$	$3.098$	$8.129{\pm}0.019$	$6.797{\pm}0.065$	$0.476{\pm}0.008$
DiT-XL/2-G ( Peebles and Xie 2023 )	2.27	2.175	$2.404{\pm}0.072$	$\mathbf{2.753}$	$8.320{\pm}0.032$	$6.910{\pm}0.064$	$0.490{\pm}0.008$
ADM-G ( Dhariwal and Nichol 2021 )	4.59	4.487	$1.293{\pm}0.054$	$2.869$	$9.807{\pm}0.019$	$7.569{\pm}0.043$	$1.329{\pm}0.026$

Table 1: Distances between each generative model and the ImageNet-anchor reference set under each metric. FID and KID use Inception-v3 ( Szegedy et al. 2016 ) features, CMMD uses CLIP embeddings ( Radford et al. 2021 ) , Gram-MMD 1 uses DINOv2 ( Oquab et al. 2024 ) features, and Gram-MMD 2 uses DC-AE ( Chen et al. 2025 ) features. The ImageNet-eval row reports the distance between two disjoint sets of real images. Reported values are averaged over five repeated evaluations. For KID, repeated runs account for the random sampling of 1,000-image subsets as in the original implementation. For Gram-MMD, repeated runs arise from randomly sampling subsets of the 50k images to reduce memory usage. CMMD is deterministic, as it compares all 50k CLIP features. For TRIG, repeated runs arise from the random projection directions used in the sliced Wasserstein distance. Bold indicates the smallest distance among the generative models.

Panel of Figure 5 — Figure 5: Each metric plots against human mean opinion score (MOS) on the RAISE realness benchmark ( $n=24$ groups). A texture-aware distance metric is expected to correlate with perceived realism (MOS) negatively. FID increases rather than decreases with MOS, whereas TRIG exhibits the strongest monotonic relationship with human judgments.

Figure 6: Distribution of the score ${\bm{S}}(I)$ . Each violin shows the kernel density for one set of images, with the inner box marking the interquartile range and the median.

Panel of Figure 8 — Figure 8: Texture-realism maps by our backbone. The top row shows the original images and the bottom row the corresponding heatmap (overlaped). Red color highlights regions that deviate from the natural texture statistics detected. The real image remains nearly uniform, whereas generated images show responses on textured regions.

Pre-trained Data
Variant	RAISE		ImageNet Study
	Spearman	Kendall	Spearman	Kendall
TRIG (main)	$-0.747$	$-0.594$	$-0.750$	$-0.611$
Random	$-0.577$	$-0.420$	$+0.300$	$+0.333$
FFHQ	$-0.558$	$-0.406$	$-0.667$	$-0.500$
ImageNet	$+0.349$	$+0.225$	$+0.283$	$+0.222$

Table 4: Ablation study measured by correlation with the RAISE benchmark and our human study.

Appendix figures & tables21 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 9 — Figure 9: Texture examples generated by the pretrained GAN generator pretrained on NUUR-Texture2953 ( Lin et al. 2024 ) dataset, paired with the discriminator we used.

Panel of Figure 10 — Figure 10: Natural defocus versus artificial over-smoothing. The top row shows real and generated images containing background defocus, and the bottom row shows their corresponding texture-realism maps. Green indicates greater consistency with natural-image texture statistics, while red indicates lower texture consistency. The texture realism scores $S$ are reported above the heatmaps.

Category 1: Blur (loss of local detail)
Operator	Definition
Gaussian	$I*G_{\sigma}$
Box	$I*B_{k}$ ( $k\times k$ uniform)
Motion	$I*M_{L}$ ( $1\times L$ horizontal)
Bicubic resample	$\operatorname{Up}(\operatorname{Down}(I,s),s)$ ( $s$ is the downscaling factor)
Category 2: Spectral (frequency-domain alteration)

Appendix

Table 5: Degradation pool

\mathcal{P}

G_{\sigma}

: Gaussian kernel;

\hat{I}=A\odot e^{j\Phi}

: 2D FFT with magnitude

A

, phase

\Phi

;

\rho

: frequency radius;

\tau_{0},\tau_{1}

: frequency thresholds.

\Lambda

the frequency mask,

\alpha

its coefficient.

Panel of Figure 11 — Figure 11: Region-level degradation for fine-tuning. Each row shows the original image, its SAM2 region segmentation, the segmentation overlay, and the result after a degradation operator sampled from the pool is applied within the selected regions. One operator is shown per row: frequency suppression, Gaussian noise, phase smoothing, and blur.

Component	Configuration
Input	$256\times 256$ RGB, $[-1,1]$
Trainable parameters	747,009
Optimizer	AdamW
Learning rate	$2\times 10^{-4}$
Weight decay	$10^{-4}$
Batch size	8

Appendix

Table 6: Our backbone training configuration.

Software	Version
Python	3.10.20
PyTorch	2.7.0+ cu128
torchvision	0.22.0+ cu128
CUDA build	12.8
NumPy	1.26.4
SciPy	1.15.3

Appendix

Table 7: Software configuration.

Set	Exact definition	Size
Backbone train set	Images from ImageNet official train set, excluding the final five lexicographically sorted filenames in each class and ImageNet-eval	1,105,841
Backbone val set	Final five lexicographically sorted training images per class	5,000
ImageNet-anchor	A subset of images from ImageNet official val set, with 50 images per class	50,000
ImageNet-eval	A subset of images from ImageNet official train set, with 50 images per class	50,000
Evaluated sources	12 generated-image directories, each with 50 images per class	50,000/source
NUUR-Texture2953	Separate real-texture dataset absent from all evaluation, anchor, and human-study manifests	2,953

Appendix

Table 8: Image splits and leakage controls.

	Fréchet distance	MMD distance	SWD (ours)
Inception embeddings	✗ Weak texture cues	✗ Weak texture cues	✗ Weak texture cues
	✗ Normality assumption	✓ Distribution-free	✓ Distribution-free
	✓ Scales, $O(n)$	✗ $O(n^{2})$ , subset-limited	✓ Scales, $O(n\log n)$
CLIP / DINOv2 embeddings	✓ Semantic embeddings	✓ Semantic embeddings	✓ Semantic embeddings
	✗ Normality assumption	✓ Distribution-free	✓ Distribution-free
	✓ Scales, $O(n)$	✗ $O(n^{2})$ , subset-limited	✓ Scales, $O(n\log n)$

Appendix

Table 9: Design space of set-level generative-evaluation metrics along two axes: the backbone (rows) and the distributional distance (columns). Along the distance axis, the Fréchet distance scales cheaply but assumes Gaussian features; MMD is distribution-free but costs

O(n^{2})

and is limited to subsets at large

n

; SWD is both distribution-free and

O(n\log n)

, scaling to the sample sizes at which FID is reported.

Fine-tuned layers
Variant	RAISE		ImageNet
Variant	Spear.	Kend.	Spear.	Kend.
TRIG (main)	$\mathbf{-0.747}$	$\mathbf{-0.594}$	$\mathbf{-0.750}$	$\mathbf{-0.611}$
FPN+b16	$-0.730$	$-0.551$	$-0.667$	$-0.556$
FPN+b16+b32	$-0.650$	$-0.500$	$-0.650$	$-0.500$
b16 only	$-0.339$	$-0.217$	$-0.733$	$-0.556$

Appendix

Table 10: Additional ablation across both studies. Values are Spearman/Kendall correlations with human ratings (lower/more negative

=

better agreement).

Property	Value
Participants (sessions)	24
Trials per participant	200
Total comparisons	4,800
Image sources	10 (2 real + 8 generators)
Responses per trial	Left / Tie / Right
Session duration	$\approx$ 10-20 min

Appendix

Table 11: Human study protocol. Sources are two disjoint real ImageNet sets and eight generators (ADM, BigGAN, DiT, PMF, StyleGAN-XL, SiT-ODE, SiT-SDE, SoFlow). The two real sets serve as a sanity check: because both are genuinely real, participants should find them indistinguishable, so a strong preference between them would indicate unreliable or biased judging.

Panel of Figure 12 — Figure 12: Study interface. Top: instruction screen defining the task and the Left/Tie/Right controls. Bottom: a trial screen showing two same-class images with the three response buttons and a progress counter.

Figure 13: Human preference ranking. Per-source win-rate for all ten sources, sorted, with $95\%$ bootstrap confidence intervals. The two real sets (green) sit well above the chance line ( $0.5$ ); the eight generators cluster at or below chance with overlapping intervals, showing that generator-vs-generator preference is weaker while real-vs-generated is decisive.

Panel of Figure 14 — Figure 14: DiT preferred over StyleGAN-XL. 8 representative same-class trials (sampled from the human study logs) in which participants judged DiT (left of each pair, preferred) to have more realistic texture than StyleGAN-XL (right).

Panel of Figure 15 — Figure 15: SiT-SDE preferred over SoFlow. 8 representative same-class trials in which participants judged SiT-SDE (left, preferred) to have more realistic texture than SoFlow (right).

Metric	$\rho$	Trial CI	Participant CI
FID	$+0.467$	$[+0.300,+0.517]$	$[+0.267,+0.533]$
KID	$-0.100$	$[-0.300,+0.033]$	$[-0.283,+0.017]$
Gram-MMD (DINOv2)	$-0.600$	$[-0.767,-0.467]$	$[-0.733,-0.500]$
Gram-MMD (DCAE)	$-0.283$	$[-0.433,-0.167]$	$[-0.450,-0.167]$
CMMD	$-0.683$	$[-0.800,-0.617]$	$[-0.783,-0.617]$
TRIG	$\mathbf{-0.750}$	$\mathbf{[-0.817,-0.550]}$	$\mathbf{[-0.817,-0.567]}$

Appendix

Table 12: Spearman correlation with the human source ranking and percentile 95% confidence intervals under different bootstrap procedures. The primary interval uses trial resampling, and an alternative resampling of participants. All correlations are computed over

n=9

non-anchor sources. More negative values indicate better agreement with human judgments.

Model	Official code	Sampling	Parameters
ADM-G	https://github.com/openai/guided-diffusion	Class-conditional DDIM sampling with 1000 steps.	Classifier guidance 1.0; FP16.
BigGAN-deep	https://github.com/huggingface/pytorch-pretrained-BigGAN	One synthesis pass with truncated-normal latents and one-hot class labels.	Truncation 1.0.
DiT-XL/2	https://github.com/facebookresearch/DiT	Class-conditional latent diffusion with 250 DDPM steps.	CFG 1.5.
SiT-XL/2 ODE	https://github.com/willisma/SiT	Class-conditional latent-flow sampling with adaptive Dopri5.	CFG 1.5; linear transport; velocity prediction; $\mathrm{atol}=10^{-6}$ , $\mathrm{rtol}=10^{-3}$ .
SiT-XL/2 SDE	https://github.com/willisma/SiT	Euler–Maruyama SDE sampling with 250 steps.	CFG 1.5; diffusion_form=sigma ; norm 1.0; final estimator Mean , size 0.04.
StyleGAN-XL	https://github.com/autonomousvision/stylegan-xl	One synthesis pass with one-hot class labels and constant synthesis noise.	Truncation $\psi=1.0$ .

Appendix

Table 13: Sampling configurations for the eight generators included in the human study. Sampling steps, guidance settings, and other hyperparameters follow the corresponding official implementations.

Model	Official code	Sampling	Parameters
Self-Flow	https://github.com/black-forest-labs/Self-Flow	Class-conditional SDE sampling with 250 steps.	CFG 1.0; guidance interval $[0,0.7]$ .
VAR-d30	https://github.com/FoundationVision/VAR	Coarse-to-fine next-scale autoregressive sampling with hard token selection.	CFG 1.5; top- $k=900$ ; top- $p=0.96$ ; more_smooth=False .
RAR-XXL	https://github.com/bytedance/1d-tokenizer	Randomized autoregressive token sampling followed by VQGAN decoding.	Temperature 1.02; guidance 8.0; guidance exponent 1.2.
LlamaGen 3B	https://github.com/FoundationVision/LlamaGen	Autoregressive sampling of $24^{2}$ VQ tokens followed by VQ decoding. The $384^{2}$ outputs are bilinearly resized to $256^{2}$ .	CFG 1.65; temperature 1.0; top- $k=0$ ; top- $p=1.0$ ; cfg_interval=-1 .

Appendix

Table 14: Additional generators in the twelve-source evaluation. Sampling steps, guidance settings, and other hyperparameters follow the corresponding official implementations.

Panel of Figure 16 — Figure 16: Example images from the RAISE benchmark ( Mukherjee et al. 2025 ) . AI-generated images shown across the perceived-realness range, from low mean opinion score (left) to high (right). Our protocol sorts the 480 AI-generated images by MOS and partitions them into 24 quality strata (Section F ).

Panel of Figure 17 — Figure 17: Metric score vs. blur strength for the real ImageNet set and 12 generators. As blur becomes severe, FID and KID rate the generated sets as more realistic than the real set (wrong direction); CMMD and Gram-MMD (DINOv2) keep separating the sources even when texture is largely destroyed; TRIG’s scores converge as texture detail is removed, consistent with a metric driven by texture quality.

Panel of Figure 18 — Figure 18: Several manual attacks on the texture-realism backbone. It is sensitive to different degradation operators and JPEG degradations, and the low responses (red regions) correspond to the manually degraded regions.

Panel of Figure 19 — Figure 19: Samples across sources (rows) and ImageNet classes (columns). Top row: real ImageNet; remaining rows: twelve generators.

Explore similar work

Apr 3, 2026

Gram-MMD: A Texture-Aware Metric for Image Realism Assessment

Evaluating the realism of generated images remains a fundamental challenge in generative modeling. Existing distributional metrics such as the Frechet Inception Distance (FID) and CLIP-MMD (CMMD) compare feature distributions at a semantic level but may overlook fine-grained textural information that can be relevant for distinguishing real from generated images. We introduce Gram-MMD (GMMD), a realism metric that leverages Gram matrices computed from intermediate activations of pretrained backbone networks to capture correlations between feature maps. By extracting the upper-triangular part of these symmetric Gram matrices and measuring the Maximum Mean Discrepancy (MMD) between an anchor distribution of real images and an evaluation distribution, GMMD produces a representation that encodes textural and structural characteristics at a finer granularity than global embeddings. To select the hyperparameters of the metric, we employ a meta-metric protocol based on controlled degradations applied to MS-COCO images, measuring monotonicity via Spearman's rank correlation and Kendall's tau. We conduct experiments on both the KADID-10k database and the RAISE realness assessment dataset using various backbone architectures, including DINOv2, DC-AE, Stable Diffusion's VAE encoder, VGG19, and the AlexNet backbone from LPIPS, among others. We also demonstrate on a cross-domain driving scenario (KITTI / Virtual KITTI / Stanford Cars) that CMMD can incorrectly rank real images as less realistic than synthetic ones due to its semantic bias, while GMMD preserves the correct ordering. Our results suggest that GMMD captures complementary information to existing semantic-level metrics.

Joé Napolitano, Pascal Nguyen

AMIAD

Image Generation Evaluation Image Quality Assessment

Ask PDF

Mar 11, 2026

Too Vivid to Be Real? Benchmarking and Calibrating Generative Color Fidelity

Recent advances in text-to-image (T2I) generation have greatly improved visual quality, yet producing images that appear visually authentic to real-world photography remains challenging. This is partly due to biases in existing evaluation paradigms: human ratings and preference-trained metrics often favor visually vivid images with exaggerated saturation and contrast, which make generations often too vivid to be real even when prompted for realistic-style images. To address this issue, we present Color Fidelity Dataset (CFD) and Color Fidelity Metric (CFM) for objective evaluation of color fidelity in realistic-style generations. CFD contains over 1.3M real and synthetic images with ordered levels of color realism, while CFM employs a multimodal encoder to learn perceptual color fidelity. In addition, we propose a training-free Color Fidelity Refinement (CFR) that adaptively modulates spatial-temporal guidance scale in generation, thereby enhancing color authenticity. Together, CFD supports CFM for assessment, whose learned attention further guides CFR to refine T2I fidelity, forming a progressive framework for assessing and improving color fidelity in realistic-style T2I generation. The dataset and code are available at https://github.com/ZhengyaoFang/CFM.

Zhengyao Fang, Zexi Jia, Yijia Zhong +5

Harbin Institute of Technology, Shenzhen · Independent Researcher · College of Computer Science and Aritificial Intelligence, Fudan University +2

Image Generation Image Generation Evaluation T2I Generation Evaluation

Ask PDF

Mar 7, 2026

TIQA: Human-Aligned Perceptual Text Quality Assessment in Generated Images

Recent text-to-image models have improved global realism, but text rendering remains a persistent failure mode: images may look convincing overall, yet local typography often contains malformed glyphs, broken strokes, irregular spacing, and other artifacts that humans heavily penalize. We formulate Text-in-Image Quality Assessment (TIQA), a no-reference task that estimates a human-aligned perceptual quality score for detected text regions while disentangling visual text quality from semantic correctness. To support this setting, we introduce two datasets. TIQA-Crops contains 120k text crops from 36k AI-generated images produced by 12 generators, with 10k mean-opinion-score (MOS) labels and 110k proxy labels for pretraining. TIQA-Images contains 1,500 text-heavy images from 10 recent generators, including proprietary systems, with paired overall-quality and text-quality subjective scores. We also propose ANTIQA, a lightweight predictor with text-specific inductive biases. Across crop-level and image-level evaluations, ANTIQA achieves the best alignment with human judgments, reaching PLCC/SROCC of 0.942/0.935 on TIQA-Crops and 0.842/0.837 for text-quality MOS on unseen generators in TIQA-Images. In best-of-5 AI-generated image ranking, ANTIQA improves the text quality of the selected image by 0.36 MOS (14%), demonstrating utility for benchmarking, filtering, and generation-time selection. Together, these findings establish perceptual text quality as a distinct evaluation target for modern text-to-image generation.

Kirill Koltsov, Aleksandr Gushchin, Anastasia Antsiferova +1

Lomonosov Moscow State University Moscow, Russia · ISP RAS Research Center for Trusted Artificial Intelligence Lomonosov Moscow State University Moscow, Russia · ISP RAS Research Center for Trusted Artificial Intelligence MSU Institute for Artificial Intelligence Moscow, Russia +1

Image Quality Assessment T2I Generation Evaluation

Ask PDF