CLPIPS: A Personalized Metric for AI-Generated Image Similarity

Khoi Trinh; Jay Rothenberger; Scott Seidenberger; Dimitrios Diochnos; Anindya Maiti

cs.CVMar 26, 2026

CLPIPS: A Personalized Metric for AI-Generated Image Similarity

Authors: Khoi Trinh, Jay Rothenberger, Scott Seidenberger, Dimitrios Diochnos, Anindya Maiti

Organizations: University of Oklahoma

Abstract

Iterative prompt refinement is central to reproducing target images with text to image generative models. Previous studies have incorporated image similarity metrics (ISMs) as additional feedback to human users. Existing ISMs such as LPIPS and CLIP provide objective measures of image likeness but often fail to align with human judgments, particularly in context specific or user driven tasks. In this paper, we introduce Customized Learned Perceptual Image Patch Similarity (CLPIPS), a customized extension of LPIPS that adapts a metric's notion of similarity directly to human judgments. We aim to explore whether lightweight, human augmented fine tuning can meaningfully improve perceptual alignment, positioning similarity metrics as adaptive components for human in the loop workflows with text to image tools. We evaluate CLPIPS on a human subject dataset in which participants iteratively regenerate target images and rank generated outputs by perceived similarity. Using margin ranking loss on human ranked image pairs, we fine tune only the LPIPS layer combination weights and assess alignment via Spearman rank correlation and Intraclass Correlation Coefficient. Our results show that CLPIPS achieves stronger correlation and agreement with human judgments than baseline LPIPS. Rather than optimizing absolute metric performance, our work emphasizes improving alignment consistency between metric predictions and human ranks, demonstrating that even limited human specific fine tuning can meaningfully enhance perceptual alignment in human in the loop text to image workflows.

Figures & tables

Panel of Figure 1 — Figure 1: Workflow of CLPIPS fine-tuning. Human-generated and ranked images are converted into pairwise tuples to fine-tune LPIPS using a margin ranking loss, producing CLPIPS, which is then evaluated against human similarity rankings.

Panel of Figure 2 — Figure 2: An example of the two main survey tasks.

ICC Range	Interpretation
$<0.50$	Poor
$0.50$ – $0.75$	Moderate
$0.75$ – $0.90$	Good
$>0.90$	Excellent

Table 1: Interpretation of Intraclass Correlation Coefficient (ICC) values by Koo and Li Koo and Li (2016)

ICC Range	Interpretation
$<0.40$	Poor
$0.4$ – $0.6$	Fair
$0.6$ – $0.75$	Good
$>0.75$	Excellent

Table 2: Interpretation of Intraclass Correlation Coefficient (ICC) values by Cicchetti Cicchetti (1994)

Model	ICC 2k	95% CI	$\mathbf{p}$ -value
CLPIPS	0.68	[0.66, 0.71]	$\ll 0.001$
LPIPS	0.60	[0.57, 0.64]	$\ll 0.001$
Spearman rank correlation ( $\rho$ ) with human ranks
CLPIPS	$\rho=0.524$			$p\ll 0.001$
LPIPS	$\rho=0.432$			$p\ll 0.001$

Table 3: Alignment of metric rankings with human rankings.

Figure 3: Comparison of human similarity rankings with LPIPS and CLPIPS rankings for the same set of generated images relative to a target image. Images are indexed according to their human-assigned rank (shown in the left column). The same index numbers are shown next to the corresponding images in the LPIPS and CLPIPS columns, indicating where each human-ranked image appears in the metric-based orderings. CLPIPS places images closer to their human-ranked positions than LPIPS, exhibiting fewer rank inversions. In this example, CLPIPS achieved an ICC of $0.45$ while LPIPS achieved an ICC of $-1.41$ .

Comparison	$\Delta$ ICC	95% CI	$\mathbf{p}$ -value
CLPIPS – LPIPS	0.084	[0.071, 0.1]	$<<0.001$

Table 4: Paired bootstrap analysis comparing ICC agreement between CLPIPS and LPIPS over target image sets.

\Delta

ICC shows the average increase in agreement in CLPIPS relative to LPIPS, while the bootstrap confidence interval indicates the stability of this improvement across resampled image sets.

T2I Generation Evaluation

Text-to-image generation evaluation measures how accurately, safely, and consistently image models turn prompts into images.

See all 157 papers

Jul 20, 2026

The Many Senses of Visual Similarity: A Text-Prompted Image Perceptual Metric

Human visual similarity judgments are context-dependent. For example, two images may be similar in shape but distinct in color. Existing perceptual similarity metrics, however, collapse these nuances into a single scalar value, offering no mechanism to condition on specific aspects. To bridge this gap, we introduce a large-scale dataset of human similarity judgments over image triplets, where each triplet is annotated across multiple, free-form semantic aspects of similarity. Benchmarking a broad range of frontier vision-language models (VLMs) reveals a considerable performance gap compared to human annotators' consensus. Leveraging our data, we fine-tune a VLM to produce our Text-Prompted Image Perceptual Similarity (TPIPS) metric, capturing multiple senses of visual similarity depending on the specified text prompt. We demonstrate that TPIPS aligns more closely with human perception and generalizes reliably beyond the training distribution. Finally, we show that TPIPS unlocks new capabilities in text-guided retrieval, compositional search, and the fine-grained evaluation of generative models. Our code, data, and trained models are at https://peterwang512.github.io/TPIPS

Sheng-Yu Wang, Yotam Nitzan, Aaron Hertzmann +4

1Carnegie Mellon University · 2Adobe Research · 3UC Berkeley

VLM Evaluation T2I Generation Evaluation Human Visual Perception

Ask PDF

Jul 7, 2026

PIPBench: A Profile-Inclusive Framework for Personalized Image Generation Evaluation

Recent text-to-image models such as DALLE-3 excel at following diverse prompts yet remain blind to individual aesthetic preferences. We study personalized image generation, where models must align outputs with a user's implicit visual preferences based on a few historically preferred images and a short prompt. To this end, we introduce PIPBench, the first profile-inclusive benchmark for evaluating personalized image generation. We further propose a novel data construction pipeline that leverages psychological and demographic profiling dimensions for both real-user data collection and scalable agent-based data generation. Using PIPBench, we conduct a thorough evaluation of representative line of methods. Our experiments reveal key limitations in existing methods, suggesting new challenges and opportunities for personalized text-to-image synthesis. Project page: https://wuyuhang05.github.io/PIPBench/

Yuhang Wu, Shuxiang Zhang, Wee Hian Ching +2

College of AI, Tsinghua University · Sun Yat-sen University · Shanghai Qi Zhi Institute

Image Generation Synthetic Benchmark Generation Image Generation Evaluation T2I Generation Evaluation

Ask PDF

Mar 7, 2026

TIQA: Human-Aligned Perceptual Text Quality Assessment in Generated Images

Recent text-to-image models have improved global realism, but text rendering remains a persistent failure mode: images may look convincing overall, yet local typography often contains malformed glyphs, broken strokes, irregular spacing, and other artifacts that humans heavily penalize. We formulate Text-in-Image Quality Assessment (TIQA), a no-reference task that estimates a human-aligned perceptual quality score for detected text regions while disentangling visual text quality from semantic correctness. To support this setting, we introduce two datasets. TIQA-Crops contains 120k text crops from 36k AI-generated images produced by 12 generators, with 10k mean-opinion-score (MOS) labels and 110k proxy labels for pretraining. TIQA-Images contains 1,500 text-heavy images from 10 recent generators, including proprietary systems, with paired overall-quality and text-quality subjective scores. We also propose ANTIQA, a lightweight predictor with text-specific inductive biases. Across crop-level and image-level evaluations, ANTIQA achieves the best alignment with human judgments, reaching PLCC/SROCC of 0.942/0.935 on TIQA-Crops and 0.842/0.837 for text-quality MOS on unseen generators in TIQA-Images. In best-of-5 AI-generated image ranking, ANTIQA improves the text quality of the selected image by 0.36 MOS (14%), demonstrating utility for benchmarking, filtering, and generation-time selection. Together, these findings establish perceptual text quality as a distinct evaluation target for modern text-to-image generation.

Kirill Koltsov, Aleksandr Gushchin, Anastasia Antsiferova +1

Lomonosov Moscow State University Moscow, Russia · ISP RAS Research Center for Trusted Artificial Intelligence Lomonosov Moscow State University Moscow, Russia · ISP RAS Research Center for Trusted Artificial Intelligence MSU Institute for Artificial Intelligence Moscow, Russia +1

Image Quality Assessment T2I Generation Evaluation Text Rendering in T2I Generation

Ask PDF

Abstract

T2I Generation Evaluation

Explore similar work

The Many Senses of Visual Similarity: A Text-Prompted Image Perceptual Metric

PIPBench: A Profile-Inclusive Framework for Personalized Image Generation Evaluation

TIQA: Human-Aligned Perceptual Text Quality Assessment in Generated Images