What Do Vision-Language Models Encode for Personalized Image Aesthetics Assessment?

Koki Ryu; Hitomi Yanaka

cs.CVApr 13, 2026

What Do Vision-Language Models Encode for Personalized Image Aesthetics Assessment?

Authors: Koki Ryu, Hitomi Yanaka

Organizations: The University of Tokyo · Riken · Tohoku University

Abstract

Personalized image aesthetics assessment (PIAA) is an important research problem with practical real-world applications. While methods based on vision-language models (VLMs) are promising candidates for PIAA, it remains unclear whether they internally encode rich, multi-level aesthetic attributes required for effective personalization. In this paper, we first analyze the internal representations of VLMs to examine the presence and distribution of such aesthetic attributes, and then leverage them for lightweight, individual-level personalization without model fine-tuning. Our analysis reveals that VLMs encode diverse aesthetic attributes that propagate into the language decoder layers. Building on these representations, we demonstrate that simple linear models can perform PIAA effectively. We further analyze how aesthetic information is transferred across layers in different VLM architectures and across image domains. Our findings provide insights into how VLMs can be utilized for modeling subjective, individual aesthetic preferences. Our code is available at https://github.com/ynklab/vlm-latent-piaa.

Figures & tables

Figure 1: Overview of PIAA using VLM representations. $s$ and $W$ denote user-specific aesthetics scores and linear estimators respectively. Aesthetic attributes encoded in VLM hidden representations are linearly transformed to predict user-specific aesthetic scores without model fine-tuning.

Attribute	Qwen3-VL						Gemma 3				DINOv3
	2B		4B		8B		4B		12B		ViT-B/16		ViT-L/16
BalancingElements	$0.325$ (	13 )	0.332 (	0 )	$0.300$ (	14 )	$0.309$ (	24 )	$0.307$ (	14 )	$0.317$ (	11 )	$0.307$ (	22 )
ColorHarmony	$0.516$ (	9 )	0.523 (	9 )	$0.515$ (	6 )	$0.493$ (	24 )	$0.504$ (	32 )	$0.479$ (	6 )	$0.482$ (	10 )
Content	0.633 (	10 )	$0.632$ (	13 )	$0.627$ (	15 )	$0.621$ (	12 )	$0.624$ (	35 )	$0.551$ (	12 )	$0.579$ (	17 )
DoF	0.535 (	10 )	$0.518$ (	10 )	$0.530$ (	16 )	$0.512$ (	9 )	$0.515$ (	15 )	$0.506$ (	7 )	$0.507$ (	18 )
Light	0.509 (	14 )	$0.507$ (	12 )	$0.490$ (	14 )	$0.452$ (	18 )	$0.468$ (	6 )	$0.439$ (	8 )	$0.436$ (	11 )

Table 1: Highest Spearman correlation achieved by linear probing on

\mathbf{LT}

layers for each aesthetic attribute in AADB. Values in parentheses indicate the corresponding layer indices.

Figure 2: Layer-wise probing performance across

\mathbf{V}

and

\mathbf{LT}

layers for Qwen3-VL 2B on AADB. The dotted line indicates the boundary between

V

and

LT

Figure 3: Layer-wise probing performance for the Content attribute in Qwen3-VL 2B and Gemma 3 4B.

Method	Support	Qwen3-VL						Gemma 3
		2B		4B		8B		4B		12B
		$\rho$ / $R^{2}$		$\rho$ / $R^{2}$		$\rho$ / $R^{2}$		$\rho$ / $R^{2}$		$\rho$ / $R^{2}$
PARA
Raw Text		$0.504$ /	$-0.571$	$0.570$ /	$-1.277$	$0.528$ /	$-0.729$	$0.462$ /	$-1.107$	$0.493$ /	$-1.879$
Few-shot	10-shot	$0.319$ /	$-1.850$	$0.197$ /	$-1.576$	$0.372$ /	$-0.547$	$0.241$ /	$-0.537$	$0.407$ /	$-0.185$
Adjust-Bias	100-shot	$0.504$ /	$-0.310$	$0.570$ /	$-0.672$	$0.528$ /	$-0.441$	$0.462$ /	$-0.321$	$0.493$ /	$-1.562$

Table 2: User-averaged PIAA performance on PARA and LAPIS. Each model column reports Spearman’s

\rho

and

R^{2}

. Best values per column are highlighted in bold.

Figure 4: Layer-wise PIAA performance across

\mathbf{V}

\mathbf{LV}

, and

\mathbf{LT}

representations for multiple models and datasets.

Panel of Figure 5 — Figure 5: Examples of the LAPIS images assigned high (top) and low (bottom) scores by different methods for a representative user.

Method	PARA
Linear-Hidden	0.611 / 0.362
PIAA-ICI (w/o User Attr)	0.620 / 0.392
PIAA-ICI (w/ User Attr)	0.619 / 0.424

Table 3: In-domain comparison on PARA. Linear-Hidden uses Qwen3-VL 4B with

\mathbf{LT}_{15}

. Values are reported as

\rho/R^{2}

Method	PARA	LAPIS
Linear-Hidden	0.611 / 0.362	0.568 / 0.319
PIAA-ICI (w/o User Attr)	0.620 / 0.392	0.206 / -0.062

Table 4: Cross-domain comparison. PIAA-ICI is pretrained on PARA, while Linear-Hidden uses pretrained VLM representations without dataset-specific pretraining. Values are reported as

\rho/R^{2}

Method	Support	Qwen3-VL						Gemma 3
		2B		4B		8B		4B		12B
		$\rho$ / $R^{2}$		$\rho$ / $R^{2}$		$\rho$ / $R^{2}$		$\rho$ / $R^{2}$		$\rho$ / $R^{2}$
Raw Text		$0.380$ /	$-2.292$	$0.405$ /	$-4.308$	$0.387$ /	$-2.943$	$0.347$ /	$-3.109$	$0.330$ /	$-6.295$
Linear-Hidden	100-shot	0.467/	$0.058$	0.472/	$0.053$	0.463/	$0.059$	0.463/	$0.079$	0.458/	$0.018$
Linear-Hidden (GIAA)	100-shot	$0.428$ /	$-0.908$	$0.435$ /	$-0.867$	$0.440$ /	$-0.894$	$0.422$ /	$-1.093$	$0.432$ /	$-0.930$
Linear-Hidden (Reduce)	100-shot	$0.431$ /	0.150	$0.447$ /	0.157	$0.447$ /	0.085	$0.436$ /	0.145	$0.450$ /	0.159

Table 5: PIAA performance on PARA for users with low agreement to GIAA (“hard” users).

Figure 6: Distribution of Spearman correlation between PIAA and GIAA scores across users in PARA and LAPIS.

Appendix figures & tables21 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 7 — Figure 7: Prompt template used for the Few-shot PIAA baseline.

Figure 8: Distribution of aesthetic attribute annotations in AADB.

Figure 9: Distribution of aesthetic attribute annotations in PARA.

Panel of Figure 10 — Figure 10: Spearman correlation between aesthetic attributes in AADB (top) and PARA (bottom).

Attribute	Qwen3-VL						Gemma 3				DINOv3
	2B		4B		8B		4B		12B		ViT-B/16		ViT-L/16
BalancingElements	$0.325$ (	21 )	0.328 (	23 )	$0.307$ (	26 )	$0.300$ (	26 )	$0.300$ (	26 )	$0.317$ (	11 )	$0.307$ (	22 )
ColorHarmony	0.516 (	23 )	$0.513$ (	23 )	$0.508$ (	22 )	$0.502$ (	10 )	$0.502$ (	10 )	$0.479$ (	6 )	$0.482$ (	10 )
Content	0.621 (	23 )	$0.621$ (	23 )	$0.608$ (	26 )	$0.579$ (	26 )	$0.579$ (	26 )	$0.551$ (	12 )	$0.579$ (	17 )
DoF	$0.532$ (	9 )	$0.530$ (	9 )	0.535 (	12 )	$0.513$ (	10 )	$0.513$ (	10 )	$0.506$ (	7 )	$0.507$ (	18 )
Light	$0.488$ (	15 )	0.500 (	12 )	$0.497$ (	14 )	$0.474$ (	10 )	$0.474$ (	10 )	$0.439$ (	8 )	$0.436$ (	11 )

Appendix

Table 6: Highest Spearman correlation achieved by linear probing on

\mathbf{V}

layers for each aesthetic attribute in AADB. Values in parentheses indicate the corresponding layer indices.

Attribute	Qwen3-VL						Gemma 3
	2B		4B		8B		4B		12B
BalancingElements	$0.331$ (	7 )	0.349 (	13 )	$0.314$ (	23 )	$0.307$ (	1 )	$0.300$ (	1 )
ColorHarmony	$0.517$ (	9 )	0.519 (	1 )	$0.517$ (	9 )	$0.478$ (	21 )	$0.489$ (	12 )
Content	0.635 (	9 )	$0.627$ (	1 )	$0.622$ (	26 )	$0.600$ (	18 )	$0.599$ (	19 )
DoF	$0.533$ (	6 )	0.534 (	3 )	$0.516$ (	1 )	$0.471$ (	14 )	$0.478$ (	17 )
Light	$0.470$ (	10 )	0.478 (	11 )	$0.474$ (	8 )	$0.423$ (	20 )	$0.428$ (	7 )

Appendix

Table 7: Highest Spearman correlation achieved by linear probing on

\mathbf{LV}

layers for each aesthetic attribute in AADB. Values in parentheses indicate the corresponding layer indices.

Attribute	Qwen3-VL						Gemma 3
	2B		4B		8B		4B		12B
BalancingElements	$0.303$ (	5 )	0.322 (	6 )	$0.295$ (	1 )	$0.306$ (	14 )	$0.302$ (	14 )
ColorHarmony	$0.511$ (	11 )	0.526 (	22 )	$0.526$ (	28 )	$0.493$ (	12 )	$0.496$ (	15 )
Content	0.629 (	11 )	$0.625$ (	13 )	$0.618$ (	25 )	$0.612$ (	6 )	$0.621$ (	34 )
DoF	0.531 (	15 )	$0.520$ (	13 )	$0.521$ (	34 )	$0.500$ (	1 )	$0.524$ (	15 )
Light	$0.483$ (	16 )	0.510 (	26 )	$0.485$ (	25 )	$0.442$ (	10 )	$0.469$ (	5 )

Appendix

Table 8: Highest Spearman correlation achieved by linear probing on

\mathbf{L\tau}

layers for each aesthetic attribute in AADB. Values in parentheses indicate the corresponding layer indices.

Method	Support	Qwen3-VL						Gemma 3
		2B		4B		8B		4B		12B
		$\rho$ / $R^{2}$		$\rho$ / $R^{2}$		$\rho$ / $R^{2}$		$\rho$ / $R^{2}$		$\rho$ / $R^{2}$
Raw Text		$0.504$ /	$-0.571$	$0.570$ /	$-1.277$	$0.528$ /	$-0.729$	$0.462$ /	$-1.107$	$0.493$ /	$-1.879$
Adjust-Bias	10-shot	$0.504$ /	$-0.385$	$0.570$ /	$-0.765$	$0.528$ /	$-0.517$	$0.462$ /	$-0.425$	$0.493$ /	$-1.727$
Few-shot	10-shot	$0.319$ /	$-1.850$	$0.197$ /	$-1.576$	$0.372$ /	$-0.547$	$0.241$ /	$-0.537$	$0.407$ /	$-0.185$
LoRA	10-shot	$0.476$ /	$-1.867$	$0.581$ /	$-1.383$	$0.567$ /	$-1.206$	$0.503$ /	$-1.464$	$0.542$ /	$-1.231$

Appendix

Table 9: Full PIAA results on PARA.

Method	Support	Qwen3-VL						Gemma 3
		2B		4B		8B		4B		12B
		$\rho$ / $R^{2}$		$\rho$ / $R^{2}$		$\rho$ / $R^{2}$		$\rho$ / $R^{2}$		$\rho$ / $R^{2}$
Raw Text		$0.098$ /	$-0.778$	$0.176$ /	$-0.937$	$0.175$ /	$-0.763$	$0.119$ /	$-1.340$	$0.233$ /	$-1.335$
Adjust-Bias	10-shot	$0.098$ /	$-0.399$	$0.176$ /	$-0.382$	$0.175$ /	$-0.345$	$0.119$ /	$-0.284$	$0.233$ /	$-0.587$
Few-shot	10-shot	$0.142$ /	$-1.265$	$0.221$ /	$-0.380$	$0.264$ /	$-0.480$	$0.127$ /	$-0.354$	$0.227$ /	$-0.459$
LoRA	10-shot	$-0.011$ /	$-0.533$	$0.188$ /	$-1.290$	$0.137$ /	$-1.465$	$0.174$ /	$-1.315$	$0.249$ /	$-1.169$

Appendix

Table 10: Full PIAA results on LAPIS.

Method	Model	Repr	PARA	LAPIS
Linear-Hidden	Qwen3-VL 2B	$V_{23}$	0.581 / 0.363	0.595 / 0.360
Linear-Hidden (Reduce)	Qwen3-VL 2B	$V_{23}$	0.520 / 0.266	0.509 / 0.259
Linear-Hidden	Gemma 3 4B	$V_{26}$	0.573 / 0.315	0.537 / 0.290
Linear-Hidden (Reduce)	Gemma 3 4B	$V_{26}$	0.481 / 0.198	0.429 / 0.175

Appendix

Table 11: PIAA performance using vision encoder representations. Values are reported as

\rho/R^{2}

Method	Support	Qwen3-VL						Gemma 3
		2B		4B		8B		4B		12B
PARA
Few-shot	10-shot	$28$ (	$14.0$ %)	$153$ (	$76.5$ %)	$42$ (	$21.0$ %)	$94$ (	$47.0$ %)	$2$ (	$1.0$ %)
Linear-Hidden	10-shot	$4$ (	$2.0$ %)	$4$ (	$2.0$ %)	$4$ (	$2.0$ %)	$4$ (	$2.0$ %)	$4$ (	$2.0$ %)
Linear-Hidden (Reduce)	10-shot	$4$ (	$2.0$ %)	$4$ (	$2.0$ %)	$4$ (	$2.0$ %)	$4$ (	$2.0$ %)	$4$ (	$2.0$ %)
Few-shot	100-shot	– (	– %)	– (	– %)	– (	– %)	$71$ (	$35.5$ %)	– (	– %)

Appendix

Table 12: Number of users for which Spearman correlation is undefined due to constant predictions or labels. Rows with zero NaN users for every model are omitted within each dataset block.

Method	Support	Qwen3-VL			Gemma 3
		2B	4B	8B	4B	12B
		$[\rho_{\text{min}},\ \rho_{\text{max}}]$	$[\rho_{\text{min}},\ \rho_{\text{max}}]$	$[\rho_{\text{min}},\ \rho_{\text{max}}]$	$[\rho_{\text{min}},\ \rho_{\text{max}}]$	$[\rho_{\text{min}},\ \rho_{\text{max}}]$
Raw Text		$[0.486,\ 0.521]$	$[0.549,\ 0.587]$	$[0.510,\ 0.546]$	$[0.442,\ 0.481]$	$[0.472,\ 0.514]$
Few-shot	10-shot	$[0.297,\ 0.339]$	$[0.154,\ 0.242]$	$[0.341,\ 0.404]$	$[0.209,\ 0.272]$	$[0.384,\ 0.430]$
Adjust-Bias	100-shot	$[0.486,\ 0.522]$	$[0.550,\ 0.588]$	$[0.510,\ 0.546]$	$[0.443,\ 0.481]$	$[0.472,\ 0.515]$
LoRA	100-shot	$[0.469,\ 0.505]$	$[0.558,\ 0.596]$	$[0.548,\ 0.586]$	$[0.468,\ 0.510]$	$[0.504,\ 0.543]$

Appendix

Table 13: User-averaged Spearman correlation with 95% bootstrap confidence intervals on PARA.

Method	Support	Qwen3-VL			Gemma 3
		2B	4B	8B	4B	12B
		$[\rho_{\text{min}},\ \rho_{\text{max}}]$	$[\rho_{\text{min}},\ \rho_{\text{max}}]$	$[\rho_{\text{min}},\ \rho_{\text{max}}]$	$[\rho_{\text{min}},\ \rho_{\text{max}}]$	$[\rho_{\text{min}},\ \rho_{\text{max}}]$
Raw Text		$[0.077,\ 0.118]$	$[0.153,\ 0.199]$	$[0.150,\ 0.200]$	$[0.098,\ 0.139]$	$[0.204,\ 0.263]$
Few-shot	10-shot	$[0.113,\ 0.170]$	$[0.193,\ 0.251]$	$[0.235,\ 0.294]$	$[0.093,\ 0.161]$	$[0.192,\ 0.259]$
Adjust-Bias	100-shot	$[0.076,\ 0.118]$	$[0.152,\ 0.200]$	$[0.149,\ 0.202]$	$[0.098,\ 0.139]$	$[0.204,\ 0.263]$
LoRA	100-shot	$[-0.001,\ 0.050]$	$[0.131,\ 0.176]$	$[0.135,\ 0.192]$	$[0.094,\ 0.138]$	$[0.171,\ 0.230]$

Appendix

Table 14: User-averaged Spearman correlation with 95% bootstrap confidence intervals on LAPIS.

Figure 11: Example images from AADB with applied augmentations.

Figure 12: Probing performance under different image augmentations for Qwen3-VL 2B.

Attribute	Qwen3-VL						Gemma 3				DINOv3
	2B		4B		8B		4B		12B		ViT-B/16		ViT-L/16
color	$0.884$ (	13 )	0.886 (	18 )	$0.886$ (	17 )	$0.874$ (	8 )	$0.880$ (	13 )	$0.845$ (	8 )	$0.855$ (	16 )
composition	$0.887$ (	12 )	0.890 (	19 )	$0.889$ (	13 )	$0.880$ (	10 )	$0.887$ (	16 )	$0.845$ (	8 )	$0.855$ (	17 )
content	$0.871$ (	13 )	0.874 (	17 )	$0.872$ (	17 )	$0.863$ (	9 )	$0.867$ (	13 )	$0.818$ (	8 )	$0.832$ (	17 )
dof	$0.883$ (	13 )	0.887 (	19 )	$0.886$ (	19 )	$0.874$ (	9 )	$0.879$ (	15 )	$0.838$ (	8 )	$0.847$ (	17 )
light	$0.869$ (	13 )	0.876 (	18 )	$0.875$ (	18 )	$0.861$ (	10 )	$0.866$ (	14 )	$0.828$ (	8 )	$0.839$ (	17 )

Appendix

Table 15: Highest Spearman correlation achieved by probing on

\mathbf{LT}

layers for PARA. Layer indices are shown in parentheses.

Figure 13: Layer-wise probing performance across

\mathbf{V}

and

\mathbf{LT}

layers for Qwen3-VL 2B on PARA.

Attribute	Base		Numeric		Attribute		Unrelated
BalancingElements	0.325 (	13 )	$0.323$ (	10 )	$0.323$ (	11 )	$0.319$ (	0 )
ColorHarmony	$0.516$ (	9 )	$0.510$ (	12 )	$0.518$ (	16 )	0.531 (	11 )
Content	$0.633$ (	10 )	0.634 (	12 )	$0.629$ (	10 )	$0.626$ (	9 )
DoF	$0.535$ (	10 )	0.541 (	11 )	$0.531$ (	21 )	$0.535$ (	9 )
Light	0.509 (	14 )	$0.505$ (	19 )	$0.508$ (	17 )	$0.491$ (	13 )
MotionBlur	$0.165$ (	12 )	$0.142$ (	11 )	0.176 (	12 )	$0.167$ (	10 )

Appendix

Table 16: Probing performance on Qwen3-VL 2B under different prompt formulations.

Model	Method	$\rho$ / $R^{2}$
Qwen3-VL 2B	Raw Text (Original)	0.504 / -0.571
	Raw Text (Resized)	0.505 / -0.511
	LoRA (Resized, 100-shot)	0.487 / -1.970
Qwen3-VL 4B	Raw Text (Original)	0.570 / -1.277
	Raw Text (Resized)	0.568 / -1.117
	LoRA (Resized, 100-shot)	0.578 / -1.751

Appendix

Table 17: Effect of image resizing on PIAA performance for Qwen3-VL models on PARA.

Panel of Figure 14 — Figure 14: User-averaged Spearman correlation obtained by Linear-Hidden PIAA using combined representations from the vision encoder ( $\mathbf{V}_{i}$ ) and language decoder ( $\mathbf{LT}_{j}$ ).

Train / Test Split	Qwen3-VL 2B	Gemma 3 4B
Train=FIGURATIVE, Test=ABSTRACT	0.183 / -0.596	0.167 / -0.697
Train=ABSTRACT, Test=FIGURATIVE	0.145 / -1.072	0.134 / -1.062

Appendix

Table 18: User-averaged Spearman correlation and

R^{2}

of Linear-Hidden on LAPIS under cross-style evaluation.

Personalized Image Aesthetics Assessment

Personalized image aesthetics assessment predicts how appealing images are to particular people rather than to an average viewer.

See all 14 papers

Personalized Image Aesthetic Assessment via Preference-rich Sample Mining and Cohort Merging
Jul 17, 2026Zhichao Yang, Tianjiao Gu, Zhixianhe Zhang +3Personalized Image Aesthetics Assessment Image Quality Assessment
Xidian University Xi’an, China · Xidian University Xian, China
Enhancing Zero-shot Personalized Image Aesthetics Assessment with Profile-aware Multimodal LLM
Apr 19, 2026Chun Wang, Chenfeng Wei, Chenyang Liu +1Zero-Shot Learning Personalized Image Aesthetics Assessment
Mashang Consumer Finance Co., Ltd., Chongqing, China · Xi’an Jiaotong-Liverpool University, Suzhou, China · Beijing University of Posts and Telecommunications, Beijing, China
Evaluating Vision-Language Models for Image Quality Assessment using Psychophysical Data
Mar 25, 2026Imran Mehmood, Imad Ali Shah, Ming Ronnier Luo +1Image Quality Assessment VLM Evaluation
School of Engineering, University of Galway, Ireland · Lero the Research Ireland Centre for Software, Ireland · State Key Laboratory of Extreme Photonics and Instrumentation, Zhejiang University, Hangzhou, China.

Jul 17, 2026J/K move · Enter open · S save

Personalized Image Aesthetic Assessment via Preference-rich Sample Mining and Cohort Merging

Zhichao Yang, Tianjiao Gu, Zhixianhe Zhang, Xiangfei Sheng+2

Xidian University Xi’an, China · Xidian University Xian, China

Personalized Image Aesthetics Assessment Image Quality Assessment Model Merging User Preference Modeling

Open paper AskPDF arXiv

Personalized Image Aesthetic Assessment (PIAA) aims to predict aesthetic ratings of images that vary across individuals. The aesthetic preferences manifest to different extents across distinct visual stimuli and exhibit cohort-specific patterns. Motivated by the above fact, this paper presents a Multimodal Large Language Model (MLLM)-based approach, which models individual aesthetic preferences by Preference-Rich sample mining and Aesthetically-resonant Cohort merging (PRAC). Specifically, PRAC first identifies preference-rich samples by analyzing both Collective Controversy and Personalized Deviation of images, maximizing the utility of limited user data. Based upon the preference-rich samples, cross-user preference similarities are measured by comparing preference embeddings. Then, a cohort-based model merging strategy, is proposed by aggregating preference patterns from aesthetically-resonant users, which further enhances the personalization for the target individual. Extensive experiments and comparisons on four benchmark PIAA databases demonstrate the superiority of the proposed PRAC model over the state-of-the-arts. The code and model will be public at https://github.com/yzc-ippl/PRAC.

Abstract

Personalized Image Aesthetics Assessment

Explore similar work

Personalized Image Aesthetic Assessment via Preference-rich Sample Mining and Cohort Merging

Enhancing Zero-shot Personalized Image Aesthetics Assessment with Profile-aware Multimodal LLM

Evaluating Vision-Language Models for Image Quality Assessment using Psychophysical Data