Interpretable Face Contrast Modeling for Smartphone Portrait Rendering

Yujin Cho; Benoît Pochon; Jean-Michel Morel; Gabriele Facciolo

cs.CVOct 19, 2026

Interpretable Face Contrast Modeling for Smartphone Portrait Rendering

Authors: Yujin Cho, Benoît Pochon, Jean-Michel Morel, Gabriele Facciolo

Organizations: Université Paris-Saclay, CNRS, ENS Paris-Saclay, Centre Borelli, France · DXOMARK, France · Lingnan University, Hong Kong SAR, China

Abstract

The quality of smartphone portrait photography has become increasingly important to both manufacturers and users. Recent devices use complex computational pipelines to improve facial rendering under varied lighting. These pipelines vary widely across manufacturers, so the same portrait can look noticeably different across smartphones. These differences affect multiple aspects of facial appearance, making perceptual preference difficult to characterize with a single measure. To address this, we propose an interpretable framework that decomposes face contrast into three components: overall facial tonal separation, eye-region contrast, and micro-contrast of fine skin details. We formulate dedicated metrics for each component and validate them via pairwise human judgments, demonstrating that our approach correlates more closely with human contrast perception than traditional contrast measures. To support this analysis, we introduce a realistic mannequin-based dataset designed for systematic facial contrast evaluation, comprising 508 full-resolution captures from 29 smartphones under diverse lighting, with region-specific masks and extensive pairwise perceptual annotations. As a downstream analysis, we show that combining the three component-wise metrics provides a meaningful basis for modeling balanced facial contrast preference in a controlled setting, and we present a preliminary evaluation on a smaller real-portrait dataset indicating that insights derived from the mannequin study can extend to real images. Our work provides a controlled first step toward defining and evaluating face contrast in portrait imaging.

Figures & tables

Figure 1: Examples from the proposed dataset for the two mannequin subjects, Eugene and Diana, shown under four illumination levels and multiple backlight conditions ( $\Delta\mathrm{EV}=0,4,7,$ and $9$ ). Each sub-image shows the full controlled capture setup together with an inset of the face crop, highlighting how facial contrast rendering varies across devices and lighting conditions.

Figure 2: Examples of face without background captured at two illumination levels (5 and 10000 lux) with different smartphones. Facial rendering varies with lighting and device-specific processing, causing noticeable differences in face contrast.

Group	Light Uniformity	$\Delta\mathrm{EV}$ / Illuminance (lux)
Group1	Flat	$\Delta\mathrm{EV}$ = 0, 5–100 lux
Group2	Flat	$\Delta\mathrm{EV}$ = 0, 1000–10000 lux
Group3	Slightly non-uniform	$\Delta\mathrm{EV}$ = 4, 100–1000 lux
Group4	Strongly non-uniform	$\Delta\mathrm{EV}$ = 7, 9; 5–1000 lux
Group5	Mixed	Groups 1–4

Table 1: Lighting conditions used to construct the five comparison groups.

\Delta\mathrm{EV}

denotes the calibrated exposure difference between the illumination on the mannequin and the light produced by the LED panel.

Figure 3: Illustration of the face mask (

\Omega_{f}

) , eye mask (

\Omega_{e}

), and skin mask (

\Omega_{s}

) used throughout our work.

Contrast	Metric	Within-lighting		Mixed-lighting
Contrast	Metric	PLCC	SRCC	PLCC	SRCC
Overall	Proposed overall contrast	0.792	0.780	0.688	0.693
	Global log-luminance std.	0.723	0.708	0.604	0.566
	Global luminance std.	0.675	0.646	0.551	0.562
	Michelson contrast (p2–p98) [ 37 ]	0.731	0.710	0.583	0.542
	Facial feature contrast [ 59 ]	-0.026	-0.005	-0.302	-0.263

Table 2: Comparison of proposed and reference measures for overall, eye, and micro-contrast under within-lighting device-disjoint evaluation on Groups 1–4 and mixed-lighting evaluation on Group 5. Results are based on 500 bootstrap resamples.

Type	Method	Expl.	Total	Train.	Before Adaptation				After Adaptation
					Within-lighting		Mixed-lighting		Within-lighting		Mixed lighting
					PLCC	SRCC	PLCC	SRCC	PLCC	SRCC	PLCC	SRCC
Hand-crafted	BRISQUE [ 38 ]	✓	3 (raw)	3 (raw)	0.115	0.063	0.223	0.190	0.169	0.134	0.235	0.316
	NIQE [ 39 ]	✓	3 (raw)	3 (raw)	0.411	0.430	0.533	0.603	0.397	0.405	0.461	0.577
	Ours (Overall)	✓	3 (raw)	3 (raw)	0.567	0.602	0.579	0.586	0.689	0.637	0.647	0.589
	Ours (Eye)	✓	3 (raw)	3 (raw)	0.456	0.447	0.623	0.596	0.647	0.570	0.645	0.634

Table 3: Balanced contrast preference prediction before and after task adaptation. We compare the proposed component-based method with hand-crafted, general IQA, portrait IQA, and face-quality baselines. “3 (raw)” denotes the fitted coefficients of

ax^{2}+bx+c

for a scalar score. Our combined model uses a second-order polynomial over three contrast components with 9 trainable parameters. Entries are omitted when a before- or after-adaptation counterpart is not applicable. All results are computed from 500 bootstrap resamples.

Target	Ours		IFQA (fine-tuned)
Target	PLCC	SRCC	PLCC	SRCC
Overall contrast	0.792	0.825	0.858	0.862
Eye contrast	0.753	0.768	0.633	0.629
Micro-contrast	0.717	0.731	0.693	0.678

Table 4: Real-portrait generalization results on 444 images across 10 scenes. PLCC and SRCC are computed separately within each scene and then macro-averaged across scenes. Ours is applied in a zero-shot manner, while IFQA is fine-tuned on the mannequin-based training annotations for each corresponding task.

Figure 4: Examples of real portraits used in our preliminary real-world analysis. Images span multiple subjects, devices, and edited contrast variants.

Appendix figures & tables47 assets

Supplementary material from the paper’s appendix.

Appendix

Figure S1 : Examples from the full-resolution dataset. The two mannequins are Eugene and Diana, and the images were captured using a Samsung Galaxy Z Fold6. The dataset includes multiple $\Delta\mathrm{EV}$ conditions (0, 4, 7, and 9), but here we show only $\Delta\mathrm{EV}=0$ and $\Delta\mathrm{EV}=4$ under backlit conditions.

Panel of Figure 10 — Figure S2 : Spectral reflectance comparison between real human skin and the corresponding realistic mannequin surface for (a) Eugene and (b) Diana. The mannequins follow similar wavelength-dependent reflectance trends to the real subjects.

Panel of Figure 11 — Figure S3 : Example scenes used to illustrate the annotation protocol for the two mannequin subjects. (a) Eugene under 100 lux, LED 4000K lighting. (b) Diana under 100 lux, LED 4000K lighting with $\Delta\mathrm{EV}=4$ . The examples highlight the facial regions used to guide the overall-contrast, eye-contrast, and micro-contrast annotation tasks.

Panel of Figure 12 — Figure S4 : Examples of the same scene captured under different illumination levels and exposure settings.

Figure S5 : Tonal contrast (Overall contrast) example

Figure S6 : Dark and bright regions to inspect for Eugene and Diana, with a washed-out example for overall contrast.

Figure S7 : Example of micro-contrast: the image on the right exhibits higher micro-contrast than the image on the left.

Figure S8 : Regions to inspect for eye contrast: pupil darkness, catch-light brightness, and sclera brightness.

Figure S9 : Regions to inspect for micro-contrast in Eugene and Diana.

Figure S10 : Visualization of the multi-scale overall-contrast formulation for three example images. For each image, we show the original face mask, the local normalized contrast map computed at each scale $r\in\mathcal{R}$ , where $\mathcal{R}=\{16,32,64,128,256\}$ , and the resulting merged contrast map. Smaller scales emphasize fine local variations, while larger scales capture broader facial shading and illumination structure.

Leave-one-scale-out analysis
Scale setting	Within-lighting condition		Mixed-lighting condition
Scale setting	PLCC	SRCC	PLCC	SRCC
$\{16,32,64,128,256\}^{\dagger}$	$0.792\pm 0.070$	$0.780\pm 0.081$	$0.688\pm 0.133$	$0.693\pm 0.097$
Without 16	$0.778\pm 0.077$	$0.755\pm 0.092$	$0.662\pm 0.147$	$0.671\pm 0.107$
Without 32	$0.781\pm 0.076$	$0.765\pm 0.090$	$0.669\pm 0.145$	$0.678\pm 0.104$
Without 64	$0.788\pm 0.074$	$0.773\pm 0.088$	$0.681\pm 0.140$	$0.687\pm 0.102$

Appendix

Table S1 : Scale-set ablation of the proposed overall-contrast measure. Scale values denote square-patch side lengths in pixels. Results are reported as the mean

\pm

standard deviation over 500 bootstrap resamples. The symbol

\dagger

indicates the reference configuration used in the main experiments, while bold values indicate the highest nominal correlation within each analysis block.

Figure S11 : Face images and corresponding eye-region crops ordered by the Michelson eye-contrast measure [ 37 ] . While this formulation provides a reasonable ranking in some cases, it can fail in low-contrast regimes. In particular, the perceptual ordering between Images 212022 and 212030 is not preserved. For visualization, sRGB images are shown, but the metric is computed on the $L^{*}$ channel only.

Figure S12 : Low-contrast eye-region examples ordered by the proposed Otsu-based eye-contrast measure [ 47 ] . Although these images exhibit weaker dark–bright separation than high-contrast cases, the proposed formulation still yields a perceptually meaningful ranking.

Variant	PLCC	SRCC
Proposed micro-contrast	$0.849\pm 0.024$	$0.842\pm 0.028$
w/o robust spatial pooling	$0.825\pm 0.026$	$0.814\pm 0.029$
w/o perceptual response compression	$0.802\pm 0.029$	$0.815\pm 0.029$
w/o local energy pooling	$0.664\pm 0.069$	$0.668\pm 0.077$

Appendix

Table S2 : Ablation study of the proposed micro-contrast formulation under Within-lighting condition . Results are reported as the mean

\pm

standard deviation over 500 bootstrap resamples.

Variant	PLCC	SRCC
Proposed micro-contrast	$0.797\pm 0.011$	$0.787\pm 0.019$
w/o robust spatial pooling	$0.775\pm 0.011$	$0.763\pm 0.000$
w/o perceptual response compression	$0.736\pm 0.019$	$0.763\pm 0.001$
w/o local energy pooling	$0.631\pm 0.102$	$0.629\pm 0.077$

Appendix

Table S3 : Ablation study of the proposed micro-contrast formulation under Mixed-lighting condition . Results are reported as the mean

\pm

standard deviation over 500 bootstrap resamples.

Figure S13 : Parameter sensitivity of the proposed micro-contrast measure under the Within-lighting condition (top) and mixed-lighting (bottom) evaluations. We vary the fine scale

\sigma_{f}

, scale ratio

\rho=\sigma_{c}/\sigma_{f}

, local integration window

W

, perceptual exponent

\gamma

, and robust pooling interval. Solid and dashed lines denote PLCC and SRCC, respectively, and error bars represent the standard deviation over 500 bootstrap resamples. The gray bands and

\dagger

symbols indicate the reference configuration used in the main experiments:

\sigma_{f}=1/\sqrt{2}\approx 0.7

\rho=3

W=11

\gamma=0.6

, and

[q_{10},q_{90}]

pooling.

Contrast	Diana		Eugene
Contrast	PLCC	SRCC	PLCC	SRCC
Overall contrast	$0.880\pm 0.017$	$0.875\pm 0.026$	$0.705\pm 0.129$	$0.691\pm 0.145$
Eye contrast	$0.934\pm 0.015$	$0.919\pm 0.007$	$0.824\pm 0.064$	$0.784\pm 0.101$
Micro-contrast	$0.863\pm 0.040$	$0.862\pm 0.043$	$0.835\pm 0.026$	$0.822\pm 0.033$

Appendix

Table S4 : Per-mannequin performance of the proposed contrast metrics under the within-lighting evaluation. Correlations are macro-averaged over Groups 1–4 for each mannequin. Uncertainty denotes the standard deviation over 500 bootstrap resamples of the four group-level correlations.

Within-lighting evaluation
Method	PLCC	SRCC	RMSE
Non-negative linear regressor	$0.635\pm 0.085$	$0.654\pm 0.073$	$1.111\pm 0.090$
Second-order polynomial regressor	$\mathbf{0.728\pm 0.055}$	$\mathbf{0.739\pm 0.054}$	$\mathbf{1.063\pm 0.115}$
Pairwise-trained MLP	$0.708\pm 0.055$	$0.734\pm 0.044$	$1.118\pm 0.114$
Mixed-lighting evaluation
Non-negative linear regressor	$0.715\pm 0.044$	$0.682\pm 0.016$	$1.094\pm 0.029$

Appendix

Table S5 : Comparison of balanced contrast preference predictors under the within-lighting and mixed-lighting evaluations.

Training target	PLCC	SRCC
Raw balanced contrast JOD	$0.773\pm 0.054$	$0.778\pm 0.046$
Group–mannequin z-normalized JOD	$0.774\pm 0.050$	$0.775\pm 0.038$

Appendix

Table S6 : Sensitivity of the second-order polynomial balanced contrast preference predictor to normalization of the Groups 1–4 training targets.

Evaluation	Fitting data	Subject	Model	$\alpha_{O}$	$\alpha_{E}$	$\alpha_{M}$	$\alpha_{O^{2}}$	$\alpha_{E^{2}}$	$\alpha_{M^{2}}$	$\alpha_{OE}$	$\alpha_{OM}$	$\alpha_{EM}$
Within-lighting	G1	Diana	Linear	0.3737	0.3475	0.1114	–	–	–	–	–	–
		Diana	Polynomial	0.6424	0.3075	0.2604	-0.4039	0.0006	-0.3136	-0.0219	0.1000	0.2704
		Eugene	Linear	0.1839	0.0000	0.2491	–	–	–	–	–	–
		Eugene	Polynomial	0.4860	-0.2704	0.4312	-0.4301	-0.0327	-0.4807	-0.4350	0.7321	0.5574
	G2	Diana	Linear	0.0000	1.3586	0.3842	–	–	–	–	–	–
		Diana	Polynomial	-0.3201	1.3867	0.8643	-2.0840	-4.3643	-1.9313	5.7795	1.2028	1.0930

Appendix

Table S7 : Coefficients of the linear and second-order polynomial preference regressors in standardized feature space. For within-lighting evaluation, models are fitted separately within Groups 1–4. For mixed-lighting evaluation, models are fitted on Groups 1–4 and applied without refitting to the held-out Group 5 set.

Figure S14 : Real-portrait ablation study on overall contrast. A total of 444 images from 29 unique capture devices/sources are used. Each scene corresponds to a different subject and contains approximately 44 images captured with an average of five devices, together with contrast-adjusted variants generated using DxO PhotoLab 9. Human judgments were collected through pairwise comparisons for overall contrast, eye contrast, and micro-contrast. The proposed method shows strong agreement with the subjective annotations across diverse real-portrait scenes. From top to bottom, the rows correspond to: scene_8005_103 and scene_1606_031; scene_0105_089 and scene_0109_2; scene_1119_105 and scene_1113_510; scene_3115_305 and Riverside; and scene_0109_3 and scene_5203_106.

Scene	Overall				Eye				Micro
	Ours		IFQA (fine-tuned)		Ours		IFQA (fine-tuned)		Ours		IFQA (fine-tuned)
	PLCC	SRCC	PLCC	SRCC	PLCC	SRCC	PLCC	SRCC	PLCC	SRCC	PLCC	SRCC
8005_103	0.914 $\pm$ 0.031	0.898 $\pm$ 0.046	0.889 $\pm$ 0.030	0.894 $\pm$ 0.049	0.487 $\pm$ 0.100	0.549 $\pm$ 0.125	-0.228 $\pm$ 0.164	-0.139 $\pm$ 0.170	0.512 $\pm$ 0.105	0.532 $\pm$ 0.145	0.472 $\pm$ 0.098	0.530 $\pm$ 0.133
1606_031	0.632 $\pm$ 0.079	0.671 $\pm$ 0.084	0.876 $\pm$ 0.033	0.868 $\pm$ 0.046	0.793 $\pm$ 0.039	0.851 $\pm$ 0.045	0.791 $\pm$ 0.045	0.837 $\pm$ 0.048	0.631 $\pm$ 0.095	0.736 $\pm$ 0.072	0.844 $\pm$ 0.043	0.749 $\pm$ 0.082
0105_089	0.835 $\pm$ 0.038	0.851 $\pm$ 0.043	0.835 $\pm$ 0.038	0.853 $\pm$ 0.043	0.832 $\pm$ 0.032	0.838 $\pm$ 0.038	0.748 $\pm$ 0.069	0.725 $\pm$ 0.076	0.868 $\pm$ 0.030	0.870 $\pm$ 0.033	0.861 $\pm$ 0.031	0.857 $\pm$ 0.043
0109_2	0.689 $\pm$ 0.049	0.717 $\pm$ 0.066	0.733 $\pm$ 0.071	0.713 $\pm$ 0.103	0.893 $\pm$ 0.020	0.923 $\pm$ 0.026	0.919 $\pm$ 0.017	0.943 $\pm$ 0.023	0.704 $\pm$ 0.070	0.670 $\pm$ 0.089	0.807 $\pm$ 0.053	0.844 $\pm$ 0.059

Appendix

Table S9 : Scene-wise PLCC and SRCC on the real-portrait dataset for overall contrast, eye contrast, and micro-contrast. The proposed metrics are applied zero-shot, while IFQA is evaluated after fine-tuning on the corresponding mannequin-based annotations. Values are reported as the point estimate

\pm

standard deviation over 500 image-level bootstrap resamples.

Target	IFQA (fine-tuned)		IFQA (non-fine-tuned)
Target	PLCC	SRCC	PLCC	SRCC
Overall contrast	0.858	0.862	-0.075	-0.086
Eye contrast	0.633	0.629	-0.117	-0.106
Micro-contrast	0.693	0.678	-0.113	-0.105

Appendix

Table S10 : Macro-average PLCC and SRCC across real-portrait scenes for IFQA before and after fine-tuning on the mannequin-based annotations.

	Diana			Eugene
Group	Original	Edited	Total	Original	Edited	Total
Group 1	38	37	75	42	33	75
Group 2	47	28	75	49	26	75
Group 3	45	30	75	41	34	75
Group 4	59	16	75	62	13	75
Group 5	61	39	100	64	36	100

Appendix

Table S11 : Image composition by group and mannequin (original vs. edited).

Brand	# Devices	Models
Samsung	8	A55 5G, A56 5G, S24 Ultra, S25 Ultra, Z Flip5, Z Flip6, Z Fold5, Z Fold6
Google	7	Pixel 8, 8 Pro, 8a, 9, 9 Pro Fold, 9 Pro XL, 9a
Xiaomi	6	13T Pro, 14, 14 Ultra, 15 Ultra, Redmi Note 13 Pro+, Redmi Note 14 Pro+
Huawei	2	Mate 60 Pro+, Pura 70 Ultra
Vivo	2	X100 Pro, X100 Ultra
Honor	2	Magic 6 Pro, Magic 7 Pro

Appendix

Table S12: Device brands and models included in the mannequin dataset.

Training devices (19)	AsusZenfone11Ultra, GooglePixel8, GooglePixel9, GooglePixel9ProFold, GooglePixel9ProXL, GooglePixel9a, HuaweiPura70Ultra, SamsungGalaxyA555G, SamsungGalaxyA565G, SamsungGalaxyS25Ultra, SamsungGalaxyZFlip6, SamsungGalaxyZFold5, SamsungGalaxyZFold6, VivoX100Pro, VivoX100Ultra, Xiaomi14Ultra, Xiaomi15Ultra, XiaomiRedmiNote13ProPlus5G, XiaomiRedmiNote14ProPlus5G
Validation devices (10)	GooglePixel8Pro, GooglePixel8a, HonorMagic6Pro, HonorMagic7Pro, HuaweiMate60ProPlus, OppoFindX7Ultra, SamsungGalaxyS24Ultra, SamsungGalaxyZFlip5, Xiaomi13TPro, Xiaomi14

Appendix

Table S13: Device sets used for Within-lighting condition on Groups 1–4. The split is strictly device-disjoint.

Group	Person	Train Dev	Val Dev
Group1	Diana	18	10
Group1	Eugene	18	10
Group2	Diana	16	10
Group2	Eugene	17	10
Group3	Diana	17	10
Group3	Eugene	17	10

Appendix

Table S14: Device split summary by group and mannequin under Within-lighting condition .

Person	Total	Train Total	Val Total	Train Ratio	Val Ratio
Diana	300	195	105	0.6500	0.3500
Eugene	300	200	100	0.6667	0.3333

Appendix

Table S15: Split summary aggregated by mannequin under Within-lighting condition .

Figure S15 : Annotation stability of the JOD scales under within-lighting and mixed-lighting evaluation. Each observer is excluded in turn, and the JOD scale reconstructed from the remaining observers is compared with the reference scale estimated from all annotations. Bars show the mean PLCC and SRCC across the four observer exclusions, and error bars indicate the corresponding standard deviation.

Figure S16 : Device-level pooled rankings with 95% confidence intervals for four JOD targets. Overall, Eye, and Micro represent amount of contrast dimensions, while Balanced contrast reflects subjective utility. Scores are macro-aggregated subgroup-normalized JOD values expressed in standard-deviation units.

Panel of Figure 39 — Figure S17 : Device-level summary under z-score-based analysis. Panel (a) provides a compact cross-metric profile per device, and panel (b) highlights relative ordering by contrast amount.

Figure S18 : Example of the annotation tool used to acquire pairwise comparisons (Overall contrast, Eugene)

Figure S19 : Example of the annotation tool used to acquire pairwise comparisons (Overall contrast, Diana)

Figure S20 : Example of the annotation tool used to acquire pairwise comparisons (Eye contrast, Eugene)

Figure S21 : Example of the annotation tool used to acquire pairwise comparisons (Eye contrast, Diana)

Figure S22 : Example of the annotation tool used to acquire pairwise comparisons (Micro-contrast, Eugene)

Figure S23 : Example of the annotation tool used to acquire pairwise comparisons (Micro-contrast, Diana)

Figure S24 : Example of the annotation tool used to acquire pairwise comparisons (Balanced contrast, Eugene)

Figure S25 : Example of the annotation tool used to acquire pairwise comparisons (Balanced contrast, Diana)

Figure S26 : Examples of Eugene images sorted by JOD for overall contrast, from lower to higher values.

Figure S27 : Examples of Diana images sorted by JOD for overall contrast, from lower to higher values.

Figure S28 : Examples of Eugene images sorted by JOD for eye contrast, from lower to higher values.

Figure S29 : Examples of Diana images sorted by JOD for eye contrast, from lower to higher values.

Figure S30 : Examples of Eugene images sorted by JOD for micro-contrast, from lower to higher values.

Figure S31 : Examples of Diana images sorted by JOD for micro-contrast, from lower to higher values.

Figure S32 : Examples of Eugene images sorted by JOD for balanced contrast preference, from lower to higher values.

Figure S33 : Examples of Diana images sorted by JOD for balanced contrast preference, from lower to higher values.

Image Quality Assessment

Image Quality Assessment measures perceived image quality, distortion, and usefulness with computational metrics and human-aligned evaluation.

See all 174 papers

AlignFace: Human-Aligned Face Similarity Metric with Interpretable Concept Relations
Aug 14, 2026Ying Huang, Wencan Zhang, Brian Y. Lim
Department of Computer Science, National University of Singapore Singapore, Singapore
Human face perception reflects inverse-generative and naturalistic discriminative objectives
May 12, 2026Wenxuan Guo, Heiko H. Schütt, Kamila Maria Jozwik +3Face Recognition Human Visual Perception
Department of Psychology, Columbia University, New York, NY, USA. · Department of Behavioural and Cognitive Sciences, Université du Luxembourg, Esch-sur-Alzette, Luxembourg. · MRC Cognition and Brain Sciences Unit, University of Cambridge, Cambridge, England. +4
PortraitAes: Intent-Conditioned Structured Portrait Aesthetic Assessment
Oct 4, 2026Junzhou Xie, Haozhong Xiong, Xunyun Tian +8Image Quality Assessment
Southeast University · Qwen Business Unit of Alibaba

Aug 14, 2026J/K move · Enter open · S save

AlignFace: Human-Aligned Face Similarity Metric with Interpretable Concept Relations

Ying Huang, Wencan Zhang, Brian Y. Lim

Department of Computer Science, National University of Singapore Singapore, Singapore

Open paper AskPDF arXiv

Computer vision models for generated facial content, such as face editing and privacy protection, increasingly affect people, requiring similarity metrics that serve as faithful proxies for human perception. While perceptual evaluation has progressed from signal-based heuristics to representation-based metrics, current approaches are limited to behavioral modeling without cognitive alignment. They rely on implicit and spurious relations while assuming a universal observer, failing to account for inherent variations across diverse human populations. This leads to inaccurate evaluative models of stakeholders and misleading guidance for generative model debugging. Rather than treating perception as a black box, we leverage scientific findings from cognitive psychology of human face similarity perception: dependence on facial featural and configural attributes, nonlinear psychophysical response scaling, and own-group biases. We introduce the FACETS dataset and propose AlignFace, an interpretable, human-aligned, face similarity metric that encodes these cognitive principles through ante-hoc modeling. It employs visual-language modeling (VLM) to encode paired face images and text-based attributes, gated cross-attention (CA) to extract attribute-specific facial difference representations, concept bottleneck modeling (CBM) to constrain reasoning via interpretable face attributes, and neural generalized additive model (GAM) to model their nonlinear influence. Experiments found AlignFace significantly improves alignment with human subpopulation perceptions compared to baseline metrics, including recent domain-free learned perceptual metrics. By bridging learned representations and human cognitive processes, this work enables more transparent and aligned perceptual evaluation metrics for face images.

Abstract

Image Quality Assessment

Explore similar work

AlignFace: Human-Aligned Face Similarity Metric with Interpretable Concept Relations

Human face perception reflects inverse-generative and naturalistic discriminative objectives

PortraitAes: Intent-Conditioned Structured Portrait Aesthetic Assessment