OpenVTON-Bench: A Large-Scale High-Resolution Benchmark for Controllable Virtual Try-On Evaluation

Jin Li; Tao Chen; Kai Wen; Siqi Yin; Shuai Jiang; Weijie Wang; Jingwen Luo; Chenhui Wu

cs.CVJan 30, 2026

OpenVTON-Bench: A Large-Scale High-Resolution Benchmark for Controllable Virtual Try-On Evaluation

Authors: Jin Li, Tao Chen, Kai Wen, Siqi Yin, Shuai Jiang, Weijie Wang, Jingwen Luo, Chenhui Wu

Organizations: Renxing Intelligence, Hangzhou, China · Hangzhou Dianzi University, Hangzhou, China

Abstract

Recent advances in diffusion models have significantly elevated the visual fidelity of Virtual Try-On (VTON) systems, yet reliable evaluation remains a persistent bottleneck. Traditional metrics struggle to quantify fine-grained texture details and semantic consistency, while existing datasets fail to meet commercial standards in scale and diversity. We present OpenVTON-Bench, a large-scale benchmark comprising approximately 100K high-resolution image pairs (up to $1536\times1536$ ). The dataset combines Gemini-powered dense captioning with stratified sampling across 20 visual clusters obtained by DINOv3 K-Means. To support reliable evaluation, we propose a multi-modal protocol that measures VTON quality along five interpretable dimensions: background consistency, identity fidelity, texture fidelity, shape plausibility, and overall realism. The protocol integrates VLM-based semantic reasoning with a novel Multi-Scale Representation Metric based on SAM3 segmentation and morphological erosion, enabling the separation of boundary alignment errors from internal texture artifacts. Under the specified evaluation rubric, our representation metric shows strong agreement with human model rankings (Kendall's $τ$ of 0.833 vs. 0.611 for SSIM). Code and dataset are available at https://github.com/RenxingIntelligence/OpenVTON-Bench.

Figures & tables

Figure 1: The Proposed Hybrid Evaluation Framework. We move beyond single-scalar metrics by decomposing VTON quality into five human-aligned dimensions. Uniquely, our framework combines (Top) a VLM-as-a-Judge module for semantic auditing with (Bottom) a Multi-Scale Representation Metric that verifies structure-aware garment consistency. This synergy ensures both semantic plausibility and accurate garment replication.

Figure 2: Data Construction Pipeline of OpenVTON-Bench. The process consists of three stages: (1) Large-scale raw data aggregation from diverse sources; (2) Hybrid annotation combining human verification for pair alignment and VLM-based dense captioning; (3) Stratified sampling across 20 DINOv3 K-Means visual clusters, subject to source availability.

Figure 3: Representative examples from OpenVTON-Bench.

Figure 4: Qualitative comparison of state-of-the-art methods on OpenVTON-Bench.

Method	$\mathcal{S}_{\text{global}}$	$\mathcal{S}_{\text{rep}}^{(0)}$	$\mathcal{S}_{\text{rep}}^{(1)}$	$\mathcal{S}_{\text{rep}}^{(2)}$	$\mathcal{S}_{\text{rep}}^{(3)}$	$\bar{\mathcal{S}}_{rep}$	$\bar{\mathcal{S}}$
OOTD [ 54 ]	0.844	0.797	0.755	0.701	0.669	0.731	0.788
EasyControl [ 60 ]	0.854	0.830	0.807	0.765	0.729	0.783	0.818
UNO [ 51 ]	0.737	0.763	0.730	0.674	0.628	0.699	0.718
FLUX.1-Kontext-dev [ 27 ]	0.849	0.813	0.778	0.731	0.694	0.754	0.802
FLUX.2-dev [ 26 ]	0.928	0.886	0.863	0.823	0.793	0.841	0.885
NanobananaPro [ 14 ]	0.936	0.894	0.865	0.827	0.807	0.848	0.892

Table 2: Representation-based Similarity Evaluation.

\mathcal{S}_{\text{global}}

measures holistic semantic consistency.

\mathcal{S}_{\text{rep}}^{(k)}

denotes garment-level similarity under increasing mask erosion levels (

0\rightarrow 3

), isolating texture quality from boundary artifacts.

Figure 5: Correlation Analysis. Scatter plot comparing objective metrics with human ratings. Both our Representation (Red) and VLM (Blue) metrics show strong positive correlations.

Table 7

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Year	Quantity	Resolution	Open-Source
VITON [ 16 ]	2018	16,253	$256\times 192$	$\checkmark$
VITON-HD [ 5 ]	2021	13,679	$1024\times 768$	$\checkmark$
DressCode [ 36 ]	2022	53,792	$1024\times 768$	$\checkmark$
SHHQ [ 12 ]	2022	231,176	$1024\times 768$	$\checkmark$
StreetTryOn [ 8 ]	2024	14,453	$512\times 320$	$\checkmark$
LH-400K [ 30 ]	2024	409,270	$512\times 512$	$\checkmark$

Appendix

Table 5: Comparison of Virtual Try-On Datasets. We compare OpenVTON-Bench with representative datasets in terms of scale, resolution, and accessibility. Unlike prior benchmarks that are either low-resolution or closed-source, our benchmark offers the largest open-source collection of high-fidelity images with

\min(H,W)\geq 1024

. Open-Source :

\checkmark

denotes publicly available,

\times

denotes closed.

Panel of Figure 6 — Figure 6: Dataset Analysis of OpenVTON-Bench. Left (a-d): t-SNE visualizations of the full dataset and the train/validation/test splits. Right (e): Counts across the 20 visual sampling clusters, distinct from the 14 caption-derived metadata labels.

Figure 7: Diversity of OpenVTON-Bench. Example triplets (Garment, Model, Mask) illustrate garment variation across the 20 visual sampling clusters C0–C19. A single example does not exhaust the variation within its cluster.

Table 11

Dimension	Bias (VLM $-$ human)	MAE
Background Consistency	$-0.033$	0.248
Person Identity & Body Consistency	$+0.039$	0.448
Texture Fidelity	$+0.189$	0.299
Shape Preservation	$+0.163$	0.305
Overall Realism	$-0.066$	0.196

Appendix

Table 8: VLM–human score disagreement. Bias and MAE are computed across nine model means per dimension. Positive bias indicates higher VLM scores.

Method	Garment $\Delta E_{00}\downarrow$		Non-edited $\Delta E_{00}\downarrow$
Method	Region-mean	Pixel-mean	Region-mean
OOTD	12.67	16.85	1.57
UNO	12.84	20.08	10.50
EasyControl	12.70	17.31	2.44
FLUX.1-Kontext-dev	10.01	15.19	4.85
FLUX.2-dev	5.21	9.38	1.91

Appendix

Table 11: Color consistency on generated test outputs. CIEDE2000 color differences are evaluated under a common protocol. Best values in each column are bold.

Figure 8: Extended qualitative comparison. We compare the generation results of different baseline models on challenging OpenVTON-Bench examples.

Asset	Role	License / Terms
IMAGDressing-v1 / IGPair [ 39 ]	Source of curated VTON pairs	Original dataset terms apply; academic / personal use only; commercial use prohibited
Public online images	Web-scale candidate pool	Benchmark release: CC BY-NC 4.0; original rights apply; takedown supported
DINOv3 [ 41 ]	Semantic clustering and representation metric	Meta DINOv3 License / Agreement
Gemini 2.0 Flash [ 44 ]	Dense garment captioning	Google API / model terms apply
Qwen-VL-Plus [ 1 ]	VLM-as-a-Judge evaluation	Qwen API / model terms apply
SAM3 [ 3 ]	Garment mask generation and segmentation	Meta SAM License

Appendix

Table 15: Main external assets used in OpenVTON-Bench.

Virtual Try-On

Virtual try-on generates images or videos showing people wearing selected garments while preserving appearance, fit, pose, and identity.

See all 55 papers

Beyond Global Realism: Virtual Try-On Evaluation and Optimization with Dimension-wise Garment Fidelity Assessment
Aug 30, 2026Kaidong Zhang, Yukang Ding, Xiaoyu Liu +1Virtual Try-On
Taobao & Tmall Group of Alibaba · Harbin Institute of Technology
Reference-Free Image Quality Assessment for Virtual Try-On via Human Feedback
Mar 13, 2026Yuki Hirakawa, Takashi Wada, Ryotaro Shimizu +6Virtual Try-On Human-in-the-Loop Evaluation
ZOZO Research · ZOZO Inc. · Keio University
OmniDiT: Extending Diffusion Transformer to Omni-VTON Framework
Mar 20, 2026Weixuan Zeng, Pengcheng Wei, Huaiqing Wang +8Virtual Try-On Diffusion Transformer
The Chinese University of Hong Kong, ShenZhen · Beihang University · KuaiShou

Aug 30, 2026J/K move · Enter open · S save

Beyond Global Realism: Virtual Try-On Evaluation and Optimization with Dimension-wise Garment Fidelity Assessment

Kaidong Zhang, Yukang Ding, Xiaoyu Liu, Ying Chen

Taobao & Tmall Group of Alibaba · Harbin Institute of Technology

Virtual Try-On

Open paper AskPDF arXiv

Virtual try-on (VTON) requires not only realistic generation but also faithful preservation of garment characteristics. However, existing evaluation metrics such as PSNR, SSIM, KID and FID struggle to measure the consistency between the generated and reference garments, particularly in capturing the multi-dimensional characteristics of garment fidelity. To address this, we propose DAT: a Dimension-wise Assessment framework for virtual Try-on, which decomposes garment consistency into seven interpretable dimensions: silhouette, color, neckline and sleeve shape, major decoration and structure, material texture, fine-detail fidelity, and logo preservation, each formulated as a discrete attribute-level prediction task. To train this specialized assessment model, we adopt a two-stage learning paradigm comprising large-scale weak supervision on 50K samples, followed by refinement on 10K higher-quality annotations obtained via multi-model voting. Furthermore, we employ weighted cross-entropy loss to mitigate the severe label imbalance inherent across evaluation dimensions. Beyond its role as an evaluation framework, the assessment model can be integrated into reinforcement learning optimization of Qwen-Image-Edit for VTON, where dimension-wise rewards are adaptively aggregated to emphasize under-optimized aspects during training. Experimental results show that our method (8B parameters) achieves state-of-the-art performance in terms of balanced accuracy, SROCC, and PLCC, outperforming strong proprietary models such as Gemini-3.1, Qwen3.7-plus, and GPT-5.5, while also serving as an effective optimization signal for reward-guided VTON generation

Abstract

Virtual Try-On

Explore similar work

Beyond Global Realism: Virtual Try-On Evaluation and Optimization with Dimension-wise Garment Fidelity Assessment

Reference-Free Image Quality Assessment for Virtual Try-On via Human Feedback

OmniDiT: Extending Diffusion Transformer to Omni-VTON Framework