Organizations: Sun Yat-sen University · WeChat Vision, Tencent Inc. · Xidian University · Fudan University · Tsinghua Shenzhen International Graduate School
Real-world image editing requires coordinating multiple visual transformations with content preservation and reference constraints, yet aggregate scores can readily obscure unmet local requirements. Effective evaluation must therefore identify local failures and translate them into priorities for subsequent model improvement. We introduce WeGenEditBench, comprising 2,740 evaluation instances and 148 task types across single-image, text-intensive, and multi-reference editing. Pre-generated checklists shared across models decompose each request into atomic requirements; item-level deductions and visual evidence are established first and then summarized into scores for edit execution, preservation, visual quality, and task-specific correctness. To support the experience-acquisition component of recursive self-improvement (RSI), we develop WeGenEvoStudio, a unified multi-agent harness for RSI-oriented learning that integrates model-conditioned failure diagnosis, retrieval-grounded data construction, supervised adaptation, and re-evaluation. Among 15 evaluated configurations, GPT-Image-2 leads single-image editing with an Overall score of 4.076, while Seedream-5.0-Pro leads text-intensive editing at 4.291. Evolution experiments across four generation and editing benchmarks demonstrate substantial targeted gains: on their report-selected Top-5 single-image targets, FLUX.2 [dev] improves Overall from 1.083 to 3.517, while Qwen-Image-Edit-2511 gains 13.2%. Together, our benchmark and harness provide a foundation for developing more reliable image models through interpretable evaluation and feedback-driven learning.
Figures & tables
Figure 1: From evaluation to evolution. Aggregate scores can obscure unmet editing requirements and leave training priorities unspecified (left). WeGenEditBench exposes these requirements through sample-specific checklists, scores, and rationales; WeGenEvoStudio turns the resulting evidence into targeted data construction, model updates, and re-evaluation (right). Only selected checklist items are shown.
Figure 2: Taxonomy and characteristics of WeGenEditBench. Left: settings, task families, examples, instance counts (inner), and subtypes (outer). Right: distributions of instruction length (a), tag multiplicity (b), input images (c), and checklist items (d). Colors indicate settings; lengths preserve source languages.
Figure 3: From scores to requirement-level diagnosis. Left: selected cases receive high scores despite visible errors. Right: WeGenEditBench identifies an omitted edit through checklist scores and rationales.
Figure 4: WeGenEvoStudio translates evaluation evidence into targeted training data. The Diagnosis Agent selects weaknesses and forms data prescriptions; the Construction Agent retrieves concepts or source images to synthesize generation and editing tasks. Supervised updates and re-evaluation close the loop. The exposure-editing example and score bars illustrate the workflow.
Model Name
EG ↑
PR ↑
VQ ↑
DS ↑
Overall ↑
Closed-source Models
GPT-Image-2
4.298
4.049
4.463
4.182
4.076
Nano-Banana-Pro
4.159
3.752
4.370
4.050
3.882
Qwen-Image-3.0
4.120
3.469
4.331
4.126
3.813
Seedream-5.0-Pro
4.207
4.084
4.434
4.206
4.029
Open-source Models
Table 1: Quantitative comparison on WeGenEditBench-SingleImageEdit. Bold and underlined scores denote the best and second best.
Model Name
Text-side
Image-side
Overall ↑
EG ↑
PR ↑
VQ ↑
DS ↑
EG ↑
PR ↑
VQ ↑
DS ↑
Closed-source Models
GPT-Image-2
4.289
4.031
4.329
4.399
4.604
3.919
4.421
4.452
4.132
Nano-Banana-Pro
4.295
4.092
4.310
4.322
4.396
4.208
4.232
4.306
4.116
Qwen-Image-3.0
4.367
3.732
4.329
4.385
4.509
3.869
4.406
4.414
4.109
Seedream-5.0-Pro
4.415
4.327
4.416
4.504
4.362
4.207
4.455
4.429
4.291
Table 2: Quantitative comparison on WeGenEditBench-TextEdit. Bold and underlined scores denote the best and second best.
Model Name
EG ↑
PR ↑
VQ ↑
Overall ↑
Closed-source Models
GPT-Image-2
4.358
4.134
3.707
4.138
Nano-Banana-Pro
4.031
3.993
3.340
3.877
Seedream-5.0-Pro
4.270
4.093
3.750
4.095
Open-source Models
HunyuanImage-3.0-Instruct-Distil
3.279
3.493
3.091
3.327
Table 3: Quantitative comparison on WeGenEditBench-MultiImageEdit. Bold and underlined scores denote the best and second best.
Model
T2I
Image editing
WeGen Bench
WeGenEditBench-SingleImageEdit
GEdit
ImgEdit
F ↑
EG ↑
PR ↑
VQ ↑
DS ↑
F ↑
SC ↑
PQ ↑
O ↑
I ↑
Q ↑
D ↑
F ↑
Qwen-Image
7.700
3.244
2.646
3.500
3.565
2.834
8.515
7.131
7.511
4.621
4.374
4.400
4.465
+ WeGenEvoStudio
8.668
3.705
2.650
4.105
3.437
3.208
8.172
7.241
7.476
4.621
4.416
4.450
4.496
Δ
+0.968
+0.461
+0.003
+0.605
-0.128
+0.373
-0.343
+0.109
-0.035
+0.000
+0.042
+0.051
+0.031
FLUX.2 [dev]
7.767
1.550
3.178
2.780
0.809
1.083
3.425
7.592
3.465
1.915
2.626
2.873
2.471
Table 4: Quantitative comparison on report-selected Top-5 categories across four benchmarks. FLUX.2 WeGenBench scores use 326 common-valid pairs. Target sets differ by backbone; green/red marks positive/negative Δ .
Figure 5: Qualitative editing comparisons of Qwen-Image-Edit-2511 and its WeGenEvoStudio adapters on GEdit, ImgEdit, and WeGenEditBench.
Appendix figures & tables67 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: Benchmark curation from visual assets to human-validated instructions and checklists.
Figure 7: Schematic of the Diagnosis Agent: category selection, visual review, failure grouping, and data prescriptions.
Figure 8: Construction Agent: task alignment, concept research or image retrieval, grounded instruction generation, and review. Accepted records retain source and instruction provenance; target supervision and training follow downstream.
Statistic
Single-image
Text-intensive
Multi-reference
Instances
1,008
1,000
732
Task families
8
5
6
Task types
65
35
48
Mean instruction length
79.2
124.6
146.3
Median instruction length
60.5
126
116.5
Instruction length range
12–255
10–344
30–439
Appendix
Table 5: Statistics of the three source files. Length is measured in tokens using o200k_base . Types are subset-qualified.
Benchmark
Taxonomy
Tokens
Task emphasis
Inputs
GEdit-Bench [ 22 ]
11 categories
2–38
General editing
1
ImgEdit [ 46 ]
9 Basic types
4–60
Single/multi-turn
1
REDEdit [ 33 ]
15 categories
—
General/text
1
RISEBench [ 56 ]
4 families / 16 types
—
Implicit reasoning
1
UnicBench [ 45 ]
4 families / 22 types
6–42
General/reasoning
1
UniREditBench [ 12 ]
8 dimensions / 18 subtypes
—
Real/game reasoning
1
Appendix
Table 6: Quantitative comparison on image-editing benchmarks: task coverage. Conventions are specified in Appendix A.4 .
Benchmark
Multi-dim.
Verification criteria
Retained judgment unit
GEdit-Bench
Yes
General rubric
SC/PQ scores; explanations in VIEScore
ImgEdit
Yes
Task-specific rubrics
Dimension scores + brief reason
REDEdit
Yes
Task/text criteria; OCR
Dimension scores; text-error metrics
RISEBench
Yes
Reasoning guidelines
Reasoning, consistency, plausibility
UnicBench
Yes
Instance reasoning points
Dimension scores + reason
UniREditBench
Yes
Text + GT image references
Reference-aided dimension scores
Appendix
Table 7: Comparison of evaluation protocols and diagnostic outputs. Conventions are specified in Appendix A.4 .
Subset
Overall
Count
Rate
95% CI
Single-image
≥4
315/644
48.9%
[45.1, 52.8]%
Text-intensive
≥4
171/666
25.7%
[22.4, 29.0]%
Multi-reference
≥4
119/503
23.7%
[19.9, 27.2]%
Single-image
≥4.5
101/323
31.3%
[26.5, 36.6]%
Appendix
Table 8: GPT-Image-2 outputs with at least one recorded item score ≤2 , conditional on the specified Overall threshold. Counts refer to model outputs; intervals are 95% cluster-bootstrap intervals.
Model
Both sides
Text only
Image only
Neither
GPT-Image-2
70 (25.0%)
29
77
104
Seedream-5.0-Pro
76 (27.1%)
49
66
89
Appendix
Table 9: Joint satisfaction of text-side and image-side requirements on 280 shared tasks. A side passes only when every applicable item scores at least 4. Counts within each row sum to 280.
Model
WeGenBench
GEdit
ImgEdit
F ↑
SC ↑
PQ ↑
O ↑
I ↑
Q ↑
D ↑
F ↑
Qwen-Image
8.170
8.774
7.676
7.962
4.704
4.438
4.490
4.544
+ WeGenEvoStudio
8.840
8.338
7.597
7.772
4.688
4.474
4.476
4.546
Δ
+0.670
-0.436
-0.079
-0.190
-0.016
+0.035
-0.014
+0.002
FLUX.2 [dev]
8.352
4.388
7.621
4.395
2.323
3.124
3.207
2.885
+ WeGenEvoStudio
8.475
7.785
8.132
7.404
4.170
4.099
4.190
4.153
Appendix
Table 10: Full-benchmark results for the adapters in Table 4 . FLUX.2 WeGenBench scores use 995 common-valid pairs.
Category
SC
PQ
O
Δ SC
Δ PQ
Δ O
motion_change
8.114
8.430
7.907
+5.747
+0.848
+5.295
subject-remove
7.272
8.316
6.911
+3.991
+0.351
+3.706
color_alter
9.275
7.812
8.308
+5.463
+0.050
+4.371
material_alter
7.625
7.600
7.002
+4.438
+0.575
+3.678
subject-replace
7.817
7.933
7.226
+3.658
+0.425
+3.173
Appendix
Table 11: FLUX.2 [dev] results after 2,346 training steps on the five report-selected GEdit categories, evaluated with GPT-4.1 VIEScore. Scores refer to the adapted model, and Δ denotes the change from its paired baseline.
Scope
Pairs
Before
After
Δ
Report Top-5
326
7.767
7.997
+0.230
Full benchmark
995
8.352
8.475
+0.124
Appendix
Table 12: FLUX.2 [dev] WeGenBench Final scores on report-selected Top-5 and full-benchmark common-valid pairs. Baseline and adapted checkpoints share the same evaluated cases.
Setting
Counter- intuitive
Icons / symbols
Animals
IP
Top-5
Full
Baseline
7.150
7.540
8.000
7.810
7.700
8.170
Without web search
8.450
8.440
8.903
8.506
8.650
8.819
With web search
8.700
8.760
8.611
8.671
8.668
8.840
Δ
+0.250
+0.320
-0.292
+0.165
+0.018
+0.021
Appendix
Table 13: Web-search comparison on WeGenBench using Qwen-Image-2512. All scores are F ↑ (1–10). Δ is search-enabled minus search-disabled. Top-5 includes all five selected categories (277 cases), although only four category columns are displayed.
Model Name
EG ↑
PR ↑
VQ ↑
DS ↑
Overall ↑
Closed-source Models
GPT-Image-2
4.424
3.966
4.445
4.225
4.143
Nano-Banana-Pro
4.317
3.640
4.402
4.056
3.963
Qwen-Image-3.0
4.228
3.517
4.359
4.125
3.893
Seedream-5.0-Pro
4.328
3.985
4.424
4.251
4.091
Open-source Models
Appendix
Table 14: Quantitative comparison on WeGenEditBench-SingleImageEdit: Basic Local Editing. Bold and underline denote the best and second-best scores.
Model Name
EG ↑
PR ↑
VQ ↑
DS ↑
Overall ↑
Closed-source Models
GPT-Image-2
4.096
4.068
4.494
4.215
3.974
Nano-Banana-Pro
3.983
3.880
4.424
4.010
3.803
Qwen-Image-3.0
3.916
3.485
4.324
4.149
3.699
Seedream-5.0-Pro
4.035
4.163
4.441
4.189
3.939
Open-source Models
Appendix
Table 15: Quantitative comparison on WeGenEditBench-SingleImageEdit: Aesthetic and Photographic Editing. Bold and underline denote the best and second-best scores.
Model Name
EG ↑
PR ↑
VQ ↑
DS ↑
Overall ↑
Closed-source Models
GPT-Image-2
4.443
4.156
4.464
4.328
4.222
Nano-Banana-Pro
4.390
3.696
4.350
4.226
4.050
Qwen-Image-3.0
4.332
3.451
4.337
4.279
3.977
Seedream-5.0-Pro
4.376
4.125
4.402
4.325
4.162
Open-source Models
Appendix
Table 16: Quantitative comparison on WeGenEditBench-SingleImageEdit: Portrait Editing. Bold and underline denote the best and second-best scores.
Model Name
EG ↑
PR ↑
VQ ↑
DS ↑
Overall ↑
Closed-source Models
GPT-Image-2
4.228
4.134
4.433
4.143
4.036
Nano-Banana-Pro
4.138
3.745
4.423
3.992
3.864
Qwen-Image-3.0
4.153
3.627
4.385
3.923
3.825
Seedream-5.0-Pro
4.155
4.074
4.429
4.184
3.990
Open-source Models
Appendix
Table 17: Quantitative comparison on WeGenEditBench-SingleImageEdit: Reasoning-based Editing. Bold and underline denote the best and second-best scores.
Model Name
EG ↑
PR ↑
VQ ↑
DS ↑
Overall ↑
Closed-source Models
GPT-Image-2
4.117
3.372
4.314
4.125
3.789
Nano-Banana-Pro
4.223
3.269
4.234
4.088
3.806
Qwen-Image-3.0
4.190
3.216
4.230
4.153
3.792
Seedream-5.0-Pro
4.003
3.806
4.471
4.264
3.874
Open-source Models
Appendix
Table 18: Quantitative comparison on WeGenEditBench-SingleImageEdit: Product and Commercial Editing. Bold and underline denote the best and second-best scores.
Model Name
EG ↑
PR ↑
VQ ↑
DS ↑
Overall ↑
Closed-source Models
GPT-Image-2
4.347
4.425
4.565
4.462
4.272
Nano-Banana-Pro
3.612
4.003
4.550
4.392
3.726
Qwen-Image-3.0
3.672
3.950
4.313
4.290
3.678
Seedream-5.0-Pro
4.117
4.202
4.465
4.362
4.044
Open-source Models
Appendix
Table 19: Quantitative comparison on WeGenEditBench-SingleImageEdit: Restoration and Low-level Enhancement. Bold and underline denote the best and second-best scores.
Model Name
EG ↑
PR ↑
VQ ↑
DS ↑
Overall ↑
Closed-source Models
GPT-Image-2
4.376
3.662
4.364
4.009
3.983
Nano-Banana-Pro
4.353
3.317
4.190
3.762
3.804
Qwen-Image-3.0
4.190
3.047
4.198
3.910
3.691
Seedream-5.0-Pro
4.225
3.613
4.312
4.047
3.882
Open-source Models
Appendix
Table 20: Quantitative comparison on WeGenEditBench-SingleImageEdit: Action and Interaction Editing. Bold and underline denote the best and second-best scores.
Model Name
EG ↑
PR ↑
VQ ↑
DS ↑
Overall ↑
Closed-source Models
GPT-Image-2
4.450
4.105
3.802
3.671
3.919
Nano-Banana-Pro
4.566
3.703
3.960
3.774
3.965
Qwen-Image-3.0
4.496
3.264
3.850
3.880
3.840
Seedream-5.0-Pro
4.410
4.340
4.097
4.082
4.104
Open-source Models
Appendix
Table 21: Quantitative comparison on WeGenEditBench-SingleImageEdit: Traditional Chinese Cultural Editing. Bold and underline denote the best and second-best scores.
Model Name
Text-side
Image-side
Overall ↑
EG ↑
PR ↑
VQ ↑
DS ↑
EG ↑
PR ↑
VQ ↑
DS ↑
Closed-source Models
GPT-Image-2
4.249
4.050
4.308
4.356
4.594
3.924
4.409
4.446
4.109
Nano-Banana-Pro
4.317
4.129
4.303
4.305
4.392
4.220
4.227
4.295
4.141
Qwen-Image-3.0
4.350
3.692
4.289
4.395
4.507
3.887
4.408
4.410
4.096
Seedream-5.0-Pro
4.366
4.327
4.412
4.475
4.352
4.218
4.447
4.428
4.260
Appendix
Table 22: Quantitative comparison on WeGenEditBench-TextEdit: Basic Text Editing. Bold and underline denote the best and second-best scores.
Model Name
Text-side
Image-side
Overall ↑
EG ↑
PR ↑
VQ ↑
DS ↑
EG ↑
PR ↑
VQ ↑
DS ↑
Closed-source Models
GPT-Image-2
4.144
3.817
4.372
4.439
4.567
3.908
4.483
4.471
4.038
Nano-Banana-Pro
4.032
4.032
4.326
4.177
4.304
4.208
4.216
4.268
3.913
Qwen-Image-3.0
4.257
3.509
4.477
4.289
4.528
3.854
4.463
4.390
4.015
Seedream-5.0-Pro
4.288
4.152
4.316
4.496
4.260
4.187
4.528
4.407
4.152
Appendix
Table 23: Quantitative comparison on WeGenEditBench-TextEdit: Posters and Layout. Bold and underline denote the best and second-best scores.
Model Name
Text-side
Image-side
Overall ↑
EG ↑
PR ↑
VQ ↑
DS ↑
EG ↑
PR ↑
VQ ↑
DS ↑
Closed-source Models
GPT-Image-2
4.341
4.120
4.329
4.423
4.639
4.139
4.500
4.583
4.187
Nano-Banana-Pro
4.260
3.949
4.333
4.434
4.595
4.351
4.351
4.351
4.103
Qwen-Image-3.0
4.385
3.856
4.299
4.297
4.639
3.833
4.167
4.500
4.106
Seedream-5.0-Pro
4.535
4.371
4.435
4.576
4.541
4.027
4.486
4.568
4.392
Appendix
Table 24: Quantitative comparison on WeGenEditBench-TextEdit: Interfaces, Documents, and Charts. Bold and underline denote the best and second-best scores.
Model Name
Text-side
Image-side
Overall ↑
EG ↑
PR ↑
VQ ↑
DS ↑
EG ↑
PR ↑
VQ ↑
DS ↑
Closed-source Models
GPT-Image-2
4.448
3.676
4.295
4.423
4.782
3.818
4.473
4.518
4.160
Nano-Banana-Pro
4.382
4.027
4.282
4.408
4.508
3.847
4.220
4.432
4.139
Qwen-Image-3.0
4.418
3.455
4.391
4.515
4.508
3.763
4.322
4.356
4.139
Seedream-5.0-Pro
4.472
4.241
4.380
4.578
4.474
3.895
4.351
4.368
4.296
Appendix
Table 25: Quantitative comparison on WeGenEditBench-TextEdit: Commercial Text–Image Editing. Bold and underline denote the best and second-best scores.
Model Name
Text-side
Image-side
Overall ↑
EG ↑
PR ↑
VQ ↑
DS ↑
EG ↑
PR ↑
VQ ↑
DS ↑
Closed-source Models
GPT-Image-2
4.457
3.837
4.511
4.454
4.571
3.839
4.304
4.348
4.227
Nano-Banana-Pro
4.274
4.105
4.274
4.541
4.228
4.404
4.018
4.351
4.127
Qwen-Image-3.0
4.448
3.615
4.521
4.549
4.534
3.810
4.483
4.466
4.209
Seedream-5.0-Pro
4.333
4.312
4.387
4.449
4.175
4.404
4.368
4.395
4.195
Appendix
Table 26: Quantitative comparison on WeGenEditBench-TextEdit: Social-media Text–Image Editing. Bold and underline denote the best and second-best scores.
Model Name
EG ↑
PR ↑
VQ ↑
Overall ↑
Closed-source Models
GPT-Image-2
4.315
4.107
3.658
4.101
Nano-Banana-Pro
4.284
4.122
3.236
4.009
Seedream-5.0-Pro
4.374
4.094
3.835
4.154
Open-source Models
HunyuanImage-3.0-Instruct-Distil
3.496
3.656
3.216
3.504
Appendix
Table 27: Quantitative comparison on WeGenEditBench-MultiImageEdit: Portrait Editing. Bold and underline denote the best and second-best scores.
Model Name
EG ↑
PR ↑
VQ ↑
Overall ↑
Closed-source Models
GPT-Image-2
4.220
4.068
3.339
3.983
Nano-Banana-Pro
3.795
3.889
3.043
3.682
Seedream-5.0-Pro
4.250
4.056
3.361
3.994
Open-source Models
HunyuanImage-3.0-Instruct-Distil
3.198
3.089
2.861
3.087
Appendix
Table 28: Quantitative comparison on WeGenEditBench-MultiImageEdit: Product and Commercial Editing. Bold and underline denote the best and second-best scores.
Model Name
EG ↑
PR ↑
VQ ↑
Overall ↑
Closed-source Models
GPT-Image-2
4.147
3.983
3.690
3.990
Nano-Banana-Pro
3.664
3.779
3.319
3.641
Seedream-5.0-Pro
4.018
3.936
3.697
3.921
Open-source Models
HunyuanImage-3.0-Instruct-Distil
2.700
3.130
2.730
2.878
Appendix
Table 29: Quantitative comparison on WeGenEditBench-MultiImageEdit: Text Editing. Bold and underline denote the best and second-best scores.
Model Name
EG ↑
PR ↑
VQ ↑
Overall ↑
Closed-source Models
GPT-Image-2
4.453
4.222
3.795
4.229
Nano-Banana-Pro
4.060
3.966
3.368
3.884
Seedream-5.0-Pro
4.272
4.123
3.658
4.089
Open-source Models
HunyuanImage-3.0-Instruct-Distil
3.209
3.549
2.978
3.299
Appendix
Table 30: Quantitative comparison on WeGenEditBench-MultiImageEdit: Scene and Spatial Editing. Bold and underline denote the best and second-best scores.
Model Name
EG ↑
PR ↑
VQ ↑
Overall ↑
Closed-source Models
GPT-Image-2
4.579
4.211
3.833
4.282
Nano-Banana-Pro
4.357
4.018
3.384
4.027
Seedream-5.0-Pro
4.463
4.176
3.926
4.241
Open-source Models
HunyuanImage-3.0-Instruct-Distil
3.787
3.734
3.468
3.702
Appendix
Table 31: Quantitative comparison on WeGenEditBench-MultiImageEdit: Pet Editing. Bold and underline denote the best and second-best scores.
Model Name
EG ↑
PR ↑
VQ ↑
Overall ↑
Closed-source Models
GPT-Image-2
4.455
4.223
3.955
4.263
Nano-Banana-Pro
3.955
4.153
3.739
3.991
Seedream-5.0-Pro
4.216
4.171
4.000
4.155
Open-source Models
HunyuanImage-3.0-Instruct-Distil
3.255
3.798
3.287
3.479
Appendix
Table 32: Quantitative comparison on WeGenEditBench-MultiImageEdit: Aesthetic and Photographic Editing. Bold and underline denote the best and second-best scores.
\TableCaptionLabel Table 33 Single-Image Task Taxonomy. Family headings give unique covered instances; rows define the second-level tasks. The subset has 1,008 instances and 65 task labels.
Second-level task
Operational definition
Basic local editing 631 unique instances
Object addition
Insert a specified object with correct location, scale, and occlusion.
Object removal
Remove a target, reconstruct its background, and preserve surrounding content.
Object replacement
Replace a target while respecting its scene role.
Object count change
Change the specified objects to the requested count.
Appendix
Table 41
\TableCaptionLabel Table 34 Text-Intensive Task Taxonomy. Family headings give unique covered instances; rows define the second-level tasks. The subset has 1,000 instances and 35 task labels.
Second-level task
Operational definition
Basic text editing 678 unique instances
Text-content replacement
Replace specified wording while preserving placement and surrounding content.
Text insertion
Insert wording without obscuring protected visual information.
Text removal
Remove text, reconstruct its carrier, and preserve non-target content.
Typography preservation
Change wording while preserving specified typography and arrangement.
Appendix
Table 42
\TableCaptionLabel Table 35 Multi-Reference Task Taxonomy. Family headings give unique covered instances; rows define the second-level tasks. The subset has 732 instances and 48 task labels.
Second-level task
Operational definition
Portrait editing 150 unique instances
Face transfer
Transfer facial identity while preserving required body, pose, and scene.
Reference-based clothing transfer
Transfer reference clothing while preserving the target person’s identity.
Hairstyle transfer
Transfer hairstyle without substituting unrelated identity attributes.
Expression transfer
Transfer expression while preserving the target person’s identity.
Appendix
Table 43
Figure 9: WeGenEditBench-SingleImageEdit: object addition (case 350). The panel shows the input image(s), editing instruction, and outputs from the labeled models.
Figure 10: WeGenEditBench-SingleImageEdit: object removal (case 110). The panel shows the input image(s), editing instruction, and outputs from the labeled models.
Figure 11: WeGenEditBench-SingleImageEdit: clothing-style editing (case 700). The panel shows the input image(s), editing instruction, and outputs from the labeled models.
Figure 12: WeGenEditBench-SingleImageEdit: lighting redesign (case 118). The panel shows the input image(s), editing instruction, and outputs from the labeled models.
Figure 13: WeGenEditBench-SingleImageEdit: exploded product view (case 359). The panel shows the input image(s), editing instruction, and outputs from the labeled models.
Figure 14: WeGenEditBench-SingleImageEdit: dieline-to-package rendering (case 495). The panel shows the input image(s), editing instruction, and outputs from the labeled models.
Figure 15: WeGenEditBench-SingleImageEdit: spatial-reasoning editing (case 142). The panel shows the input image(s), editing instruction, and outputs from the labeled models.
Figure 16: WeGenEditBench-SingleImageEdit: causal-scene editing (case 272). The panel shows the input image(s), editing instruction, and outputs from the labeled models.
Figure 17: WeGenEditBench-SingleImageEdit: old-photo restoration (case 356). The panel shows the input image(s), editing instruction, and outputs from the labeled models.
Figure 18: WeGenEditBench-SingleImageEdit: drawing colorization (case 241). The panel shows the input image(s), editing instruction, and outputs from the labeled models.
Figure 19: WeGenEditBench-SingleImageEdit: hand–object interaction (case 661). The panel shows the input image(s), editing instruction, and outputs from the labeled models.
Figure 20: WeGenEditBench-SingleImageEdit: chinese traditional culture editing (case 550). The panel shows the input image(s), editing instruction, and outputs from the labeled models.
Figure 21: WeGenEditBench-TextEdit: multilingual text replacement (case 418). The panel shows the input image(s), editing instruction, and outputs from the labeled models.
Figure 22: WeGenEditBench-TextEdit: perspective-aware text editing (case 541). The panel shows the input image(s), editing instruction, and outputs from the labeled models.
Figure 23: WeGenEditBench-TextEdit: text removal (case 650). The panel shows the input image(s), editing instruction, and outputs from the labeled models.
Figure 24: WeGenEditBench-TextEdit: product-packaging text editing (case 399). The panel shows the input image(s), editing instruction, and outputs from the labeled models.
Figure 25: WeGenEditBench-TextEdit: product-feature labels (case 502). The panel shows the input image(s), editing instruction, and outputs from the labeled models.
Figure 26: WeGenEditBench-TextEdit: poster layout reorganization (case 111). The panel shows the input image(s), editing instruction, and outputs from the labeled models.
Figure 27: WeGenEditBench-TextEdit: magazine cover (case 287). The panel shows the input image(s), editing instruction, and outputs from the labeled models.
Figure 28: WeGenEditBench-TextEdit: multi-format and multi-platform adaptation (case 980). The panel shows the input image(s), editing instruction, and outputs from the labeled models.
Figure 29: WeGenEditBench-TextEdit: interface and screenshot text editing (case 446). The panel shows the input image(s), editing instruction, and outputs from the labeled models.
Figure 30: WeGenEditBench-TextEdit: local table and document editing (case 291). The panel shows the input image(s), editing instruction, and outputs from the labeled models.
Figure 31: WeGenEditBench-TextEdit: meme and sticker design (case 27). The panel shows the input image(s), editing instruction, and outputs from the labeled models.
Figure 32: WeGenEditBench-TextEdit: pet identification card (case 794). The panel shows the input image(s), editing instruction, and outputs from the labeled models.
Figure 33: WeGenEditBench-MultiImageEdit: interpersonal action transfer (case 164). The panel shows the input image(s), editing instruction, and outputs from the labeled models.
Figure 34: WeGenEditBench-MultiImageEdit: model–product–scene composition (case 279). The panel shows the input image(s), editing instruction, and outputs from the labeled models.
Figure 35: WeGenEditBench-MultiImageEdit: multi-product composition (case 573). The panel shows the input image(s), editing instruction, and outputs from the labeled models.
Figure 36: WeGenEditBench-MultiImageEdit: font-reference transfer (case 439). The panel shows the input image(s), editing instruction, and outputs from the labeled models.
Figure 37: WeGenEditBench-MultiImageEdit: scene lighting and time-of-day transfer (case 173). The panel shows the input image(s), editing instruction, and outputs from the labeled models.
Figure 38: WeGenEditBench-MultiImageEdit: map or floor-plan to scene (case 674). The panel shows the input image(s), editing instruction, and outputs from the labeled models.
Figure 39: WeGenEditBench-MultiImageEdit: pet-to-toy transformation (case 277). The panel shows the input image(s), editing instruction, and outputs from the labeled models.
Figure 40: WeGenEditBench-MultiImageEdit: style transfer (case 203). The panel shows the input image(s), editing instruction, and outputs from the labeled models.
Figure 41: Qualitative text-to-image comparisons before and after WeGenEvoStudio adaptation. Each triplet shows the prompt, baseline, and adapted output, with FLUX.2 [dev] on the left and Qwen-Image-2512 on the right.
Recent advances in image editing have enabled models to handle complex instructions with impressive realism. However, existing evaluation frameworks lag behind: current benchmarks suffer from narrow task coverage, while standard metrics fail to adequately capture visual consistency, i.e., the preservation of identity, structure and semantic coherence between edited and original images. To address these limitations, we introduce GEditBench v2, a comprehensive benchmark with 1,200 real-world user queries spanning 23 tasks, including a dedicated open-set category for unconstrained, out-of-distribution editing instructions beyond predefined tasks. Furthermore, we propose PVC-Judge, an open-source pairwise assessment model for visual consistency, trained via two novel region-decoupled preference data synthesis pipelines. Besides, we construct VCReward-Bench using expert-annotated preference pairs to assess the alignment of PVC-Judge with human judgments on visual consistency evaluation. Experiments show that our PVC-Judge achieves state-of-the-art evaluation performance among open-source models and even surpasses GPT-5.1 on average. Finally, by benchmarking 16 frontier editing models, we show that GEditBench v2 enables more human-aligned evaluation, revealing critical limitations of current models, and providing a reliable foundation for advancing precise image editing.
Zhangqi Jiang, Zheng Sun, Xianfang Zeng +7
Nanyang Technological University · StepFun · Southeast University
Recent image editing models have achieved remarkable progress in instruction following, multimodal understanding, and complex visual editing. However, existing benchmarks often fail to faithfully reflect human judgment, especially for strong frontier models, due to limited task difficulty and coarse-grained evaluation protocols. In parallel, reward models have become increasingly important for RL-based image editing optimization, yet existing reward model benchmarks still rely on unrealistic evaluation settings that deviate from practical RL scenarios. These limitations hinder reliable assessment of both image editing models and reward models. To address these challenges, we introduce Edit-Compass and EditReward-Compass, a unified evaluation suite for image editing and reward modeling. Edit-Compass contains 2,388 carefully annotated instances spanning six progressively challenging task categories, covering capabilities such as world knowledge reasoning, visual reasoning, and multi-image editing. Beyond broad task coverage, Edit-Compass adopts a fine-grained multidimensional evaluation framework based on structured reasoning and carefully designed scoring rubrics. In parallel, EditReward-Compass contains 2,251 preference pairs that simulate realistic reward modeling scenarios during RL optimization.
Recent text-guided image editing (TIE) models have achieved remarkable progress, however, many edited results still suffer from artifacts, unintended modifications, and suboptimal aesthetics. Although several benchmarks and evaluation methods have been proposed, most existing approaches rely on scalar scores and lack interpretability. This limitation largely stems from the absence of high-quality interpretation datasets for TIE and effective reward models to train interpretable evaluators. To address these challenges, we introduce ReasonEdit-22K, the first dataset that combines 22K edited images with 113K Chain-of-Thought (CoT) samples, along with 1.3M human judgments assessing these interpretations in terms of logicality, accuracy, and usefulness. Building upon this dataset, we propose RE-Reward, a multimodal large language model (MLLM)-based reward model designed to provide human-aligned feedback for evaluating interpretable reasoning in image editing. Furthermore, we develop ReasonEdit, which is trained using reward signals derived from RE-Reward and the Group Relative Policy Optimization (GRPO) algorithm to learn an interpretable evaluation model. Extensive experiments demonstrate that ReasonEdit achieves superior alignment with human preferences and exhibits strong generalization across public benchmarks. In addition, it is capable of generating high-quality interpretable evaluation text, enabling more transparent and trustworthy assessment for image editing. The code is available at https://github.com/IntMeGroup/ReasonEdit.
Honghua Chen, Zitong Xu, Huiyu Duan +3
University of Electronic Science and Technology of China · 2Shanghai Jiao Tong University