Post-training aims to enhance desired attributes in visual generative models while preserving their general generation capabilities. Reward-guided post-training faces two challenges: real training data may mismatch the generator's output distribution, and attribute supervision may be confounded by unrelated content. Unbounded reward maximization can also cause overoptimization and degrade generation quality. We propose a framework combining Generator-Aligned Reward Modeling (GARM) and Target-Aware Bounded Optimization (TABO). GARM mitigates distribution mismatch through generator-support matching and reduces content confounding using content-matched real contrasts, yielding generator-relevant, attribute-focused reward supervision. TABO maps the learned reward to a smooth, upper-bounded function calibrated from real target-positive examples. This transformation is independent of the optimization algorithm and supports both direct reward-gradient and policy-based optimization. We further introduce Target-Aware GRPO (TA-GRPO), an adaptation of Flow-GRPO that combines fixed stage-wise advantage scaling, a generator-to-target curriculum, and timestep-balanced credit assignment for flow-based video optimization. Experiments show that GARM improves target-attribute expression while maintaining competitive generation quality. TABO better balances attribute enhancement and visual quality than unconstrained reward maximization, while TA-GRPO improves target enhancement over vanilla Flow-GRPO and reduces unintended attribute leakage on neutral prompts. These results highlight the importance of generator-aligned reward supervision and controlled, data-calibrated optimization for targeted visual generation.
Figures & tables
Figure 1: Targeted attribute enhancement with largely preserved general generation performance. Top: Base vs. Ours across four attributes (Vintage Faded, Oil Painting, Muscular, and 2D Japanese anime animation). Bottom: radar comparisons of Base (gray dashed, normalized to 100) and Ours.
Figure 2: Overview of our framework. GARM aligns real supervision with the generator-relevant regions defined by paired Base/Target anchors and introduces content-matched negatives to form an attribute-focused decision boundary. TABO maps the learned reward to a smooth, upper-bounded reward function, preserving sensitivity below a target level calibrated from real positives while reducing the marginal benefit of further score increases near and above it. Because this reward transformation is independent of the update rule, it can be used with differentiable optimization (VADER) and policy optimization (TA-GRPO).
Task
Matched positive
Matched negative
Content-matched negative
Vintage Faded
1,110
38,498
7,092
Oil Painting
585
39,117
4,169
Muscular
616
37,502
4,328
Anime video
7,737
52,975
7,933
Table 1: Core GARM pools used in the image and video experiments. The full pool definitions and sampling ratios are provided in the appendix.
Method
Mean ↑
P20 ↑
HPSv3 ↑
Q-Align Q. ↑
Q-Align Aes. ↑
HPSv2 ↑
PickScore ↑
VQA ↑
Muscular
Base (ref.)
51.838
24.835
9.469
4.826
3.881
0.307
20.582
0.706
Baseline
57.341
29.196
9.340
4.753
3.778
0.308
20.554
0.684
Generated-only
55.034
31.043
8.983
4.568
3.534
0.305
20.507
0.680
Content-Matched-only
57.445
29.880
9.427
4.812
3.915
0.310
20.564
0.700
Support-only
59.173
31.807
9.551
4.703
3.754
0.310
20.602
0.697
Table 2: Post-target results under different reward-supervision constructions. Target strength is measured by percentile statistics calibrated on the RGB Target-val anchors. Base is shown only as a reference; bold denotes the best post-trained result and underlining denotes the second-best post-trained result for each metric.
Method
Leakage ↓
HPSv3 ↑
Q-Align Q. ↑
Q-Align Aes. ↑
HPSv2 ↑
PickScore ↑
VQA ↑
Muscular
Base (ref.)
0.000
10.847
4.751
3.765
0.305
20.356
0.786
Baseline
0.011
10.678
4.696
3.667
0.306
20.328
0.783
Generated-only
0.002
10.584
4.618
3.568
0.304
20.307
0.788
Content-Matched-only
0.015
10.779
4.743
3.770
0.308
20.323
0.785
Support-only
0.020
10.823
4.653
3.628
0.306
20.354
0.786
Table 3: Post-neutral preservation under different reward-supervision constructions. Leakage measures the increase in target-tag probability relative to Base and is lower-is-better. Base is shown only as a reference; bold denotes the best post-trained result and underlining denotes the second-best post-trained result for each metric.
Figure 3: Qualitative comparison of Base, SFT, Baseline, Generated-only, and GARM on Muscular, Oil Painting, and Vintage Faded. The examples illustrate target-attribute expression and preservation of scene content and overall appearance.
(a) Post-target enhancement
Attribute
VBench
VideoScore2
Method
Mean ↑
P20 ↑
SC ↑
DD ↑
Aes. ↑
Mean ↑
VQ ↑
TA ↑
PC ↑
Mean ↑
Base (ref.)
59.263
43.430
0.9725
0.2117
0.6655
0.6166
3.9720
3.8826
3.7742
3.8763
Baseline
68.579
55.417
0.9717
0.1350
0.6362
0.5810
3.9586
4.1870
3.7643
3.9700
Generated-only
68.079
53.974
0.9684
0.1150
0.6288
0.5707
3.9713
3.9709
3.7016
3.8813
Content-Matched-only
65.589
51.195
0.9770
0.1400
0.6444
0.5871
3.9384
4.0143
3.7400
3.8976
Table 4: Post-target enhancement and post-neutral preservation on Anime video (Wan2.1-T2V-14B). Mean and P20 are RGB target-support percentiles; Leakage is the mean RGB logit increase relative to Base on neutral prompts. SC, DD, and Aes. denote VBench subject consistency, dynamic degree, and aesthetic quality; VQ, TA, and PC denote VideoScore2 visual quality, text alignment, and physical consistency. VBench Mean and VideoScore2 Mean are arithmetic means of the three corresponding metrics. Base is a reference; bold and underlining indicate the best and second-best post-trained values, respectively, with ties determined at the displayed precision.
Figure 4: Qualitative comparison of video generation methods for the target attribute 2D Japanese anime animation using Wan2.1-T2V-14B. GARM enhances the target animation style while preserving semantic content and temporal coherence.
Muscular
Oil Painting
Vintage Faded
Anime (video)
Method
Median ↑
Mean ↑
P20 ↑
Median ↑
Mean ↑
P20 ↑
Median ↑
Mean ↑
P20 ↑
Median ↑
Mean ↑
P20 ↑
Base (ref.)
48.184
51.838
24.835
48.018
51.282
26.436
49.264
50.733
22.606
60.778
59.263
43.430
SFT
47.776
50.460
24.319
64.091
60.614
33.835
57.300
56.889
26.660
65.087
66.774
50.822
Baseline
57.867
57.341
29.196
80.128
74.455
55.177
77.205
72.862
54.527
68.066
68.579
55.417
GARM
65.802
62.862
36.896
82.459
76.102
58.436
80.903
74.425
54.624
69.045
68.722
54.267
Table 5: SFT versus reward-guided post-training. We report RGB target-strength percentiles for three image attributes and Anime video generated with Wan2.1-T2V-14B. SFT applies LoRA updates using only real positive examples, while Baseline uses the corresponding real supervision to train a reward model for reward-guided post-training. Base is shown only as a reference; bold denotes the best post-trained result and underlining denotes the second-best post-trained result for each metric.
(a) VADER
Attribute
VBench
VideoScore2
Method
Mean ↑
P20 ↑
SC ↑
DD ↑
Aes. ↑
Mean ↑
VQ ↑
TA ↑
PC ↑
Mean ↑
Base (ref.)
59.263
43.430
0.9725
0.2117
0.6655
0.6166
3.9720
3.8826
3.7742
3.8763
Raw Reward Maximization
69.543
56.695
0.9763
0.1200
0.6359
0.5774
3.9897
3.9715
3.7782
3.9131
TABO
68.722
54.267
0.9772
0.1450
0.6434
0.5885
4.0182
4.1391
3.8467
4.0013
(b) FlowGRPO
Table 6: Post-target enhancement for target-aware optimization. The metrics follow Table 4 : target strength is reported by RGB percentile mean and P20, VBench uses subject consistency (SC), dynamic degree (DD), aesthetic quality (Aes.), and their arithmetic mean, and VideoScore2 uses visual quality (VQ), text alignment (TA), physical consistency (PC), and their arithmetic mean. Results are compared within each optimizer and backbone. VADER uses Wan2.1-T2V-14B and FlowGRPO uses Wan2.1-T2V-1.3B. A dash denotes an experiment that is not yet available. Base is a reference only; bold marks the best post-trained value within each panel.
Figure 5: Image-space diagnostic of TABO on Vintage Faded and Oil Painting. Bounded reward optimization is compared with direct raw-reward maximization using training reward, HPSv3, Q-Align aesthetic quality, and Q-Align quality.
Figure 6: Qualitative comparison between raw-reward maximization and TABO on 2D Japanese anime. Raw-reward maximization exhibits more pronounced visual degradation, while TABO better preserves subject appearance and scene content. Top: Raw Reward; Bottom: Ours (TABO).
Figure 7: Qualitative comparison of Base, Flow-GRPO, and our TA-GRPO for 2D Japanese anime animation. TA-GRPO strengthens style alignment while preserving scene semantics and composition.
Table 11: Video RGB evaluator validation results for Anime.
(a) Post-target enhancement
VBench
VideoReward
Method
SC ↑
BC ↑
MS ↑
DD ↑
Aes. ↑
IQ ↑
Mean ↑
VQ ↑
MQ ↑
TA ↑
Mean ↑
Base (ref.)
0.9725
0.9797
0.9932
0.2117
0.6655
0.6888
0.7519
-0.0836
-0.4643
0.1657
-0.1274
Baseline
0.9717
0.9763
0.9929
0.1350
0.6362
0.6824
0.7324
-0.1735
-0.5249
0.1437
-0.1849
Generated-only
0.9684
0.9763
0.9941
0.1150
0.6288
0.6877
0.7284
-0.2787
-0.4970
0.1179
-0.2193
Graph-only
0.9770
0.9803
0.9940
0.1400
0.6444
0.7218
0.7429
-0.1349
-0.5329
0.1712
-0.1656
Appendix
Table 12: Complete post-target and post-neutral VBench and VideoReward results on Anime video (Wan2.1-T2V-14B). SC, BC, MS, DD, Aes., and IQ denote VBench subject consistency, background consistency, motion smoothness, dynamic degree, aesthetic quality, and imaging quality; VQ, MQ, and TA denote VideoReward visual quality, motion quality, and text alignment. Base is a reference; bold and underlining indicate the best and second-best post-trained values within each panel, respectively, with ties determined at the displayed precision.
(a) VADER
Attribute
VBench
VideoScore2
Method
Leakage ↓
SC ↑
DD ↑
Aes. ↑
Mean ↑
VQ ↑
TA ↑
PC ↑
Mean ↑
Base (ref.)
0.0000
0.9706
0.3467
0.6689
0.6621
4.3207
4.4130
4.1911
4.3083
Raw Reward Maximization
0.0341
0.9742
0.3050
0.6526
0.6439
4.3217
4.3756
4.1806
4.2926
TABO
0.0101
0.9738
0.3050
0.6546
0.6445
4.3854
4.4665
4.2143
4.3554
(b) Flow-GRPO
Appendix
Table 13: Post-neutral preservation for target-aware optimization. The metrics follow Table 4 : leakage is the mean increase in the attribute-specific RGB logit relative to Base and is lower-is-better; VBench and VideoScore2 report the same component metrics and arithmetic means as in the post-target table. Results are compared within each optimizer and backbone.
Modality / tag
Pair count
Preference rate
Target–Base AUROC
Image / Vintage Faded
2,000
0.9265
0.823734
Image / Oil Painting
2,000
0.9395
0.841797
Image / Muscular
2,000
0.7455
0.686098
Video / Anime
250
0.9200
0.871304
Appendix
Table 14: Latent reward-model validation and checkpoint-selection statistics.