CRAFT: Constrained Reward via Attention Fine-Tuning for Subject Personalization without Composed Targets

Jihun Park; Kyoungmin Lee; Jongmin Gim; Hyeonseo Jo; Jaeyeul Kim; Han Zou; Zhenpeng Zhan; Yan Zhang

cs.CVAug 14, 2026

CRAFT: Constrained Reward via Attention Fine-Tuning for Subject Personalization without Composed Targets

Authors: Jihun Park, Kyoungmin Lee, Jongmin Gim, Hyeonseo Jo, Jaeyeul Kim, Han Zou, Zhenpeng Zhan, Yan Zhang,

Organizations: DGIST, Daegu, South Korea · Baidu, Inc., Shenzhen, China · KAIST, Daejeon, South Korea

Abstract

Subject-driven image personalization---generating new images that preserve the identity of one or several reference subjects in novel scenes---is a foundational capability for modern visual content creation. It is currently dominated by generalized methods that fine-tune a pretrained multimodal diffusion transformer (MMDiT) on hundreds of thousands to millions of paired (reference, composed-target) examples, where each composed target is a synthesized image of the subject in a novel scene. Producing such targets demands a costly multi-stage curation pipeline---LLM-based prompt generation, T2I-based composed-target synthesis, reference-subject extraction, VLM-based quality filtering, and correspondence labeling---and tightly couples each method to a particular target synthesizer and curation choice. We introduce CRAFT (Constrained Reward via Attention Fine-Tuning), a single-step ReFL framework that fine-tunes a pre-trained reference-aware MMDiT via LoRA adapters using a compact reference-only data construction--- $10$ K reference images and subject masks, with no composed-target supervision. CRAFT realizes a Where to look principle: attention-level rewards align noise- and phrase-token attention with the correct reference subject, and the resulting per-subject attention masks gate a pixel-level identity reward to keep image-space supervision consistent with the learned attention routing. Applied to FLUX.2-klein-9B, CRAFT achieves state-of-the-art performance on XVerseBench \rev{while using no composed-target supervision---only $10$ K reference-only samples, whereas prior generalized methods require $150$ K to over $2$ M composed-target pairs}. The same recipe transfers to other reference-aware backbones, consistently improving performance. Project page: https://jihun999.github.io/projects/CRAFT/.

Figures & tables

Figure 1. Overall pipeline of CRAFT. The denoising prefix is rolled out without gradients up to the reward step $t^{\ast}$ , where two forward passes are performed: the frozen backbone $f_{\text{base}}$ produces $\mathbf{v}_{\text{base}}$ , while the LoRA-adapted model $f_{\text{LoRA}}$ produces $\mathbf{v}_{\text{LoRA}}$ together with the cross-modal attention sub-blocks $\mathbf{A}_{N2R_{k}}$ , $\mathbf{A}_{N2P_{k}}$ , and $\mathbf{A}_{P_{k}2R_{k}}$ . Following the Where to look principle, $\mathcal{R}_{\text{ref}}$ and $\mathcal{R}_{\text{cons}}$ shape noise- and phrase-token attention toward each reference subject (Section 4.2 ); the resulting per-subject mask $\mathbf{m}^{\text{noise}}_{k}$ then gates the pixel-level identity reward $\mathcal{R}_{\text{id}}$ on the VAE-decoded pre-image $\hat{\mathbf{I}}$ (Section 4.3 ). Auxiliary terms $\mathcal{R}_{\text{CLIP-T}}$ , $\mathcal{R}_{\text{AES}}$ , and a velocity-space regularizer $\mathcal{L}_{\text{anchor}}$ are added for prompt fidelity, aesthetics, and stability (Section 4.4 ). A single backward pass updates only the LoRA parameters.

			Single subject (90 prompts)					Multi subject (210 prompts)					Overall $\uparrow$
Method	Target.	#Data	DPG $\uparrow$	ID $\uparrow$	IP $\uparrow$	AES $\uparrow$	AVG $\uparrow$	DPG $\uparrow$	ID $\uparrow$	IP $\uparrow$	AES $\uparrow$	AVG $\uparrow$	Overall $\uparrow$
UNO ( Wu et al., 2025d )	✓	245K	89.65	47.91	80.40	55.90	68.47	85.28	31.82	67.00	54.24	59.59	64.03
OmniGen ( Xiao et al., 2025b )	✓	6.5M	83.90	76.51	78.46	51.41	72.57	78.23	55.53	62.32	49.84	61.48	67.03
OmniGen2 ( Wu et al., 2025b )	✓	180K	92.60	62.41	74.08	52.34	70.36	91.55	40.81	67.15	51.40	62.73	66.55
DreamO ( Mou et al., 2025 )	✓	150K	96.93	75.48	70.84	54.57	74.46	88.80	50.24	64.63	52.47	64.04	69.25
UMO † ( Cheng et al., 2025 )	✓	—	86.75	77.36	76.99	61.71	75.70	87.18	58.24	60.52	58.74	66.17	70.94

Table 1. Quantitative comparison on XVerseBench. “Target.” marks composed-target supervision (✓/✗); “#Data” is the reported training-sample count. † denotes results reproduced with official code; ( — ) denotes not reported. Best/second-best in bold / underlined . ∗ CRAFT (mask-free) operates on raw, unsegmented references at inference and is reported for reference (not included in the ranking). Because masks are reward-side training annotations only, this mask-free setting is CRAFT’s intended inference mode, and it performs on par with or above the segmented-input configuration (Overall

77.80

vs.

76.47

Figure 2. Qualitative comparison with state-of-the-art subject-driven personalization models.

	Component			Single subject					Multi subject					Overall $\uparrow$
	$\mathcal{R}_{\text{ref}}$	$\mathcal{R}_{\text{cons}}$	$\mathcal{R}_{\text{id}}$	DPG $\uparrow$	ID $\uparrow$	IP $\uparrow$	AES $\uparrow$	AVG $\uparrow$	DPG $\uparrow$	ID $\uparrow$	IP $\uparrow$	AES $\uparrow$	AVG $\uparrow$	Overall $\uparrow$
(a)				97.22	66.29	78.15	59.76	75.36	88.91	50.22	71.23	56.52	66.72	71.04
(b)	✓			96.89	70.15	79.75	60.84	76.91	88.94	48.11	72.69	57.86	66.90	71.91
(c)	✓	✓		97.22	70.63	81.19	61.94	77.75	88.09	50.43	70.70	58.71	66.98	72.37
(d)			✓	96.67	73.27	81.93	61.96	78.46	87.32	50.77	72.90	60.24	67.81	73.14
(e)	✓	✓	✓	96.81	84.22	84.23	61.24	81.62	88.71	61.16	77.25	58.16	71.32	76.47

Table 2. Component ablations on XVerseBench. (a) FLUX.2-klein ( Labs, 2025 ) backbone; (b)

+\mathcal{R}_{\text{ref}}

; (c)

+\mathcal{R}_{\text{cons}}

(cumulative); (d)

\mathcal{R}_{\text{id}}

alone on the backbone; (e) full CRAFT (all three rewards). ✓marks an enabled reward. Best/second-best in bold / underlined .

Method	Target.	Single AVG $\uparrow$	Multi AVG $\uparrow$	Overall $\uparrow$
UNO ( Wu et al., 2025d )	✓	68.47	59.59	64.03
UNO + Ours	✗	76.61	67.04	71.83
UMO † ( Cheng et al., 2025 )	✓	75.70	66.17	70.94
UMO + Ours	✗	79.75	69.49	74.62

Table 3. CRAFT applied to UNO ( Wu et al., 2025d ) and UMO ( Cheng et al., 2025 ) on XVerseBench. “Target.” marks composed-target supervision (✓/✗); † denotes results reproduced with official code. Best in bold .

Method	IC $\uparrow$	PF $\uparrow$	IQ $\uparrow$
UMO ( Cheng et al., 2025 )	15.0	5.9	26.0
XVerse ( Chen et al., 2025 )	15.0	18.3	16.5
MOSAIC ( She et al., 2025 )	14.8	13.2	23.3
Ours	55.2	62.6	34.2

Table 4. User study preference (%).

Figure 3. Qualitative results on XVerseBench under the official segmented-reference protocol.

Figure 4. Qualitative results of CRAFT on XVerseBench with un-segmented raw reference images at inference (mask-free protocol).

Figure 5. Qualitative results of CRAFT for the OmniContext benchmark.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

(a) Same block ( single_1 ) across steps
Step	Attention map	BCE $\downarrow$	MSE $\downarrow$	soft mIoU $\uparrow$	BinIoU $\uparrow$
$0$	$\mathbf{A}_{N2R}$ (noise $\to$ ref-image)	0.327	0.099	0.497	0.483
$0$	$\mathbf{A}_{N2P}$ (noise $\to$ prompt)	0.317	0.102	0.503	0.440
$1$	$\mathbf{A}_{N2R}$ (noise $\to$ ref-image)	0.240	0.071	0.574	0.622
$1$	$\mathbf{A}_{N2P}$ (noise $\to$ prompt)	0.305	0.098	0.521	0.565
$\mathbf{2}$	$\mathbf{A}_{N2R}$ (noise $\to$ ref-image)	0.232	0.071	0.586	0.666

Appendix

Table 5. Alignment of attention heatmaps with the Grounded-SAM segmentation of the generated image across all four denoising steps of the

4

-step distilled FLUX.2-klein-9B, on

100

subjects under the measurement protocol of Section A (BinIoU at the

75

-th percentile binary mask; un-thresholded heatmap for BCE, MSE, soft mIoU;

64{\times}64

resolution; no Gaussian smoothing). Bold

=

better between the two key types within the same step. (a) same block single_1 ; (b) the per-step top-

3

block mean, where the top-

3

blocks are selected per step per key type by average rank across the four metrics.

Figure 6. Qualitative comparison of raw (un-thresholded) $\mathbf{A}_{N2R}$ (noise-to-reference) and $\mathbf{A}_{N2P}$ (noise-to-prompt) attention heatmaps at $(t^{\ast}{=}2,\texttt{single\_1})$ , shown alongside the Grounded-SAM segmentation of the generated image. $\mathbf{A}_{N2R}$ concentrates on the actual subject region and aligns with the segmentation mask; $\mathbf{A}_{N2P}$ is more diffuse and frequently leaks outside the subject.

Figure 7. Overall pipeline of dataset curation.

	Single subject (90 prompts)					Multi subject (210 prompts)					Overall $\uparrow$
Method	DPG $\uparrow$	ID $\uparrow$	IP $\uparrow$	AES $\uparrow$	AVG $\uparrow$	DPG $\uparrow$	ID $\uparrow$	IP $\uparrow$	AES $\uparrow$	AVG $\uparrow$	Overall $\uparrow$
MS-Diffusion ( Wang et al., 2024b )	96.89	6.52	55.71	59.63	54.69	87.21	3.77	46.21	55.91	48.28	51.49
MIP-Adapter ( Huang et al., 2025 )	87.56	39.59	71.97	52.12	62.81	84.56	24.58	57.00	51.81	54.49	58.65
CRAFT (Ours)	96.81	84.22	84.23	61.24	81.62	88.71	61.16	77.25	58.16	71.32	76.47

Appendix

Table 6. XVerseBench results for the encoder-adapter baselines, with CRAFT included as a reference. Numbers for the encoder-adapter rows are from XVerse ( Chen et al., 2025 ) . Best/second-best in bold / underlined .

	Single subject (90 prompts)					Multi subject (210 prompts)					Overall $\uparrow$
Method	DPG $\uparrow$	ID $\uparrow$	IP $\uparrow$	AES $\uparrow$	AVG $\uparrow$	DPG $\uparrow$	ID $\uparrow$	IP $\uparrow$	AES $\uparrow$	AVG $\uparrow$	Overall $\uparrow$
UNO ( Wu et al., 2025d )	89.65	47.91	80.40	55.90	68.47	85.28	31.82	67.00	54.24	59.59	64.03
UNO + CRAFT (Ours)	85.49	74.43	84.90	61.60	76.61	86.37	49.60	74.14	58.04	67.04	71.83
UMO ( Cheng et al., 2025 )	86.75	77.36	76.99	61.71	75.70	87.18	58.24	60.52	58.74	66.17	70.94
UMO + CRAFT (Ours)	85.92	86.26	85.51	61.32	79.75	86.45	62.36	70.10	59.05	69.49	74.62

Appendix

Table 7. Per-metric breakdown of Table 3 : CRAFT applied to UNO ( Wu et al., 2025d ) and UMO ( Cheng et al., 2025 ) on XVerseBench. Best in each column in bold .

Closed-source proprietary
	Single		Multiple			Scene
Method	Char. $\uparrow$	Obj. $\uparrow$	Char. $\uparrow$	Obj. $\uparrow$	C. $+$ O. $\uparrow$	Char. $\uparrow$	Obj. $\uparrow$	C. $+$ O. $\uparrow$	AVG $\uparrow$
GPT-4o	8.96	8.91	8.90	8.95	8.81	8.92	8.40	8.44	8.78
Gemini-2.5-Flash-Image	8.79	9.12	8.27	8.60	7.71	7.63	7.65	6.81	8.07
Open-source
FLUX.2-klein ( Labs, 2025 ) (base)	8.92	9.20	8.56	8.75	8.52	8.51	8.24	8.25	8.62

Appendix

Table 8. Quantitative comparison on OmniContext ( Wu et al., 2025b ) . “Char.

+

Obj.” indicates Character

+

Object. Methods are partitioned into closed-source proprietary and open-source systems following Scone ( Wang et al., 2025 ) ; numbers for prior methods are from Scone ( Wang et al., 2025 ) and OmniGen2 ( Wu et al., 2025b ) , while CRAFT and the FLUX.2-klein backbone are evaluated under the same protocol. The FLUX.2-klein backbone is shown as a reference and is not included in the ranking. Best/second-best within each group in bold / underlined .

	Single subject					Multi subject
#Data	DPG $\uparrow$	ID $\uparrow$	IP $\uparrow$	AES $\uparrow$	AVG $\uparrow$	DPG $\uparrow$	ID $\uparrow$	IP $\uparrow$	AES $\uparrow$	AVG $\uparrow$	Overall $\uparrow$	$\Delta$
$20{,}000$	$97.78$	$84.09$	$84.09$	$61.34$	$81.83$	$87.85$	$62.73$	$77.23$	$58.52$	$71.58$	$76.70$	$+0.23$
$10{,}000$	96.81	84.22	84.23	61.24	81.62	88.71	61.16	77.25	58.16	71.32	76.47	—
$5{,}000$	96.94	84.78	79.90	60.95	80.64	88.15	61.43	77.40	57.81	71.20	75.92	$-0.55$
$2{,}000$	96.94	84.44	80.73	60.62	80.68	88.37	60.80	76.72	57.70	70.90	75.79	$-0.68$
$1{,}000$	96.11	82.10	81.76	61.02	80.25	86.45	59.27	76.45	59.18	70.34	75.30	$-1.18$

Appendix

Table 9. Effect of training-set size on XVerseBench. The recipe is retrained on uniformly-sampled subsets of the

10{,}000

-instance training dataset (Section B ); all other hyperparameters are unchanged.

\Delta

is the change in Overall relative to the full

10{,}000

-instance baseline used in the main paper. The

10{,}000

row ( bold ) is the configuration used in the main paper. The

20{,}000

row extends the corpus with the same recipe (a superset of the

10{,}000

set) and is shown for reference.

Method	CLIP-I $\uparrow$	CLIP-T $\uparrow$	DINO $\uparrow$
DreamBooth ( Ruiz et al., 2023 )	80.30	30.52	66.81
BLIP-Diffusion ( Li et al., 2023 )	80.47	30.24	69.82
SSR-Encoder ( Zhang et al., 2024 )	82.10	30.79	61.22
MS-Diffusion ( Wang et al., 2024b )	80.82	31.05	70.32
UNO ( Wu et al., 2025d )	83.50	30.41	75.97
DreamO ( Mou et al., 2025 )	83.35	30.61	76.03

Appendix

Table 10. Quantitative comparison on the single-subject DreamBench ( Ruiz et al., 2023 ) benchmark (

30

subjects,

25

prompts each). All scores are computed by the official dreambench_plus evaluator. Best in each column in bold .

Figure 8. Example interface used in the user study. Participants selected the best-performing method among four candidates for each question.

	Single subject					Multi subject
Variant	DPG $\uparrow$	ID $\uparrow$	IP $\uparrow$	AES $\uparrow$	AVG $\uparrow$	DPG $\uparrow$	ID $\uparrow$	IP $\uparrow$	AES $\uparrow$	AVG $\uparrow$	Overall $\uparrow$	$\Delta$	MR $\downarrow$
Baseline ( $w_{\text{id}}=1.0$ , $w_{\text{c}}=1.0$ , $w_{\text{nr}}=w_{\text{tr}}=0.5$ )	96.81	84.22	84.23	61.24	81.62	88.71	61.16	77.25	58.16	71.32	76.47	—	2.62
$w_{\text{nr}}=w_{\text{tr}}=1.0$ ( $2\times$ )	97.22	84.68	83.86	60.62	81.59	88.21	61.47	77.08	58.13	71.22	76.40	$-0.07$	3.12
$w_{\text{c}}=0.5$ ( $0.5\times$ )	97.78	82.20	81.43	60.82	80.56	88.67	59.78	76.95	58.40	70.95	75.76	$-0.71$	3.50
$w_{\text{c}}=2.0$ ( $2\times$ )	97.33	83.83	83.21	60.95	81.33	85.98	60.51	76.25	57.62	70.09	75.71	$-0.76$	4.25
$w_{\text{id}}=0.5$ ( $0.5\times$ )	97.50	81.33	83.51	61.90	81.06	87.50	57.61	75.97	59.21	70.07	75.57	$-0.90$	3.62

Appendix

Table 11. Reward-weight sensitivity on XVerseBench. Each row perturbs a single weight by

0.5\times

2\times

relative to the main configuration (

w_{\text{nr}}=w_{\text{tr}}=0.5

w_{\text{c}}=1.0

w_{\text{id}}=1.0

); all other weights and training hyperparameters are unchanged.

\Delta

is the change in Overall vs. the baseline row. “MR” is the mean rank across the eight raw per-metric columns (Single/Multi

\times

DPG, ID, IP, AES); lower is more balanced. Bold marks the strict best Overall and the strict lowest mean rank; the baseline row is shown in italics.

Variant	Single AVG	Multi AVG	Overall ( $\Delta$ )
Baseline ( $\sigma{=}2.0$ , $k{=}0.5$ )	$81.62$	$71.32$	$76.47$
$\sigma{=}1.0$ ( $0.5\times$ )	$81.19$	$71.03$	$76.11$ ( $-0.36$ )
$\sigma{=}4.0$ ( $2\times$ )	$80.72$	$71.05$	$75.88$ ( $-0.59$ )
mask $k{=}0.3$	$80.69$	$71.30$	$76.00$ ( $-0.47$ )
mask $k{=}0.7$	$81.12$	$71.13$	$76.12$ ( $-0.34$ )

Appendix

Table 12. Sensitivity to non-weight design choices on XVerseBench: Gaussian smoothing

\sigma

and mask top-

k

threshold. Each row re-trains the main recipe with a single knob changed. Overall stays within

0.59

of the baseline.

LoRA rank	Single AVG	Multi AVG	Overall ( $\Delta$ )
$64$ (default)	$81.85$	$71.30$	$76.57$
$128$ ( $2\times$ )	$81.41$	$70.97$	$76.19$ ( $-0.38$ )

Appendix

Table 13. Sensitivity to LoRA rank on XVerseBench. Doubling the rank (

64{\to}128

) leaves Overall nearly unchanged.

Subject-Driven Image Generation

Subject-driven image generation uses visual references to create new images that preserve a chosen subject while following prompts.

See all 60 papers

CopyCat: Improving Fine-Grained Subject Consistency in Subject-to-Image Models within Seconds
Aug 1, 2026Peng Zheng, Ruiqi Liu, Rui Ma +1Diffusion Transformer Subject-Driven Image Generation
School of Artificial Intelligence, Jilin University · 2Shanghai Innovation Institute · 4Engineering Research Center of Knowledge-Driven Human-Machine Intelligence, MOE, China +1
MultiCompose: Multi-Concept Personalized Composition with Per-Subject Attribute Binding
Aug 4, 2026Ruirui Zhang, Zhengkai Zhao, Pan GaoImage Generation Compositional T2I Generation
Nanjing University of Aeronautics and Astronautics College of Computer Science and Technology Nanjing, Jiangsu, China · Nanjing University of Aeronautics and Astronautics College of Artificial Intelligence Nanjing, Jiangsu, China
FlowFixer: Towards Detail-Preserving Subject-Driven Generation
Feb 24, 2026Jinyoung Jun, Won-Dong Jang, Wenbin Ouyang +2Subject-Driven Image Generation Image Generation
Amazon · Korea University

Aug 1, 2026J/K move · Enter open · S save

CopyCat: Improving Fine-Grained Subject Consistency in Subject-to-Image Models within Seconds

Peng Zheng, Ruiqi Liu, Rui Ma, Zuxuan Wu

School of Artificial Intelligence, Jilin University · 2Shanghai Innovation Institute · 4Engineering Research Center of Knowledge-Driven Human-Machine Intelligence, MOE, China+1

Diffusion Transformer Subject-Driven Image Generation Image Generation Diffusion Model Fine-Tuning

Open paper AskPDF arXiv

Recent subject-to-image models have achieved impressive progress in personalized image generation, yet they still struggle to preserve fine-grained subject-specific details. A major reason is the lack of high-quality fine-grained identity supervision: real paired data are expensive to collect, while synthesized training pairs often preserve only coarse subject appearance and fail to capture subtle subject-specific details. In this work, we propose CopyCat, a lightweight model-refinement framework that improves fine-grained subject consistency within only a few seconds. CopyCat performs a one-time refinement of a pretrained subject-to-image model by attaching a lightweight Fine-grained Consistency LoRA (FCLoRA) and optimizing it using a single proxy image, which is used as both the conditioning image and the reconstruction target. This exact self-reconstruction objective substantially simplifies the optimization task, enabling effective fine-grained refinement within only a few seconds. The refinement is performed only once; the resulting model can be directly applied to diverse unseen reference subjects and prompts without further subject-specific optimization. We further revisit subject-to-image LoRA training in double-stream diffusion transformers and find that adapting only the visual stream consistently improves subject consistency. Extensive experiments on DreamBench and XVerseBench demonstrate consistent improvements in fine-grained subject consistency across representative subject-to-image models under both single- and multi-subject settings.

Abstract

Subject-Driven Image Generation

Explore similar work

CopyCat: Improving Fine-Grained Subject Consistency in Subject-to-Image Models within Seconds

MultiCompose: Multi-Concept Personalized Composition with Per-Subject Attribute Binding

FlowFixer: Towards Detail-Preserving Subject-Driven Generation