Direct prompt-based editing often fails on complex transformations because vague and subjective prompts often require nuanced understanding of what should be changed in the image. Our core intuition is that leveraging compositional image editing tools rather than direct prompting profits from structured agent-level planning with explicit reasoning, leading to better results. This structured planning framework enables efficient offline RL post-training on quality-scored trajectories to improve performance. We present a tool-based agentic RL post-training framework that addresses this through structured planning with chain-of-thought reasoning. Our key contributions include: (1) A tool-based agentic planning methodology that combines a compositional library of orthogonal primitive transformations, structured context representation, and explicit per-step reasoning to decompose complex styling into interpretable tool sequences. (2) A synthetic data generation pipeline producing three large-scale datasets (each ∼10K trajectories) with reasoning chains, plans, and quality scores, as no existing datasets provide such supervision. Our datasets and code are publicly available at the HuggingFace repository. (3) Offline RL training methods for learning planners with reasoning as our core algorithmic contributions, which consistently improve over the Edit-Only baseline in visual quality and instruction following. (4) Comprehensive evaluation across 4B and 8B parameter Qwen3-VL models showing that our methods outperform other baselines in the majority of compositional tasks, validated by human evaluations.
Figure 6: RW and SW Strengths Across Datasets: Examples where reward-aware weighting methods excel. Each row shows (left to right): Original, Baseline (B), Edit-Only (E), Standard (S), R (R), DPO (D), RW , SW , and GPT-4o Planner. Red box highlights RW and SW columns. Row 1 (Text-8B, Complex): Gallery art scene with bokeh lens effect and character depth— SW handles complex multi-element transformations. Row 2 (Vision-4B, Regular): Market scene with visual grounding— RW excels with continuous reward weighting. Row 3 (Vision-4B, Regular): Prehistoric stone age transformation— RW achieves strong temporal consistency. Row 4 (Text-4B, Regular): Temple shrine transformation— SW handles architectural and cultural elements effectively. Row 5 (Vision-8B, Complex): Canyon landscape with lens effects— SW produces superior depth and lighting. Reward-aware methods consistently outperform filtering ( R ) and preference learning ( DPO ) by leveraging continuous quality signals while preserving data diversity.
Figure 7: DPO Strengths Across Datasets: Examples where DPO outperforms other methods through fine-grained contrastive learning. Each row shows (left to right): Original, Baseline (B), Edit-Only (E), Standard (S), R ( R ), RW , SW , DPO (D), and GPT-4o Planner. Red box highlights RW and SW columns for comparison. Row 1 (Text-8B, Simple): Renaissance time period transformation from futuristic dome theater to ancient era. Row 2 (Vision-4B, Simple): Holographic future architecture combined with charcoal drawing artistic medium—a complex multi-action transformation. Row 3 (Vision-4B, Regular): Classic library with paradoxical snowstorm and summer plants visible outside windows. Row 4 (Vision-4B, Complex): Cafe transformed with surrealist art movement, dreamy textures, and soft edges. Row 5 (Text-8B, Complex): Dragon scene with subtle depth enhancement via color grading. DPO ’s contrastive learning captures fine-grained preference distinctions across diverse datasets and complexity levels, consistently producing results closer to GPT-4o reference than other methods. Edit-Only (E) shows inconsistent quality without structured action planning.
Figure 8: R Strengths Across Datasets: Examples where reward-filtering performs effectively. Each row shows (left to right): Original, Baseline (B), Edit-Only (E), Standard (S), R ( R ), RW , SW , DPO (D), and GPT-4o Planner. Red box highlights RW and SW columns for comparison. Row 1 (Text-8B, Simple): Winter to summer coastal town transformation—multi-action sequence changing location, architecture, time of day, and weather. Row 2 (Text-8B, Complex): Spatial depth enhancement with atmospheric haze and cinematic lighting. Row 3 (Text-8B, Complex): Wolf scene with winter atmosphere enhancement via cool-toned color grading and soft haze. Row 4 (Vision-8B, Regular): Urban kitchen transformed to snowy winter wonderland under blue twilight. Row 5 (Vision-4B, Simple): Bridge to Asgard Bifrost with Norse architecture and pencil sketch artistic medium. R ’s filtering strategy ( ri<3.5 threshold) effectively removes low-quality data while maintaining training signal across diverse task types and complexity levels. Its computational efficiency and consistent quality make it attractive for large-scale training. Edit-Only (E) demonstrates the necessity of structured action planning.
Score Range
Quality Tier
RW Weight (at r )
DPO Usage
[4.5,5.0]
Excellent
2.0 (at r=5.0 )
Chosen
[4.0,4.5)
Good
1.5 (at r=4.5 )
Chosen
[3.5,4.0)
Medium
1.0 (at r=4.0 )
Neutral
[3.0,3.5)
Poor
0.5 (at r=3.5 )
Rejected
[0,3.0)
Very Poor
0.0
Rejected
Appendix
Table 2: Reward Thresholds and Training Usage
Statistic
Simple Dataset
Regular Dataset
Total trajectories
9,824
10,142
Avg. actions per trajectory
2.8
4.1
Avg. reward score
3.92
3.74
% High quality ( r≥4.0 )
62%
48%
Action library size
10 actions
20 actions
Train/Val/Test split
80%/10%/10%
80%/10%/10%
Appendix
Table 3: Dataset Statistics
Figure 9: Simple Dataset Example - Autumn Vineyard to Spring Tulip Field. (a) Original: Autumn vineyard scene with golden foliage, wine barrels, grape clusters, and a rustic farmhouse. (b) Edited: Transformed to spring tulip field with vibrant pink tulips, modern technology, and clear morning atmosphere. The transformation successfully replaces location (vineyard→tulip field), season (autumn→spring), and era (modern→futuristic) while maintaining compositional structure.
Figure 10: Regular Dataset Example - Scandinavian Studio to Cyberpunk Industrial. (a) Original: Modern Scandinavian studio with gray sofa, wooden table, plants, and city view through large windows. (b) Edited: Transformed to cyberpunk industrial loft with neon lighting, metallic textures, and futuristic tech elements. The transformation demonstrates constraint-based planning: dramatic neon accents added while attempting to preserve natural wood tones and organic plant colors (partial success, reward 3/5).
Property
Normal Example
Complex Example
Number of actions
3
3
Preservation constraints
0
2
Overall reward
4.0/5.0 (Good)
3.0/5.0 (Medium)
Planning difficulty
Low
High
Execution difficulty
Medium
High
Adherence to plan
5/5 (Perfect)
3/5 (Partial)
Appendix
Table 4: Comparison of Normal vs. Complex Synthesis Examples
Figure 11: Complex Dataset Example - Arctic Glacier to Desert Canyon. (a) Original: Arctic glacier scene with deep crevasse revealing turquoise meltwater, surrounded by expansive snowfields and distant snow-capped mountain peaks. Strong diagonal composition with cold blue-white color palette. (b) Edited: Transformed to dramatic desert canyon with layered sandstone and rocky canyon walls in warm ochre and amber tones. The diagonal crevasse structure is preserved but now rendered as desert canyon walls. Background mountains maintain imposing presence with reddish-brown rock faces. Depth perception and spatial relationships successfully maintained through consistent atmospheric perspective (reward 5.0/5.0).
Characteristic
Simple
Regular
Complex
Dataset Size
n=10,000
n=10,000
n=10,000
Action Library
10 actions
20 actions
30 actions
Action Types
9 THEME + 1 STYLE
10 atomic + 10 constraint
30 styling & transformation
Avg Actions/Sample
2-3
3-5
2-4
Theme Diversity
31 locations
10 interior design styles
83 diverse themes
Transformation Type
Simple, 1-2 distinct changes
Compositional, 3-5 interacting changes
Moderate, 2-4 styling changes
Appendix
Table 5: Comparison of three synthetic datasets. Normal uses atomic actions on diverse locations. Regular adds constraint/compositional actions for interior design styles. Complex expands to 30 actions and 83 diverse themes, balancing complexity with broad distribution coverage.
Property
RW
DPO
Learning signal
Absolute rewards
Relative preferences
Data usage
All data
Paired data only
Optimization
Direct likelihood
Contrastive likelihood
Implicit KL
Through weights
Through log-ratio
Sample efficiency
High
Medium
Distinction quality
Coarse-grained
Fine-grained
Appendix
Table 6: Theoretical Comparison: RW vs. DPO
Property
SL
R
RW
DPO
Uses reward signals
No
Yes
Yes
Yes
Uses all data
Yes
No (65%)
Yes
Paired only
Manual tuning required
None
Threshold
Weight fn
Beta
Computational cost
1 ×
1 ×
1 ×
2 ×
Contrastive learning
No
No
No
Yes
Data efficiency
Medium
Low
High
Medium
Appendix
Table 7: Comprehensive Comparison of Training Algorithms
Figure 12: GPT-4o image quality evaluation for Regular Dataset, Text-8B models (8 methods including GPT-4o Planner). SW achieves highest Overall score among trained models (77.86), with RW winning visual quality metrics and E showing lowest performance (71.24). We outperform GPT-4o zero-shot baseline on image quality.
Figure 13: GPT-4o image quality evaluation for Simple Dataset, Vision-4B models (8 methods including GPT-4o Planner). RW achieves highest Overall score among trained models (79.33) and dominates 5/6 metrics, demonstrating strong performance with visual grounding. We outperform GPT-4o zero-shot baseline on image quality.
Figure 14: GPT-4o image quality evaluation for Simple Dataset, Vision-8B models (6 methods including GPT-4o Planner; E/ SW not evaluated). R achieves highest Overall score among trained models (79.62) and wins 4/6 metrics, with RW winning visual quality dimensions. GPT-4o zero-shot baseline achieves highest overall score on this configuration.
Figure 15: GPT-4o image quality evaluation for Regular Dataset, Text-4B models (8 methods including GPT-4o Planner). SW achieves highest Overall score among trained models on highly complex triple-action transformations, demonstrating superior handling of extreme compositional complexity. We outperform GPT-4o zero-shot baseline on image quality.
Figure 16: GPT-4o image quality evaluation for Complex Dataset, Text-8B models (8 methods including GPT-4o Planner). R achieves highest Overall score among trained models, showing that larger models enable simpler filtering strategies to handle highest complexity effectively. We outperform GPT-4o zero-shot baseline on image quality.
Figure 17: GPT-4o image quality evaluation for Regular Dataset, Vision-4B models (8 methods including GPT-4o Planner). DPO achieves highest Overall score among trained models, showing that visual grounding enables preference-based learning to excel on highest complexity. We outperform GPT-4o zero-shot baseline on image quality.
Figure 18: GPT-4o image quality evaluation for Regular Dataset, Vision-8B models (8 methods including GPT-4o Planner). DPO achieves the highest Overall score among trained models (85.41), demonstrating that preference learning reaches peak effectiveness at maximum scale with visual grounding. We outperform GPT-4o zero-shot baseline on image quality.
Configuration
B
E
S
R
RW
SW
D
Regular Dataset
Text-4B
76.03
71.49
75.03
77.12
77.18
78.77
74.88
Text-8B
74.79
71.24
74.23
77.62
77.34
77.86
75.85
Simple Dataset
Vision-4B
77.28
78.04
77.60
78.35
79.33
78.65
78.27
Vision-8B
78.07
-
78.73
79.62
78.79
-
78.98
Appendix
Table 8: Overall Scores Across All Configurations (7 Methods)
Figure 19: GPT-4o action plan quality evaluation for Simple Dataset, Vision-4B models: RW achieves highest Overall Score (82.09), outperforming SW (80.37), D (81.12), R (79.50), S (79.32), and B (82.01). RW dominates reasoning quality metrics: Reasoning Conciseness (88.50), Reasoning Completeness (77.14), and Reasoning Specificity (71.79), achieving Overall Reasoning Quality of 79.15. On action quality, B surprisingly leads Overall Action Quality (85.84), but RW ranks second (84.89) while excelling at reasoning. E shows N/A across all metrics, unable to generate action plans. These results demonstrate that RW ’s reward-weighting effectively improves both action planning coherence and reasoning quality on normal complexity tasks. GPT-4o Planner serves as performance ceiling with Overall Score 84.23, Overall Action Quality 87.15, and Overall Reasoning Quality 81.21.
Figure 20: GPT-4o action plan quality evaluation for Regular Dataset, Vision-4B models: SW achieves highest Overall Score (81.57), outperforming R (81.27), S (81.14), D (79.82), RW (81.10), and B (79.83). SW dominates reasoning quality: Reasoning Conciseness (88.31), Reasoning Completeness (76.92), Reasoning Specificity (72.69), achieving Overall Reasoning Quality of 79.26 (highest among all methods). SW also leads Overall Action Quality (86.39). R wins individual action metrics like Relevance (91.77) and Theme/Style Focus (78.77), while D excels at Efficiency (86.57). E shows N/A on all metrics. These results demonstrate SW ’s advantage on diverse theme distributions with broad action spaces (83 themes, 30 actions), where standardized reward weighting adapts better to compositional tasks. GPT-4o Planner serves as performance ceiling with Overall Score 83.23, Overall Action Quality 85.80, and Overall Reasoning Quality 80.57.
Figure 21: GPT-4o action plan quality evaluation for Regular Dataset, Vision-8B models: SW achieves highest Overall Score (83.08), outperforming S (82.87), R (82.58), B (82.66), RW (82.59), and D (82.57). SW dominates reasoning quality: Reasoning Conciseness (89.15), Reasoning Completeness (78.68), Reasoning Specificity (71.32), achieving Overall Reasoning Quality of 79.72 (highest). SW also leads Overall Action Quality (86.39) and wins 6/10 individual metrics including Relevance (92.25), Completeness (79.46), and Efficiency (87.60). B shows surprisingly strong performance (82.66 Overall), particularly on Reasoning Specificity (72.71). E shows N/A on all metrics. With larger models (8B) on complex datasets, SW ’s standardized weighting provides consistent advantages across both action planning and reasoning dimensions. GPT-4o Planner serves as performance ceiling with Overall Score 83.72, Overall Action Quality 86.27, and Overall Reasoning Quality 81.06.
Figure 22: Location and style transformation from desert to moss garden with pencil sketch. Left to right: Original image, Baseline result, SW result, Ground truth. Both models use 3 actions, but SW provides more detailed, contextual reasoning (shown below).
Figure 23: Complex scene transformation from European church to Angkor Wat temple with jungle and snowy winter. Left to right: Original, Baseline result, SW result, Ground truth. SW achieves the transformation with 2 actions instead of 4 by recognizing that location_setting encompasses architectural changes.
Figure 24: Contrast enhancement task for restaurant scene. Left to right: Original, Baseline result, SW result, Ground truth. Baseline refuses the task (0 actions), claiming it contradicts the photorealistic style, while SW successfully solves it with 2 actions.
Parameter
Value
LoRA Rank
16
LoRA Alpha
32
LoRA Dropout
0.05
Learning Rate
2×10−5
LR Schedule
Cosine with warmup
Warmup Steps
100
Appendix
Table 9: Training Hyperparameters
Configuration
Training Time
Peak Memory
Text-Only (per method)
1.5-2 hours
28 GB
Vision (with caching)
3-4 hours
45 GB
Vision (without caching)
9-12 hours
45 GB
Speedup from caching
3×
0×
Appendix
Table 10: Training Time and Resources
Figure 25: Web-based annotation interface used by human evaluators. Annotators rate each sample on four quality dimensions (Edit Quality, Action Plan Quality, Reasoning Quality, Overall Sample Quality) using a Pass/Partial/Fail scale. The interface displays the original image (left), edited image (right), editing goal, context extraction, action plan with per-step reasoning, and synthesized instruction. Optional comments allow annotators to provide detailed feedback on edge cases.
Rating
Count
Percentage
Pass
672
77.0%
Partial
130
14.9%
Fail
71
8.1%
Total
873
100%
Appendix
Table 11: Overall Quality Distribution Across All Samples
Dataset
Samples
Pass
Partial
Fail
Regular
1000
73.8%
15.6%
10.5%
Complex
1000
77.9%
13.4%
8.7%
Complex
1000
79.4%
15.7%
5.0%
Appendix
Table 12: Quality Distribution by Dataset Variant
Figure 26: Overall quality pass rates by annotator across three dataset variants. All annotators achieve >70% pass rates on Regular and Complex datasets, with Regular showing more variance (62.5%-87%), reflecting subjective assessment of atomic transformations. The consistent high pass rates on complex datasets validate the robustness of our synthetic data generation pipeline.
Figure 27: Distribution of agreement types across dataset variants. Exact agreement ranges from 62.8% (Regular) to 66.2% (Complex), with adjacent agreement (Pass-Partial or Partial-Fail) at 25-27%. Complete disagreement (Pass vs Fail) remains below 11% for all datasets, indicating strong consistency in fundamental quality judgments despite subjective differences on borderline cases.
Figure 28: GPT-4o validation study interface: Annotators view 7 images side-by-side (original + 6 edited versions) and select top-3 ranked methods, then rate the winner on Visual Quality, Instruction Following, and Overall Quality. Study design validates GPT-4o correlation with human judgment and identifies best-performing training methods across 279 samples.
Figure 29: Method win rates (1st place) by dataset. Left: Simple dataset shows SW leading (20.8%), followed by RW (20.8%) and D (16.7%). Center: Regular dataset shows SW winning (21.7%), followed by R (17.8%) and RW/D (16.4%). Right: Complex shows D leading (23.1%), followed by R (17.9%) and SW/S/RW ( ∼17% ). Win rate differences are small (typically 5-10 percentage points), indicating similar method performance.
Method
Wins
Pass
Partial
Fail
SW
83
59 (71.1%)
17 (20.5%)
7 (8.4%)
D
77
59 (76.6%)
11 (14.3%)
7 (9.1%)
RW
74
58 (78.4%)
11 (14.9%)
5 (6.8%)
R
69
46 (66.7%)
13 (18.8%)
10 (14.5%)
S
61
48 (78.7%)
9 (14.8%)
4 (6.6%)
B
49
41 (83.7%)
7 (14.3%)
1 (2.0%)
Appendix
Table 13: Quality Distribution for Winning Samples by Method (Combined Datasets)
Modern image editing models produce realistic results but struggle with abstract, multi step instructions (e.g., ``make this advertisement more vegetarian-friendly''). Prior agent based methods decompose such tasks but rely on handcrafted pipelines or teacher imitation, limiting flexibility and decoupling learning from actual editing outcomes. We propose an experiential framework for long-horizon image editing, where a planner generates structured atomic decompositions and an orchestrator selects tools and regions to execute each step. A vision language judge provides outcome-based rewards for instruction adherence and visual quality. The orchestrator is trained to maximize these rewards, and successful trajectories are used to refine the planner. By tightly coupling planning with reward driven execution, our approach yields more coherent and reliable edits than single-step or rule-based multistep baselines.
Anirudh Sundara Rajan, Krishna Kumar Singh, Yong Jae Lee
University of Wisconsin–Madison, WI, USA · Adobe Research, San Jose, CA, USA
Recent image editing models have achieved strong visual fidelity but often struggle with tasks requiring complex reasoning. To investigate and enhance the reasoning-grounded planning for image editing, we propose DDA-Thinker, a Thinker-centric framework designed for the independent optimization of a planning module (Thinker) over a fixed generative model (Editor). This decoupled Thinker-centric paradigm facilitates a controlled analysis of the planning module and makes its contribution under a fixed Editor easier to assess. To effectively guide this Thinker, we introduce a dual-atomic reinforcement learning framework. This framework decomposes feedback into two distinct atomic rewards implemented through verifiable checklists: a cognitive-atomic reward to directly assess the quality of the Thinker's executable plan, which serves as the actionable outcome of the Thinker's reasoning, and a visual-atomic reward to assess the final image quality. To improve checklist quality, our checklist synthesis is grounded not only in the source image and user instruction but also in a rational reference description of the ideal post-edit scene. To support this training, we further develop a two-stage data curation pipeline that first synthesizes a diverse and reasoning-focused dataset, then applies difficulty-aware refinement to curate an effective training curriculum for reinforcement learning. Extensive experiments on reasoning-driven image editing benchmarks, including RISE-Bench and KRIS-Bench, demonstrate that our approach substantially improves overall performance. Our method enables a community model to achieve results competitive with strong proprietary models, highlighting the practical potential of Thinker-centric optimization under a fixed-editor setting.
Instruction-based image editing enables natural-language control over visual modifications, yet existing models falter under Instruction-Visual Complexity (IV-Complexity), where intricate instructions meet cluttered or ambiguous scenes. We introduce RePlan (Region-aligned Planning), a plan-then-execute framework that couples a vision-language planner with a diffusion editor. The planner decomposes instructions via step-by-step reasoning and explicitly grounds them to target regions; the editor then applies changes using a training-free attention-region injection mechanism, enabling precise, parallel multi-region edits without iterative inpainting. To strengthen planning, we apply GRPO-based reinforcement learning using 1K instruction-only examples, yielding substantial gains in reasoning fidelity and format reliability. We further present IV-Edit, a benchmark focused on fine-grained grounding and knowledge-intensive edits. Across IV-Complex settings, RePlan consistently outperforms strong baselines trained on far larger datasets, improving regional precision and overall consistency. Our project page: https://replan-iv-edit.github.io
Tianyuan Qu, Lei Ke, Xiaohang Zhan +6
Tencent AI Lab, China · The Chinese University of Hong Kong, China · The Hong Kong University of Science and Technology, China