cs.CVMar 17, 2026
SaveMSRAMIE: Multimodal Structured Reasoning Agent for Multi-instruction Image Editing
Organizations: University Of Melbourne, Grattan Street, Parkville, Victoria 3010, Australia
Abstract
Existing instruction-based image editing models perform well with simple, single-step instructions but degrade in realistic scenarios that involve multiple, lengthy, and interdependent directives. A main cause is the scarcity of training data with complex multi-instruction annotations. However, it is costly to collect such data and retrain these models. To address this challenge, we propose MSRAMIE, a training-free agent framework built on Multimodal Large Language Model (MLLM). MSRAMIE takes existing editing models as plug-in components and handle multi-instruction tasks via structured multimodal reasoning. It orchestrates iterative interactions between an MLLM-based Instructor and an image editing Actor, introducing a novel reasoning topology that comprises the proposed Tree-of-States and Graph-of-References. During inference, complex instructions are decomposed into multiple editing steps which enable state transitions, cross-step information aggregation, and original input recall, which enables systematic exploration of the image editing space and flexible progressive output refinement. The visualizable inference topology further provides interpretable and controllable decision pathways. Experiments show that as the instruction complexity increases, MSRAMIE can improve instruction following over 15% and increases the probability of finishing all modifications in a single run over 100%, while preserving perceptual quality and maintaining visual consistency.
Figures & tables
Figure 1: Performance Degradation Under Multi-instruction. Existing models degrade when directly dealing with lengthy multi-instructions, manifested by poor instruction following and identity preservation that worsen with increasing complexity (1st row). MSRAMIE mitigates these issues (2nd row). Colored text under the input image stands for different parts of the editing multi-instructions, corresponding to color blocks under each image. A red cross stands for a failed instruction.
Figure 2: MSRAMIE Architecture. Iterative interactions between the Instructor and Actor progressively construct an inference topology, where each state corresponds to one round of interaction. Transition and reference links among states form a Tree-of-States and a Graph-of-References , enabling multiple decision paths, or Chain-of-States , containing progressively refined candidate solutions in the image editing space. All states retain access to the root state for the original inputs during their creation.
Figure 3: Image Editing Space Exploration by Structured Multimodal Reasoning. MSRAMIE creates a visualizable reasoning topology. States on a decision path dynamically focus on different parts of the original lengthy multi-instruction, with referencing to nearby states to prevent duplicated exploration. Inference topology expands in depth-first style, providing a transparent and controllable editing process.
Figure 4: Instructor - Actor Interaction Process (left) and Instructor Submodules Process (right). Four Instructor submodules coordinate with the Actor to create a new state. One may refer to Figure 2 for the meaning of state colors.
Figure 5: A Sample of Complex-Edit. Each non-synthetic image is accompanied with lengthy multi-instructions of 7 complexity levels. Higher complexity leads to more and lengthier editing requirements.
| VQAScore (↑) | |||||||||
| Models | QwenImageEdit | FluxKontext | FluxKlein | ||||||
| Complexity | Direct | MSRAMIE | Improvement | Direct | MSRAMIE | Improvement | Direct | MSRAMIE | Improvement |
| 1 | 0.8088 | 0.8760 | 8.31% | 0.7884 | 0.8659 | 9.84% | 0.8898 | 0.9315 | 4.68% |
| 2 | 0.8041 | 0.8947 | 11.27% | 0.7779 | 0.8692 | 11.74% | 0.9134 | 0.9441 | 3.36% |
| 3 | 0.7795 | 0.8854 | 13.59% | 0.7667 | 0.8705 | 13.54% | 0.9134 | 0.9543 | 4.49% |
| 4 | 0.7530 | 0.8768 | 16.44% | 0.7331 | 0.8599 | 17.30% | 0.8934 | 0.9383 | 5.03% |
Table 1: Performance enhancement by MSRAMIE. Baseline models exhibit consistent performance degradation as instruction complexity increases. In contrast, MSRAMIE delivers progressively greater IF (VQAScore & CIF) improvements under higher instruction complexity, with enhanced IP (CLIP-I) and largely preserved PQ (FID). This demonstrates its effectiveness in challenging multi-instruction scenarios.
Figure 6: Direct Editing vs. MSRAMIE-augmented Editing. Colored blocks beneath each image denote individual instruction items, and a red cross indicates failure to satisfy the corresponding item. Regions where visual content of original images is not preserved in Direct Editing but retained by MSRAMIE are highlighted with red boxes.
| Metrics | Direct | Resampling-Only | CoS-Only | ToS-Only | MSRAMIE |
|---|---|---|---|---|---|
| VQAScore(↑) | 0.7623 | 0.7917 | 0.8115 | 0.8666 | 0.8819 |
| Improvement | - | 3.85% | 6.50% | 13.83% | 15.89% |
| CIF(↑) | 0.3497 | 0.4036 | 0.5056 | 0.5895 | 0.6008 |
| Improvement | - | 17.45% | 57.52% | 90.19% | 94.82% |
Table 2: Instruction Following Ablation Test. Results aggregated across all instruction complexities and all samples, with improvements reported relative to direct editing.
| Metrics | Direct | Resampling-Only | Direct | CoS-Only | Direct | ToS-Only | Direct | MSRAMIE |
|---|---|---|---|---|---|---|---|---|
| CLIP-I(↑) | 0.9370 | 0.9388 | 0.9355 | 0.9354 | 0.9333 | 0.9365 | 0.9338 | 0.9376 |
| Improvement | - | 0.20% | - | 0.00% | - | 0.35% | - | 0.41% |
| FID(↓) | 91.27 | 90.89 | 107.54 | 110.38 | 108.24 | 108.81 | 107.91 | 107.78 |
| Improvement | - | 0.42% | - | -2.64% | - | -0.52% | - | 0.12% |
Table 3: Identity Preservation & Perceptual Quality Ablation Test. Because of the trade-off among IF, IP, and PQ, IF is controlled in the IP/PQ ablation by selecting only samples from direct editing and MSRAMIE with identical VQAScore. Enhancement is then quantified by the CLIP-I percentage increase and the FID percentage decrease.
Figure A1 : Inference Topology Size Across Complexities.
| Topology Size Statistics | ||||||
|---|---|---|---|---|---|---|
| Complexity | QwenImageEdit | FluxKontext | FluxKlein | |||
| Average Size | Std Err | Average Size | Std Err | Average Size | Std Err | |
| 1 | 2.39 | 1.09 | 2.17 | 1.05 | 2.12 | 1.02 |
| 2 | 3.84 | 1.53 | 3.65 | 1.49 | 3.69 | 1.59 |
| 3 | 4.76 | 2.04 | 4.70 | 2.09 | 4.49 | 2.00 |
| 4 | 6.51 | 2.50 | 6.24 | 2.48 | 6.32 | 2.38 |
Table A1 : Average Topology Size vs . Complexity & Linear Model Fitting.
Figure B1 : Performance vs . Inference Steps. (1) As complexity increases, all the four metrics degrades. (2) On the same level of complexity, as the number of carried steps increases, Instruction Following metrics are greatly improved while Identity Preservation and Perceptual Quality metrics degrade, reflecting a trade-off among these three dimensions.
Figure B2 : Performance vs . Inference Depth. (1) As complexity increases, all the four metrics degrades. (2) As the inference depth increases, Instruction Following is improved, Identity Preservation and Perceptual Quality degrades because more edited contents deviate the result further from the original image, and repeated image encoding and decoding lead to the loss of details.
| VQAScore( ) | |||||
|---|---|---|---|---|---|
| Complexity | Direct | Resampling-Only | CoS-Only | ToS-Only | MSRAMIE |
| 1 | 0.8088 | 0.8514 | 0.8396 | 0.8836 | 0.8760 |
| 2 | 0.8041 | 0.8283 | 0.8449 | 0.8750 | 0.8947 |
| 3 | 0.7795 | 0.8037 | 0.8280 | 0.8738 | 0.8854 |
| 4 | 0.7530 | 0.7794 | 0.8000 | 0.8674 | 0.8768 |
| 5 | 0.7412 | 0.7706 | 0.8183 | 0.8578 | 0.8870 |
Table C1 : VQAScore Ablation Test Results.
| CIFScore( ) | |||||
|---|---|---|---|---|---|
| Complexity | Direct | Resampling-Only | CoS-Only | ToS-Only | MSRAMIE |
| 1 | 0.6064 | 0.6930 | 0.6855 | 0.7382 | 0.7307 |
| 2 | 0.5292 | 0.5819 | 0.6554 | 0.7100 | 0.7307 |
| 3 | 0.4011 | 0.4407 | 0.5687 | 0.6441 | 0.6478 |
| 4 | 0.3032 | 0.3503 | 0.4769 | 0.5857 | 0.6008 |
| 5 | 0.2241 | 0.2976 | 0.4444 | 0.5141 | 0.5405 |
Table C2 : CIF Ablation Test Results.
| CLIP-I( ) | ||||||||
|---|---|---|---|---|---|---|---|---|
| Complexity | Direct | Resampling-Only | Improvement | P-value | Direct | CoS-Only | Improvement | P-value |
| 1 | 0.9542 | 0.9562 | 0.20% | 2.62E-09 | 0.9524 | 0.9536 | 0.12% | 8.15E-02 |
| 2 | 0.9441 | 0.9463 | 0.24% | 3.02E-09 | 0.9420 | 0.9421 | 0.02% | 8.63E-01 |
| 3 | 0.9389 | 0.9409 | 0.22% | 2.24E-07 | 0.9381 | 0.9362 | -0.20% | 6.16E-02 |
| 4 | 0.9324 | 0.9341 | 0.18% | 3.90E-04 | 0.9317 | 0.9299 | -0.19% | 1.56E-01 |
| 5 | 0.9318 | 0.9336 | 0.19% | 1.44E-07 | 0.9261 | 0.9266 | 0.06% | 7.12E-01 |
Table C3 : CLIP-I Ablation Test Results.
| FID( ) | ||||||
|---|---|---|---|---|---|---|
| Complexity | Direct | Resampling-Only | Improvement | Direct | CoS-Only | Improvement |
| 1 | 72.56 | 72.33 | 0.32% | 76.86 | 76.71 | 0.20% |
| 2 | 80.09 | 80.01 | 0.10% | 88.37 | 89.92 | -1.75% |
| 3 | 88.58 | 87.57 | 1.14% | 100.31 | 105.25 | -4.92% |
| 4 | 95.34 | 96.12 | -0.82% | 127.35 | 132.21 | -3.82% |
| 5 | 99.51 | 97.09 | 2.43% | 132.91 | 135.53 | -1.97% |
Table C4 : FID Ablation Test Results. P-value is not computed as for CLIP-I, because FID is not computed image-wisely but between the original and edited image sets.
| Modules | Hyperparams | Value |
| General | MaxSteps | 2*(C+1) |
| MLLM | Qwen3VL | |
| MaxNTry | 3 | |
| Instructor | InstructionVolume | 2 |
| Actor | ImageEditingPipeline | AnyModel |
| SearchRange | 2 |
Table D1 : Quantitative Study Hyperparameter Settings. C stands for instruction complexity, which ranges from 1 to 7.
| Modules | Hyperparams | Resampling-Only | CoS-Only |
| General | MaxSteps | 2*(C+1) | 2*(C+1) |
| MLLM | - | Qwen3VL | |
| MaxNTry | - | 3 | |
| Instructor | InstructionVolume | - | 2 |
| Actor | ImageEditingPipeline | QwenImageEdit | QwenImageEdit |
| SearchRange | 0 | 0 |
Table D2 : Ablation Study Hyperparameter Settings. C stands for instruction complexity, which ranges from 1 to 7. Key hyperparameters that differentiate involved methods are colored.
Figure F1 : Visualizable Inference Topology. Task decomposition constructs diverse decision paths that lead to different editing results. Visualization of the inference topology show this process in a transparent way, enabling information check of every state.
Figure F2 : Performance Degradation Across Complexities & Enhancement by MSRAMIE. Colored blocks beneath each image denote individual instruction items, and a red cross indicates failure to satisfy the corresponding item. Regions where visual content of original images is not preserved in Direct Editing but retained by MSRAMIE are highlighted with red boxes.