Instruction-based image editing has achieved remarkable success in semantic alignment, yet state-of-the-art models frequently fail to render physically plausible results when editing involves complex causal dynamics, such as refraction or material deformation. We attribute this limitation to the dominant paradigm that treats editing as a discrete mapping between image pairs, which provides only boundary conditions and leaves transition dynamics underspecified. To address this, we reformulate physics-aware editing as predictive physical state transitions and introduce PhysicTran38K, a large-scale video-based dataset comprising 38K transition trajectories across five physical domains, constructed via a two-stage filtering and constraint-aware annotation pipeline. Building on this supervision, we propose PhysicEdit, an end-to-end framework equipped with a textual-visual dual-thinking mechanism. It combines a frozen Qwen2.5-VL for physically grounded reasoning with learnable transition queries that provide timestep-adaptive visual guidance to a diffusion backbone. Experiments show that PhysicEdit improves over Qwen-Image-Edit by 5.9% in physical realism and 10.1% in knowledge-grounded editing, setting a new state-of-the-art for open-source methods, while remaining competitive with leading proprietary models.
Figures & tables
Figure 1: Bridging semantic alignment and physical plausibility. (Top) Despite high semantic fidelity, existing editing models frequently violate physical principles. (Bottom) Traditional image editing treats editing as a black box, learning a discrete mapping with underspecified constraints. Our approach reformulates editing as a Physical State Transition , leveraging continuous dynamics to constrain the state transition space from unreal hallucinations to physically valid trajectories .
Figure 2: Overview of the PhysicTran38K construction pipeline. Starting from hierarchical physics categories, we synthesize videos using Wan2.2-T2V-A14B, filtered by ViPE with an adaptive strategy to preserve high-dynamic transitions. Candidate videos conduct principle-driven verification by GPT-5-mini, adhering to a rigorous retention rule. Finally, Qwen2.5-VL-7B performs constraint-aware annotation, generating instructions and structured reasoning while incorporating verification results to prevent hallucinations.
Figure 3: Overview of the PhysicEdit framework. (a) Training: We distill physical transition priors from video data into learnable transition queries. These queries are supervised by complementary visual features extracted from intermediate keyframes. (b) Inference: PhysicEdit follows a sequential workflow. The frozen MLLM first generates physically-grounded reasoning, which is then concatenated with the learned transition queries to serve as the condition for the diffusion backbone.
Figure 4: Qualitative comparison on PICABench. We visualize editing results across diverse physical domains, including Optics, Mechanics, Global State, and Local State. Compared to the backbone Qwen-Image-Edit and proprietary models, PhysicEdit consistently generates more physically plausible and visually natural results, avoiding the physical inconsistencies observed in baseline methods.
Model
Mechanics
Optics
State
Overall
Qwen-Image-Edit
55.04
64.29
62.85
61.26
+ SFT
53.43
66.33
63.33
61.79
+ Physical Reasoning
57.43
64.15
64.21
62.31
+ Visual Thinking
52.38
68.05
64.05
62.41
PhysicEdit
59.79
68.19
65.10
64.86
Table 3: Ablation Study on physically-grounded reasoning and implicit visual thinking effectiveness.
Model
Mechanics
Optics
GST
LST
Overall
Only DINO
54.73
66.61
70.16
57.50
63.00
Only VAE
58.31
65.45
66.23
60.40
63.05
Hard Switching
57.39
65.43
68.84
62.08
63.52
PhysicEdit
59.79
68.19
67.67
60.52
64.86
Table 4: Ablation Study on DINO and VAE feature effectiveness.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Primary Domain
Sub-domains
Transition Types
Filtered Videos
Mechanical
3
12
10,245
Biological
5
12
10,242
Thermal
2
8
6,602
Optical
3
8
6,245
Material
3
6
5,286
Total
16
46
38,620
Appendix
Table 5: Detailed statistics of PhysicTran38K across five primary physical domains. The video counts represent the final data used for training after filtering.
Figure 5: More detailed illustrations on Mechanical, Biological and Thermal data.
Figure 6: More detailed illustrations on Optical and Material data.
Figure 7: Comparison with explicit visual thinking (ChronoEdit). ChronoEdit employs an explicit paradigm that synthesizes intermediate video frames to model the editing process. As visualized in the rollouts, this pixel-level generation suffers from severe error accumulation over long horizons, causing the final results to degenerate (e.g., background distortion or object disappearance). In contrast, by encoding dynamics into latent transition queries ( Implicit Visual Thinking ), our method avoids these synthesis errors, maintaining structural integrity and yielding precise, high-fidelity edits.
Table 11
Figure 8: More qualitative results on PICABench, showing light propogation related results.
Figure 9: More qualitative results on PICABench, showing light source effects results.
Figure 10: More qualitative results on PICABench, showing refraction results.
While instruction-based image editing, enabled by multi-modal generative models, has advanced significantly, existing benchmarks lack a comprehensive evaluation of physics-based reasoning, a critical capability for handling real-world scenarios. To address this, we introduce PhyEditBench, a benchmark designed to assess the physical understanding of editing models. Guided by a hierarchical taxonomy, we establish 4 primary classes and 12 subclasses. It comprises 238 high-quality, high-resolution, real-world instances meticulously extracted from videos to capture authentic physical dynamics, alongside 35 synthetic Anti-Physics instances. Our empirical analysis of current SOTA editing methods exposes substantial limitations in their physics-based reasoning. We further propose a training-free baseline named PhyWorld that uses test-time scaling and a latent reduction strategy. PhyWorld outperforms comparable models and suggests that the video generation process can effectively serve as a reasoning mechanism for image editing. The project page is available at https://github.com/Previsior/PhyEditBench.
Visual demonstrations provide a natural interface for specifying image transformations that are difficult to describe exhaustively with text. However, existing visual in-context learning (VICL) methods primarily focus on appearance-level relation transfer and provide limited support for physically grounded transformations, whose outcomes depend on material properties, geometry, object interactions, and environmental conditions. Given a source--target exemplar pair and a query image, physically grounded VICL requires a model to infer the demonstrated transformation, adapt its effects to the query-specific scene context, and preserve rule-irrelevant content. We introduce PhysVICL-74, comprising 74 physically grounded transformation rules and 5,240 source--target image pairs that form nearly 75K training and evaluation contexts. Its benchmark split separately evaluates novel-instance transfer and unseen-rule generalization. We further propose TransPhy, a framework that decomposes physically grounded VICL into physical-rule induction and transition-aligned rendering. TransPhy first predicts the demonstrated rule and an explicit query-specific target-state description, and then synthesizes the target image through token-wise mixture-of-experts adaptation, with expert routing guided by localized transition cues. Experiments show that TransPhy improves physical-rule adherence, query consistency, and unseen-rule generalization over existing visual in-context editing methods.
Siyi Xie, Xuanke Shi, Jinsheng Quan +4
Peking University · SenseTime Research · Zhejiang University
Image editing instructions are heterogeneous: a color swap, an object insertion, and a physical-action edit all demand different spatial coverage and different reasoning depth, yet existing reasoning-based editors apply a single fixed inference recipe to every instruction. We argue that adaptivity along both the spatial and temporal axes is the missing degree of freedom, and we present PhysEdit, an editing framework built around this principle. PhysEdit introduces two inference-time modules that compose without retraining the backbone. At its core, (1) Complexity-Adaptive Reasoning Depth (CARD) predicts edit complexity directly from the instruction and reference image and allocates the reasoning step count N_r and reasoning-token length r per sample -- turning a previously fixed inference schedule into a conditional-computation problem. CARD is supported by (2) a Spatial Reasoning Mask (SRM) that extracts an instruction-conditioned spatial prior from cross-attention to confine reasoning to regions that semantically require it. On the full 737-case ImgEdit Basic-Edit Suite, PhysEdit delivers a 1.18x wall-clock speedup (64.3s vs. 76.1s per sample) over a strong reasoning baseline while slightly improving instruction adherence (CLIP-T 0.2283 vs. 0.2266, +0.7%) and matching identity preservation within noise (CLIP-I 0.8246 vs. 0.8280). The speedup is category-dependent and reaches 1.52x on appearance-level edits, validating CARD's adaptive allocation as the principal source of efficiency gain. A 30-sample pilot with full ablations isolates the contribution of each module.