Catalyst4D: High-Fidelity 3D-to-4D Scene Editing via Dynamic Propagation
Authors: Shifeng Chen, Yihui Li, Jun Liao, Hongyu Yang, Di Huang
Organizations: State Key Laboratory of Complex and Critical Software Environment, Beihang University, China · School of Computer Science and Engineering, Beihang University, China · School of Artificial Intelligence, Beihang University, China · Zhejiang Industrial Big Data and Robot Intelligent System Key Laboratory, Hangzhou Innovation Institute, Beihang University, China
Recent advances in 3D scene editing using NeRF and 3DGS enable high-quality static scene editing. In contrast, dynamic scene editing remains challenging, as methods that directly extend 2D diffusion models to 4D often produce motion artifacts, temporal flickering, and inconsistent style propagation. We introduce Catalyst4D, a framework that transfers high-quality 3D edits to dynamic 4D Gaussian scenes while maintaining spatial and temporal coherence. At its core, Anchor-based Motion Guidance (AMG) builds a set of structurally stable and spatially representative anchors from both original and edited Gaussians. These anchors serve as robust region-level references, and their correspondences are established via optimal transport to enable consistent deformation propagation without cross-region interference or motion drift. Complementarily, Color Uncertainty-guided Appearance Refinement (CUAR) preserves temporal appearance consistency by estimating per-Gaussian color uncertainty and selectively refining regions prone to occlusion-induced artifacts. Extensive experiments demonstrate that Catalyst4D achieves temporally stable, high-fidelity dynamic scene editing and outperforms existing methods in both visual quality and motion coherence.
Figures & tables
Figure 2: Overview of Catalyst4D. Given the first-frame edited dynamic Gaussians, our (a) Anchor-based Motion Guidance establishes region-level correspondences with the original Gaussians via anchor construction and optimal transport, enabling reliable deformation transfer. Then, (b) Color Uncertainty-guided Appearance Refinement leverages first-frame warping and Gaussian color consistency to identify and correct motion-induced artifacts across time.
Figure 3: Qualitative editing results across multiple scenes: Cut-beef , Coffee-martini and Sear-steak (DyNeRF), Discussion and Trimming (MeetRoom), 3Dprinter and Torchocolate (HyperNerf). Our method successfully edits dynamic scenes while adhering to user instructions.
Figure 4: Qualitative comparison with Instruct 4D-to-4D, Instruct-4DGS and CTRL-D. Red boxes indicate magnified regions. While competing methods often cause unintended modifications to non-target regions, Catalyst4D demonstrates precise, localized editing.
Scene
Method
CLIP sim. ↑
Consistency ↑
Time ↓
Sear-steak
Ours
0.252
0.983
50 min
IN4D
0.246
0.962
2 h ∗
I4DGS
0.220
0.980
40 min
CTRL-D
0.249
0.985
55 min
Coffee-martini
Ours
0.249
0.986
50 min
IN4D
0.241
0.981
2 h ∗
Table 1: Quantitative comparison with Instruct 4D-to-4D (IN4D), Instruct-4DGS (I4DGS) and CTRL-D. Bold and underlined denote the best and second-best results, respectively. * indicates 2 GPUs are used. Our method achieves superior semantic fidelity while maintaining highly competitive temporal stability.
Figure 5: Comparison of anchor construction methods.
Figure 6: Ablation study on the Anchor-based Motion Guidance. Our method effectively avoids geometric distortions caused by incorrect motion propagation.
Figure 7: Ablation study on the Color Uncertainty-guided Appearance Refinement (CUAR). It significantly enhances visual fidelity by mitigating color artifacts.
Method
CLIP Sim. ↑
Consistency ↑
w/o AMG
0.245
0.966
w/o CUAR
0.248
0.969
Full model
0.252
0.971
Table 2: Quantitative Ablation studies on AMG and CUAR modules. The results show that AMG is critical for establishing correct motion propagation, while CUAR further enhances both semantic and temporal scores by refining appearance consistency.
Metric
IN4D
I4DGS
CTRL-D
Ours
EditScore ↑
4.034
5.618
4.326
7.375
VE-Bench ↑
0.155
0.256
0.163
1.080
Table A1 : Quantitative comparison using EditScore and VE-Bench.
Metric
w/o both
w/ AMG
w/ CUAR
w/ both
EditScore ↑
2.713
5.112
5.005
7.375
VE-Bench ↑
0.059
0.452
0.580
1.080
Table A2 : Ablation studies on AMG and CUAR modules.
Figure A1 : Qualitative comparison of localized editing. In contrast to CTRL-D, which introduces inconsistencies in non-edited regions, our method achieves more precise and localized editing by constraining dynamic Gaussians via 3D editing gradients.
Figure A2 : Comparison on HyperNeRF Dataset. The results show that Instruct-4DGS fails to correctly localize edits and CTRL-D produces blurry and temporally inconsistent results, whereas our method demonstrates significant advantages in both visual quality and temporal consistency.
Figure A3 : Effect of Sinkhorn regularization on motion transfer.
Figure A4 : Comparative results for global style transfer. Unlike the over-smoothed results of CTRL-D and the chaotic textures from CLIPGaussian, our method produces finer visual textures, better preserves the original scene’s geometry, and demonstrates a color distribution more consistent with the reference style image.
Figure A5 : Failure case (motion jitter illustration).