Generative World Models Enable Predictive Control of Laser Melt Pool Dynamics

Yiyang Yan; Markus Bambach; Mohamadreza Afrasiabi

cs.LGOct 5, 2026

Generative World Models Enable Predictive Control of Laser Melt Pool Dynamics

Authors: Yiyang Yan, Markus Bambach, Mohamadreza Afrasiabi

Organizations: Advanced Manufacturing Lab, ETH Zürich, Leonhardstrasse 21, Zürich 8092, Switzerland

Abstract

World models, which learn how environments respond to actions, are emerging as a powerful paradigm for planning through imagined futures, transforming decision-making across games, robotics and autonomous driving. Bringing this capability to manufacturing could enable process decisions on timescales inaccessible to high-fidelity simulation. Here we introduce a generative world model for localized highly dynamic laser melt pool that predicts evolution from histories of temperature and phase morphology under candidate actions. Its generative latent dynamics capture the effects of unresolved melt flow, enabling more accurate recursive rollouts than deterministic regressors under transient laser inputs. Because the learned dynamics are differentiable, the model can serve directly as a predictive control plant. Gradients through imagined futures optimize laser schedules that regulate melt-pool depth over previously unseen geometry, path, initialization. We further distil this optimization into an amortized policy that produces control actions in a single forward pass, providing a proof of concept for real deployment on machines.

Figures & tables

Fig. 1: Melt-pool WM with generative latent dynamics. Top: the autoencoder compresses the partially observed state $\mathbf{X}_{t}=(\mathbf{T}_{t},\boldsymbol{\alpha}_{t})$ , without the full CFD thermal-fluid state, into a discrete code $\mathbf{z}^{\mathrm{q}}_{t}$ . Two decoder branches reconstruct the smooth $\hat{\mathbf{T}}_{t}$ and the sharp $\hat{\boldsymbol{\alpha}}_{t}$ from the shared latent code, and the liquid fraction $\hat{\boldsymbol{\mu}}_{t}$ is computed from the reconstructed fields. Bottom: the transition $p_{\theta}(\mathbf{z}^{\mathrm{q}}_{t+1}\mid\mathbf{z}^{\mathrm{q}}_{t-h:t},\mathbf{c}_{t})$ : diffusion-transformer (DiT) blocks, conditioned on the control parameters $\mathbf{c}_{t}$ and scalar flow-matching time $\tau$ through adaptive layer normalization (adaLN), regress the flow-matching velocity $\mathbf{v}_{\tau}$ , which an Euler sampler integrates into a latent residual that is added to $\mathbf{z}^{\mathrm{q}}_{t}$ and re-quantized.

Fig. 2: World-model fidelity, stability and generalization. All panels show the base WM, pretrained on constant schedules, on the constant-parameter test cases. (a) Recursive rollout on a representative scan: WM versus CFD temperature, rendered on the phase indicator $\boldsymbol{\alpha}$ (material versus air), at four rollout steps. (b) Final-frame WM versus CFD for three cases selected to span high, intermediate and low final penetration, shown as the three-dimensional melt pool, top view ( $x$ - $y$ ) and depth cross-sections ( $y$ - $z$ and $x$ - $z$ ). The computational domain spans $1~\mathrm{mm}\times 1~\mathrm{mm}\times 0.32~\mathrm{mm}$ . (c) Temperature and phase-indicator relative $L_{2}$ error and melt-pool intersection over union versus rollout time, aggregated across cases. Lines show the mean across cases and bands show the interquartile range across cases. (d) Cumulative distribution of melt-pool depth error over all cases, with dotted lines marking one, two and three voxels ( $\mathrm{d}x=\mathrm{d}y=\mathrm{d}z=10~\mu\mathrm{m}$ ).

Fig. 3: A compact discrete representation at small reconstruction cost, and generative dynamics that support prediction under transient laser inputs. (a) Reconstruction of a held-out constant-parameter frame ( $t=1.7~\mathrm{ms}$ ), chosen at the median melt-pool-field error of the FSQ autoencoder: CFD temperature with the melt-pool boundary ( $\boldsymbol{\alpha}\boldsymbol{\mu}=0.5$ ) outlined, and absolute temperature error $|\mathbf{T}-\mathbf{T}_{\mathrm{CFD}}|$ of the FSQ autoencoder and of a continuous bottleneck of identical architecture. IoU, melt-pool intersection over union with CFD. (b) Reconstruction error of the continuous autoencoder, the FSQ autoencoder for ablation and the FSQ autoencoder of the WM (WM), relative to the rollout error of the base WM (100%), on the $88$ transient test cases: median relative $L_{2}$ error of temperature and phase indicator, and melt-pool depth error (depth CRPS for the rollout). FSQ denotes the pretrained autoencoder and FSQ (WM) the same autoencoder fine-tuned on time-varying frames and used by all transition models (Supplementary Tables S5 and S8). (c-g) Transition-model evaluation on all $88$ transient test cases. (h-l) As c-g , on the $19$ fast random-walk cases. (c,h) Temperature, (d,i) phase-indicator and (e,j) melt-pool-field continuous ranked probability score (CRPS) over the $4~\mathrm{ms}$ autoregressive rollout. (f,k) Melt-pool depth and volume CRPS pooled over all laser-on steps, normalized in both rows by the mean CFD value over the laser-on steps of all $88$ transient cases. (g,l) Laser-on depth CRPS, pooled as in f,k , after fine-tuning on $n=10$ , $20$ , $50$ , $75$ or $100$ time-varying cases. Horizontal lines show the reference models. Base DiT is the flow-matching transition pretrained on constant schedules ( $360$ training and $40$ validation cases), and Fine Tuned is its early-stopped adaptation to $100$ time-varying cases, used by both controllers. ViT [ 38 ] and Transolver [ 39 ] are deterministic baselines trained on the same constant-schedule data. Pure Transient is the flow-matching transition trained from scratch on all $800$ time-varying cases, including the fine-tuning and validation cases. Generative models use eight rollout samples per case, whereas for deterministic models CRPS reduces to the mean absolute error. Curves in c-e and h-j show case means, and bands and error bars in c-l indicate $95\%$ bootstrap confidence intervals over cases.

Fig. 4: Two controllers operating on the same latent WM. (a) At each control step, the gradient-based receding-horizon controller encodes the current observation into the latent state $\mathbf{z}^{\mathrm{q}}_{t}$ and rolls the frozen WM forward under a candidate control sequence $\mathbf{C}_{t}$ . The predicted latent state is decoded to evaluate the melt-pool depth tracking objective against the target $d^{*}$ , and gradients are propagated through the rollout to update the control sequence. (b) The latent policy is trained within the same frozen WM using SHAC. The actor $\pi_{\psi}$ generates power and speed commands from the latent state, while imagined rollouts provide the objective and reward used to update the actor and critic $V_{\phi}$ . After training, only the actor is required for deployment, mapping the current latent state directly to the control command in a single forward pass.

Fig. 5: World-model control and CFD re-validation in a representative overhang case. (a) Melt-pool depth along the normalized scan path for the unoptimized scan, gradient receding-horizon controller (MPC), and amortized latent policy (RL), together with the local support thickness $\delta$ and target depth $d_{\mathrm{target}}=50~\mu\mathrm{m}$ . Solid curves show CFD re-simulations and dashed curves the corresponding world-model rollouts. The unoptimized scan progresses toward full penetration, whereas both controllers maintain the melt-pool depth near the target across the ten tracks. (b) Laser-power and scan-speed schedules selected by MPC and RL, with the unoptimized settings shown for reference. (c) Three-dimensional temperature fields for the common initial state and the final CFD states obtained with MPC, RL, and the unoptimized scan. Both controlled cases suppress the large high-temperature region observed without control.

Fig. 6: Generalization and computational benchmark across unseen overhang cases. (a,b) Distributions of mean and peak melt-pool depth for the unoptimized scan, MPC and RL across the 30 cases. World-model predictions (open) are compared with CFD re-simulations (filled) to assess generalization under control-induced trajectories. Dashed lines indicate the

50~\mu\mathrm{m}

target depth in (a) and the full-penetration threshold in (b) . Both controllers bring the mean depth toward the target and suppress the large peak depths observed in the unoptimized scans. The WM under-predicts the depths under MPC and over-predicts those under RL. (c) Computational cost per decision for RL (one control interval) and MPC (one replanning cycle, covering two control intervals), separated into controller and surrogate-rollout time. RL requires

692~\mu\mathrm{s}

for policy inference and

11.47~\mathrm{ms}

for the surrogate rollout (

12.16~\mathrm{ms}

total), whereas MPC with

H=10

requires

35.36~\mathrm{ms}

for optimization and

3.34~\mathrm{s}

for repeated surrogate rollouts (

3.37~\mathrm{s}

total). All latency measurements were obtained on a single NVIDIA GeForce RTX 5090 GPU.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 7 — Fig. A.1: Initial conditions and prescribed laser scan paths. Representative initial temperature fields span distinct residual-heat distributions and substrate geometries. Green dashed arrows indicate the prescribed laser scan paths used to evaluate the world model across these conditions.

Panel of Figure 8 — Fig. A.3: Additional world-model control example 2. (a) Melt-pool depth for the unoptimized scan, MPC and RL, comparing the world-model rollout (dashed) with the corresponding CFD re-simulation (solid). The horizontal dashed line marks the target depth. (b) Laser-power and scan-speed schedules for the three control strategies. (c) Initial temperature field and final CFD temperature fields obtained with MPC, RL and the unoptimized scan.

Panel of Figure 9 — Fig. A.5: Additional world-model control example 4. (a) Melt-pool depth for the unoptimized scan, MPC and RL, comparing the world-model rollout (dashed) with the corresponding CFD re-simulation (solid). The horizontal dashed line marks the target depth. (b) Laser-power and scan-speed schedules for the three control strategies. (c) Initial temperature field and final CFD temperature fields obtained with MPC, RL and the unoptimized scan.

Explore similar work

Jun 22, 2026cs.CV

Physics-Guided Spatiotemporal State Space Modeling for Lookahead Molten Pool Segmentation in Laser Wire-Feed Welding

Real-time weld-pool perception is critical for closed-loop control in laser wire-feed welding, where sensing, computation, and actuator response introduce unavoidable delay. This paper presents a physics-guided spatiotemporal state space network for lookahead weld-pool segmentation. The model uses historical coaxial grayscale images, welding process parameters, and aligned wire-state electrical signals to predict the future semantic layout of three physically meaningful regions: keyhole, wire, and molten pool. It combines a visual encoder, process- and sensor-conditioned feature normalization, patch-level temporal state space modeling, horizon-conditioned latent prediction, dense future feature prediction, and a motion-aware mask decoder. Auxiliary signed-distance-function supervision, temporal consistency, feature distillation, and fine-grained keyhole losses further constrain the predicted geometry and local motion. Experiments on a 43-sequence laser welding dataset show that the proposed WeldMamba reaches 74.63% mIoU at a 500 ms lookahead. Ablation studies further show that temporal history, patch-level state space modeling, and keyhole motion awareness are the main contributors to robust future segmentation.

Sen Li, Haichao Cui, Changhao Yin +4

Shanghai Key Laboratory of Materials Laser Processing and Modification, School of Materials Science and Engineering, Shanghai Jiao Tong University, Shanghai 200240, PR China

Molten Pool Physics-Guided Machine Learning

Ask PDF

Jul 29, 2026cs.RO

Enfold: Folding World Model Imagination into Predictive Representations for Ultra-Efficient Embodied Control

World generative models are typically used through what they produce: a rendered future, a video-conditioned action, or latent context computed by a costly generative branch. We argue that their more reusable asset is the computation that constructs a future. As a generator transforms a corrupted future into a coherent trajectory, its intermediate states organize appearance, spatial layout, and interaction across levels of abstraction. Can this future-generative computation be internalized in a representation inferred from the present alone? We present Enfold, which transfers this computation into a representation predicted from the current visual context and language instruction. During training, multi-level states exposed as the generator processes the observed future supervise a current-only encoder. The learned representation is fed back to condition future generation and is read by task heads without allowing task gradients to reshape the encoder. At deployment, action prediction no longer executes the generator. Across LIBERO, RoboTwin2.0, and real-robot tasks, Enfold supports strong control while reducing action latency by

3.7\times

relative to Fast--WAM, Enfold-Flash reaches

10.1\times

. Representation analyses show that it suppresses nuisance variation and preferentially captures changes that emerge over longer horizons. When the current scene is altered by human intervention, both the generated continuation and the executed actions adapt, which is inconsistent with fixed trajectory replay. These results recast a world generator as a source of predictive control representations: its future need not be materialized at every step if its internal structure can be enfolded into the present.

Weili Zeng, Yitong Xing, Fulong Liu +10

Shanghai Jiao Tong University · Qilu University of Technology (Shandong Academy of Sciences)

Playable Video World Generation Goal-Conditioned Dynamic Manipulation

Ask PDF

Jul 1, 2026cs.LG

Valdi: Value Diffusion World Models

World models can enable Model Predictive Control (MPC), but this requires dynamics prediction that is both fast enough for online use and expressive enough to represent uncertain futures. Diffusion models offer a natural mechanism for modeling uncertain dynamics, yet their iterative inference procedure makes them difficult to use for low-latency latent planning. We bridge this gap with Value Diffusion World Models (Valdi), combining end-to-end online training for MPC with a latent diffusion dynamics model. In preliminary experiments on the CarRacing environment, we show that Valdi, using a single diffusion step at both training and inference, matches a deterministic MLP baseline. Our experiments expose a trade-off between predictive multimodality and control performance in this setup. Code is available at https://github.com/Kit115/ValueDiffusionWorldModels.

Christopher Lindenberg, Kashyap Chitta

University of Tübingen, Germany · 2KE:SAI – Kyutai ELLIS Scalable Autonomous Intelligence, Germany

Latent World Models Model Predictive Control Low-Latency Latent Planning World Models Latent Diffusion Model

Ask PDF