PhysFormer: A Physics-Embedded Generative Model for Physically Self-Consistent Spectral Synthesis

Siqi Wang; Mengmeng Zhang; Yude Bu; Chaozhou Mou

astro-ph.IMMar 2, 2026

PhysFormer: A Physics-Embedded Generative Model for Physically Self-Consistent Spectral Synthesis

Authors: Siqi Wang, Mengmeng Zhang, Yude Bu, Chaozhou Mou

Organizations: School of Mathematics and Statistics, Shandong University, Weihai, 264209, Shandong, China · Weihai Institute for Interdisciplinary Research, Shandong University, Weihai 264209, Shandong, China

Abstract

In scientific and engineering domains, modeling high-dimensional complex systems governed by partial differential equations (PDEs) remains challenging in terms of physical consistency and numerical stability. However, existing approaches, such as physics-informed neural networks (PINNs), typically rely on known physical fields or coefficients and enforce physical constraints via external loss functions, which can lead to training instability and make it difficult to handle high-dimensional or unobservable scenarios. To this end, we propose PhysFormer, a generative modeling framework that is self-consistent at both the data and physical levels. PhysFormer leverages a low-dimensional, physically interpretable latent space to learn key physical quantities directly from data without requiring known high-dimensional physical field parameters, and embeds the physical process of radiative flux generation within the network to ensure the physical consistency of the generated spectra. In high-dimensional, degenerate inversion tasks, PhysFormer constrains generation within physical limits and enhances spectral fidelity and inversion stability under varying signal-to-noise ratios (SNRs). More broadly, this approach shifts the physical processes from external loss functions into the generative mechanism itself, providing a physically consistent generative modeling paradigm for complex systems involving unknown or unobservable physical quantities.

Figures & tables

Figure 1: The architecture of PhysFormer. (a) Physical generation module that embeds physical processes directly into the network architecture. (b) Physically consistent spectral autoencoder. (c) Bottleneck architecture for generating a low-dimensional physical latent space. (d) Mapping network from stellar fundamental parameters to the physical latent space. (e) Latent Space Expansion Module. (f) PhysFormer decoder, used to generate key physical quantities required for subsequent physical computations. (g) Radiative Intensity Generation Module. (h) Transformer Block structure.

Figure 2: Visualization of Spectra Generated by PhysFormer. Left: comparison between PhysFormer-generated spectra and ground truth. Middle: comparison among PhysFormer, Symmetric Autoencoder, and SPECULATOR. Top right: median spectral error across wavelengths. Bottom right: visualization of the learned low-dimensional physical latent space.

Model	RMSE	MAE	$R^{2}$
PhysFormer (Ours)	0.0054	0.0033	0.9997
PhysGNN	0.1959	0.1622	0.7216
Symmetric Autoencoder	0.0200	0.0123	0.9956
SPECLATOR	0.0106	0.0064	0.9985
The Payne	0.0618	0.0357	0.9372
Kurucz-a1	0.0502	0.0204	0.9760

Table 1: Comparison of PhysFormer with Other Baselines

Model Configuration	MSE(AE 200epoch)	MSE(P2F)	RTE Loss	ET Loss
PhysFormer (Ours)	$\mathbf{6.24\times 10^{-5}}$	$\mathbf{4.41\times 10^{-5}}$	$\mathbf{5.16\times 10^{-5}}$	0.0195
w/o Physics Process	$8.41\times 10^{-5}$	$4.28\times 10^{-5}$	-	-
w/o Bottleneck	$3.75\times 10^{-5}$	0.0489	$5.58\times 10^{-5}$ (AE) $3.09\times 10^{-5}$ (P2F)	0.0236(AE) 0.1210(P2F)
Smaller Bottleneck	$8.60\times 10^{-4}$	-	$7.67\times 10^{-5}$	0.0164
MLP Decoder	$2.83\times 10^{-4}$	-	$1.17\times 10^{-4}$	0.0189
Reduced Angular Integration	$8.87\times 10^{-4}$	-	$2.66\times 10^{-5}$	0.0160

Table 2: Ablation Experiment Results. AE refers to the autoencoder stage results (pretrained for 200 epochs), and P2F refers to the results from the parameter-to-flux prediction stage. RTE Loss represents the mean squared error of the radiative transfer equation residuals, while ET Loss denotes the mean squared error of the energy theorem residuals. Due to the limited training epochs in the AE stage, its errors are generally higher than those in the P2F stage.

Figure 3: Residual Distributions of RTE and ET on the Test Set. Top row: Residual distributions of the Radiative Transfer Equation (RTE) and Energy Theorem (ET) for the three models: PhysFormer, PhysFormer without Bottleneck, and MLP Decoder. Bottom row: Absolute residual distributions of RTE and ET for the first 100 wavelength points for PhysFormer.

Figure 4: Loss landscape in

T_{\mathrm{eff}}

–

\log g

space for a single observed spectrum. Color indicates the spectrum reconstruction MSE. The red cross marks the recovered parameter. The elongated and tilted contours indicate a strong degeneracy between the two parameters.

Figure 5: Example of Parameter Inversion Results and Error Distributions. Top: Example spectra generated from the inverted parameters, comparing the output of PhysFormer with PhysFormer w/o Physics Process. Bottom: Distributions of absolute errors in the inversion of four-dimensional stellar parameters under different signal-to-noise ratio (SNR) conditions for the two models.

Radiative Transfer

See all 10 papers

Stabilizing Physics-Informed Consistency Models via Structure-Preserving Training
Feb 10, 2026Che-Chia Chang, Chen-Yang Dai, Te-Sheng Lin +2Physics-Informed Generative Modeling PDE Surrogate Modeling
Institute of Artificial Intelligence Innovation National Yang Ming Chiao Tung University Hsinchu, Taiwan · Department of Applied Mathematics National Yang Ming Chiao Tung University Hsinchu, Taiwan · Department of Applied Mathematics National Yang Ming Chiao Tung University Hsinchu, Taiwan National Center for Theoretical Sciences National Taiwan University Taipei, Taiwan
Gen-PINNs: Generative Adversarial Physics Informed Neural Networks for solving partial differential equations
Oct 7, 2026Muhammad M. Akmal, Kamy Sepehrnoori, Michael J. PyrczPDE Solving Physics-Informed ML
Hilderbrand Department of Petroleum and Geosystems Engineering, The University of Texas at Austin · Department of Earth and Planetary Sciences, The University of Texas at Austin
Physics-Informed Generative Solver: Bridging Data-Driven Priors and Conservation Laws for Stable Spatiotemporal Field Reconstruction
May 21, 2026Ziyuan Zhu, Keyu Hu, Zhifei Chen +10Physics-Informed Generative Modeling Physics-Informed ML
School of Advanced Manufacturing Engineering, Nanjing University, Suzhou, 215163, China. · National Laboratory of Solid State Microstructures, Nanjing University, Nanjing, 210093, China. · Suzhou Acoustics Industry Technology Research Institute Co., Ltd., Suzhou, 215513, China. +8

Feb 10, 2026J/K move · Enter open · S save

Stabilizing Physics-Informed Consistency Models via Structure-Preserving Training

Che-Chia Chang, Chen-Yang Dai, Te-Sheng Lin, Ming-Chih Lai+1

Institute of Artificial Intelligence Innovation National Yang Ming Chiao Tung University Hsinchu, Taiwan · Department of Applied Mathematics National Yang Ming Chiao Tung University Hsinchu, Taiwan · Department of Applied Mathematics National Yang Ming Chiao Tung University Hsinchu, Taiwan National Center for Theoretical Sciences National Taiwan University Taipei, Taiwan

Physics-Informed Generative Modeling PDE Surrogate Modeling Few-Step Diffusion Sampling Physics-Informed ML Consistency Models

Open paper AskPDF arXiv

We propose a physics-informed consistency modeling framework for solving partial differential equations (PDEs) via fast, few-step generative inference. We identify a key stability challenge in physics-constrained consistency training, where PDE residuals can drive the model toward trivial or degenerate solutions, degrading the learned data distribution. To address this, we introduce a structure-preserving two-stage training strategy that decouples distribution learning from physics enforcement by freezing the coefficient decoder during physics-informed fine-tuning. We further propose a two-step residual objective that enforces physical consistency on refined, structurally valid generative trajectories rather than noisy single-step predictions. The resulting framework enables stable, high-fidelity inference for both unconditional generation and forward problems. We demonstrate that forward solutions can be obtained via a projection-based zero-shot inpainting procedure, achieving consistent accuracy of diffusion baselines with orders of magnitude reduction in computational cost.

Abstract

Radiative Transfer

Explore similar work

Stabilizing Physics-Informed Consistency Models via Structure-Preserving Training

Gen-PINNs: Generative Adversarial Physics Informed Neural Networks for solving partial differential equations

Physics-Informed Generative Solver: Bridging Data-Driven Priors and Conservation Laws for Stable Spatiotemporal Field Reconstruction