Assessing Language Models for Scientific Applications: Inertial Confinement Fusion as a Use Case

P. B. Radha; D. D. Meyerhofer; M. Bhattarai; B. Cheng; L. Diaz; C. L. Rousculp; A. G. Seaton; G. Gupta

cs.AIOct 27, 2026

Assessing Language Models for Scientific Applications: Inertial Confinement Fusion as a Use Case

Authors: P. B. Radha, D. D. Meyerhofer, M. Bhattarai, B. Cheng, L. Diaz, C. L. Rousculp, A. G. Seaton, G. Gupta,

Organizations: Los Alamos National Laboratory, Los Alamos, NM · NVIDIA Corporation, Santa Clara, CA

Abstract

Advancing language models for specialized scientific applications requires evaluations of scientific fluency, physical reasoning, and quantitative analyses. Using inertial confinement fusion (ICF) as a test case, we have developed an expert-curated benchmark of 250 questions spanning qualitative reasoning, numerical calculation, symbolic derivation, and equation formulation. Responses have been evaluated using deterministic numerical and symbolic checks together with language-model judges. We compared a variety of out-of-the-box closed-source and open-source large language models with Prospero, a model specialized through combined domain-adaptive pretraining and supervised fine-tuning for ICF. We also conducted initial retrieval-assisted evaluations. Prospero improved its aggregate score by 10.7 percentage points over its base model, with gains across all question types. Frontier models achieved the highest aggregate scores, but critical physics errors persisted, particularly in numerical calculations. We note that domain-specific performance can improve through specialization, but benchmark gains alone do not establish scientific reliability. Retrieval improved performance for a smaller model but yielded no statistically significant improvement for a frontier model. Two scientific stress tests further illustrate useful capabilities and the need for expert assessment of physical assumptions and model validity. Based on these findings, we recommend evaluating existing domain capabilities and retrieval benefits before committing to substantial additional specialization.

Figures & tables

Figure 2: Categories of questions developed for the golden-set - the set of 250 question and answer pairs generated by ICF experts. The numbers at the end of the bars indicate the number of questions of each type in the golden-set.

Question categories	Example topics
Fundamental physics	fusion reactivities, waves, rocket equation, electromagnetic forces
Plasma properties	plasma frequency, collision frequency, Debye screening
ICF concepts	hot spot ignition, fast ignition, volumetric ignition, aneutronic fusion
Laser plasma interactions	cross beam energy transfer, two plasmon decay, stimulated Raman scattering, preheat, compression
Engineering (targets, lasers, diagnostics)	instability seeds, characterizing laser speckle, laser optics, neutron diagnostics
Hydrodynamics	hydroefficiency, shocks, instabilities, compression, energy scaling

Table 1: Examples of topics within each area of the questions in the golden-set. Note that some topics can be listed in multiple categories.

Question Type	Percent in set
Qualitative/Reasoning	29.2
Numerical calculations	25.6
Symbolic derivation/Reasoning	24.0
Equation formulation	20.8
Multiple-choice	0.4

Table 2: Percentage of questions of each type within the golden-set of 250 questions

Figure 3: Workflow used to assess LLM outputs. Blocks 4 and 5 together form an effective single scoring stage: Block 4 lists the physics-aware equivalence criteria (numerical tolerance, unit consistency, algebraic equivalence) that the judges in Block 5 apply when comparing each answer with the answers in the golden-set. API: Application Programming Interface; CAS: Computer Algebra System; 10,000 bootstrap samples: statistical uncertainty estimated from 10,000 resamples of the fixed scores.

Question Type	Rubric Weighting
Numeric	55% value; 15% units and dimensions; 20% method; 10% assumptions and uncertainty.
Symbolic	60% expression; 15% signs, variables, and units; 20% derivation; 5% assumptions.
Conceptual	60% required claims; 25% factual correctness; 10% causal explanation; 5% relevance.
Multi-part	Weighted mean of explicit subpart scores; every subpart is required for completeness.

Table 3: Scoring rubric across each question type

Figure 4: Pairwise agreement between judges on scores (blue) and on the critical-error rates (orange).

Figure 5: Stability of judge ranking. Consensus among the three judges is used for ranking. The top models are unanimously assessed as the best performing models. Models following those change places but never more than two spots. Only the top eight models of the 22 assessed are shown. Model references in text.

Figure 6: Process followed for Domain adaptive pretraining (DAPT) and supervised fine-tuning (SFT)

Figure 7: Example of an intermediate stage during the training process for the standardized benchmark GPQA. With increasing ICF training or continuous pretraining (CPT) performance drops (blue line) but can be recovered with supervised fine-tuning (SFT). The drop in performance is correlated with the increasing percentage of questions that are unanswered (orange line). Accuracy is recovered when the percentage of unanswered questions drops to zero.

Figure 8: Overall performance of the top 4 models, Prospero and its base (Llama-3.3-Nemotron-49B-Instruct-128K) used in this work on the 250-question ICF benchmark. Bars show the three-judge rubric score; horizontal intervals are 95% confidence intervals from a 10,000-replicate question bootstrap. Each question contributes the mean of three attempts, and missing attempts count as zero.

Figure 9: Heat map showing scores of the top 4 models on the golden-set by ICF subarea/category compared with Prospero, and its base model (Llama-3.3-Nemotron-49B-Instruct-128K).

Figure 10: Heat map showing scores of the top 4 models on the golden-set by question type compared with Prospero and its base model (Llama-3.3-Nemotron-49B-Instruct-128K). The single multiple-choice question is not included.

Figure 11: Prospero performance on golden-set questions versus the base model.

Figure 12: Model performance for (a) the smaller Gemma 4 model; and (b) the larger GPT-5.4 model as a function of the literature available to the model. (c) Gemma 4 improves because its critical or physics errors drop with retrieval whereas GPT-5.4 performance does not improve significantly with additional retrieval provided by URSA or ODR.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 13: Rayleigh-Taylor mode under three explicit assumptions. Purely growing hyperbolic cosine function (solid blue); amplitude from the piecewise method using the terminal bubble velocity $v(t_{0.1})$ given by Eq. 8 (dashed purple); amplitude from the piecewise method using the terminal bubble velocity $v_{NL}$ given by Eq. 5 (dashed-dot green).

Model	Observations
Claude Sonnet 5	Uses $\cosh$ solution for both times and flags nonlinear behavior but does not calculate the appropriate nonlinear value
Inkling Small	Explains both choices for the initial mode, flags nonlinear behavior but does not calculate the appropriate nonlinear value
GPT-5.4	Uses exponential solution for both times; ignores nonlinear behavior
Nemotron Ultra	Uses $\cosh$ solution for both times; ignores nonlinear behavior

Appendix

Table 4: Observations about the solutions for the top four performing models. The reference solution uses the cosh form, corresponding to an initially stationary interface. None of the models correctly calculates the nonlinear amplitude using either model, although some flag nonlinearity.

Figure 14: Calculated p- 11 B reactivity (line) compared to values obtained by inspection of Fig. 5 in Ref. [ 53 ] versus ion temperature

Figure 15: Analytic solution for the ratio of the fusion to radiation energy as a function of ion temperature. This is for the analytically derived optimum ratio of boron number fraction. At higher ratios of electron-to-ion temperature no solution is found.

Figure 16: Ignition threshold for a hot spot p- 11 B ignition problem. Sensitivity to uncertainties in fusion reactivity, radiation loss, and alpha particle range is also shown. Some lines are missing as they were studied but the model found no plausible physical result.

Scenario	T i (keV)	T e (keV)	x B	P f /P Br	$\rho$ R (g/cm 2 )	$f_{\alpha}$
Equal temperature 48.36% recycled	286.9	286.9	0.088	1.22	82.3	0.92
$T_{e}/T_{i}=0.33$ No recycling	320.2	106.7	0.088	1.11	58.8	0.97
$T_{e}/T_{i}=0.30$ No recycling	306.7	92.0	0.092	1.16	36.6	0.97
$T_{e}/T_{i}=0.10$ No recycling	243.7	24.37	0.14	1.74	4.84	0.96

Appendix

Table 5: Conditional power balance solutions identified for the p–

{}^{11}\mathrm{B}

ignition model under different temperature and recycling scenarios.

Scientific Reasoning in Language Models

Scientific reasoning in language models studies how AI systems use evidence, domain knowledge, and tools to solve problems and support discovery.

See all 149 papers

Decomposition-Guided Diffusion Language Models for Inertial Confinement Fusion Prediction
Sep 7, 2026Xiang Zhang, Varchas Gopalaswamy, Rahman Ejaz +2Time Series Forecasting Physics-Informed ML
Purdue University · University of Rochester
Rethinking Domain Specialization for Open-Ended Scientific Reasoning in Astronomy Language Models
Sep 15, 2026Vanessa Lama, Sanjay Das, Emily Herron +5LLM Evaluation Scientific QA
Oak Ridge National Laboratory, Oak Ridge, Tennessee, USA · The Ohio State University, Columbus, Ohio, USA · Max-Planck-Institut für Astronomie, Königstuhl 17, D-69117 Heidelberg, Germany +1
Teaching Language Models to Forecast Research Success Through Comparative Idea Evaluation
Apr 6, 2026Srujan P Mule, Aniketh Garikaparthi, Manasi PatwardhanScientific Idea Evaluation Large Language Model-Guided Scientific Discovery
IISER Pune · TCS Research

Sep 7, 2026J/K move · Enter open · S save

Decomposition-Guided Diffusion Language Models for Inertial Confinement Fusion Prediction

Xiang Zhang, Varchas Gopalaswamy, Rahman Ejaz, Riccardo Betti+1

Purdue University · University of Rochester

Time Series Forecasting Physics-Informed ML Diffusion Language Models

Open paper AskPDF arXiv

Inertial confinement fusion (ICF) is a leading pathway toward clean energy, but each shot at the National Ignition Facility costs on the order of one million dollars, making accurate AI surrogates a high-value target. We study exogenous-driven ICF waveform prediction, where a 512-step neutron-rate diagnostic must be inferred directly from a laser pulse and target design parameters, with no historical response observed. The regime stresses standard time-series predictors with temporal sparsity (picosecond peak in a nanosecond window), input-output scale mismatch (under 300 real shots), and peak sensitivity (picosecond timing). We propose ICF-DLM, to our knowledge the first LM-based ICF predictor, combining (i) a physics-typed decomposition into yield

Y_{DT}

, peak timing

t_{\mathrm{peak}}

, and local waveform

w_{\mathrm{local}}

; (ii) bidirectional denoising that defers commitment to peak location; and (iii) a physics-driven PPO reward re-injecting metric structure across numeric tokens. On ICFBench (50K simulations + 232 experimental shots), ICF-DLM cuts peak-timing error from 11.6 to 9.2 steps over a matched autoregressive LLaMA-3-8B and outperforms classical sequence models and LLM-based time-series predictors. Beyond ICF, the recipe shows potential to address science domains with low data and sparse events.

Abstract

Scientific Reasoning in Language Models

Explore similar work

Decomposition-Guided Diffusion Language Models for Inertial Confinement Fusion Prediction

Rethinking Domain Specialization for Open-Ended Scientific Reasoning in Astronomy Language Models

Teaching Language Models to Forecast Research Success Through Comparative Idea Evaluation