Scientific ML

ML: Machine Learning

Ask about this topic

Overview

Scientific machine learning studies how to use machine learning for scientific prediction, simulation, and discovery while accounting for governing equations, physical constraints, and measurement processes. Common approaches include physics-informed neural networks, neural operators, hybrid models that augment classical solvers, and symbolic methods for learning equations; differentiable solvers and uncertainty estimates support inference and decision-making. Applications span problems such as fluid dynamics, climate, chemistry, materials, and astronomy, often where data are sparse or simulations are expensive. A central challenge is that physical structure can improve reliability when correctly specified but mislead when it does not fit the system, so predictive accuracy must be weighed against physical fidelity, generalization, uncertainty, and careful validation.

Latest papers

CardsList

125 papers11 papers in the last four weeks, level with the four weeks before. 0.1% of all new papers.

Apr 22, 2026

AI models of unstable flow exhibit hallucination

We report the first systematic evidence of hallucination in AI models of fluid dynamics, demonstrated in the canonical problem of hydrodynamically unstable transport known as viscous fingering. AI-based modeling of flow with instabilities remains challenging because rapidly evolving, multiscale fingering patterns are difficult to resolve accurately. We identify solutions that appear visually realistic yet are physically implausible, analogous to hallucinations in large language models. These hallucinations manifest as spurious fluid interfaces and reverse diffusion that violate conservation laws. We show that their origin lies in the spectral bias of AI models, which becomes dominant at high flow rates and viscosity contrasts. Guided by this insight, we introduce DeepFingers, a new framework for AI-driven fluid dynamics that enforces balanced learning across the full spectrum of spatial modes by combining the Fourier Neural Operator with a Deep Operator Network to predict the spatiotemporal evolution of viscous fingers. By conditioning on both time and viscosity contrast, DeepFingers learns mappings between successive concentration fields across regimes. The framework accurately captures tip splitting, finger merging, and channel formation while preserving global metrics of mixing. The results open a new research direction to investigate fundamental limitations in AI models of physical systems.

Ramdhan Wibawa, Birendra Jha

Department of Chemical Engineering and Materials Science, University of Southern California, 925 Bloom Walk, Los Angeles, 90089, CA, USA.

Scientific MLNeural OperatorsComputational Fluid Dynamics
AskPDF
Apr 22, 2026

Machine learning moment closure models for the radiative transfer equation IV: enforcing symmetrizable hyperbolicity in two dimensions

This is our fourth work in the series on machine learning (ML) moment closure models for the radiative transfer equation (RTE). In the first three papers of this series, we considered the RTE in slab geometry in 1D1V (i.e. one dimension in physical space and one dimension in angular space), and introduced a gradient-based ML moment closure [1], then enforced the hyperbolicity through a symmetrizer [2], or together with physical characteristic speeds by learning the eigenvalues of the Jacobian matrix [3]. Here, we extend our framework to the RTE in 2D2V (i.e. two dimensions in physical space and two dimensions in angular space). The main idea is to preserve the leading part of the classical PNP_N model and modify only the highest-order block row. By analyzing the structural properties of the PNP_N model, we show that its coefficient matrices are symmetric and admit a block-tridiagonal structure. Then we use this property to introduce a block-diagonal symmetrizer for the ML moment model and derive explicit algebraic conditions on the closure blocks which guarantee the symmetrizable hyperbolicity of the resulting ML system. These conditions lead to a natural parametrization of the closure in terms of a symmetric positive definite matrix together with symmetric closure blocks, which can be learned from data while automatically enforcing symmetrizable hyperbolicity by construction. The numerical results show that the proposed framework improves upon the classical PNP_N model while maintaining hyperbolicity.

Juntao Huang

Department of Mathematical Sciences, University of Delaware, Newark, DE, 19716, USA.

Reduced-Order ModelingScientific MLRadiative Transfer
AskPDF
Apr 21, 2026

Design Rules for Extreme-Edge Scientific Computing on AI Engines

Extreme-edge scientific applications use machine learning models to analyze sensor data and make real-time decisions. Their stringent latency and throughput requirements demand small batch sizes and require that model weights remain fully on-chip. Spatial dataflow implementations are common for extreme-edge applications. Spatial dataflow works well for small networks, but it fails to scale to larger models due to inherent resource scaling limitations. AI Engines on modern FPGA SoCs offer a promising alternative with high compute density and additional on-chip memory. However, the architecture, programming model, and performance-scaling behavior of AI Engines differ fundamentally from those of the programmable logic, making direct comparison non-trivial and the benefits of using AI Engines unclear. This work addresses how and when extreme-edge scientific neural networks should be implemented on AI Engines versus programmable logic. We provide systematic architectural characterization and micro-benchmarking and introduce a latency-adjusted resource equivalence (LARE) metric that identifies when AI Engine implementations outperform programmable logic designs. We further propose spatial and API-level dataflow optimizations tailored to low-latency scientific inference. Finally, we demonstrate the successful deployment of end-to-end neural networks on AI Engines that cannot fit on programmable logic when using the hlsml toolchain.

Zhenghua Ma, G Abarajithan, Dimitrios Danopoulos +3

University of California San Diego, La Jolla, CA, USA · European Organization for Nuclear Research (CERN), Geneva, Switzerland

Scientific MLEdge MLFPGA-Based Neural Network Acceleration
AskPDF
Apr 17, 2026

Physics-Informed Neural Networks with Learnable Loss Balancing and Transfer Learning

We propose a self-supervised physics-informed neural network (PINN) framework that adaptively balances physics-based and data-driven supervision for scientific machine learning under data scarcity. Unlike prior PINNs that rely on fixed or heuristic weighting of physics residuals and data loss, our approach introduces a learnable blending neuron that dynamically adjusts the relative contribution of each term based on their uncertainties. This mechanism enables stable training and improved generalization without manual tuning. To further enhance efficiency, we integrate a transfer learning strategy that reuses representations from related domains and adapts them to new physical systems with limited data. We validate the framework for the prediction of heat transfer in liquid-metal miniature heat sinks using only 87 CFD datapoints, where the adaptive PINN achieves an error <8%, outperforming shallow neural networks, kernel methods, and physics-only baselines. Our framework provides a general recipe for embedding physics adaptively into neural networks, offering a robust and reproducible approach for data-scarce problems across various scientific domains, including fluid dynamics and material modeling.

Reza Pirayeshshirazinezhad

Texas A&M University College Station, TX, USA

Self-Supervised LearningAdaptive Loss WeightingScientific MLPhysics-Informed ML
AskPDF
Apr 13, 2026

Machine-learning modeling of magnetization dynamics in quasi-equilibrium and driven metallic spin systems

We review recent advances in machine-learning (ML) force-field methods for large-scale Landau-Lifshitz-Gilbert (LLG) simulations of metallic spin systems. We generalize the Behler-Parrinello (BP) ML architecture -- originally developed for quantum molecular dynamics -- to construct scalable and transferable ML models capable of capturing the intricate dependence of electron-mediated exchange fields on the local magnetic environment characteristic of itinerant magnets. A central ingredient of this framework is the implementation of symmetry-aware magnetic descriptors based on group-theoretical bispectrum formalisms. Leveraging these ML force fields, LLG simulations faithfully reproduce hallmark non-collinear magnetic orders -- such as the 120∘120^\circ and tetrahedral states -- on the triangular lattice, and successfully capture the complex spin textures emerging in the mixed-phase states of a square-lattice double-exchange model under thermal quench. We further discuss a generalized potential theory that extends the BP formalism to incorporate both conservative and nonconservative electronic torques, thereby enabling ML models to learn nonequilibrium exchange fields from computationally demanding microscopic approaches such as nonequilibrium Green's-function techniques. This extension yields quantitatively accurate predictions of voltage-driven domain-wall motion and establishes a foundation for quantum-accurate, multiscale modeling of nonequilibrium spin dynamics and spintronic functionalities.

Gia-Wei Chern, Yunhao Fan, Sheng Zhang +1

Department of Physics, University of Virginia, Charlottesville, VA 22904, USA

Scientific MLMaterials Science
AskPDF
Apr 10, 2026

Learning What's Real: Disentangling Signal and Measurement Artifacts in Multi-Sensor Data, with Applications to Astrophysics

Data collected from the physical world is always a combination of multiple sources: an underlying signal from the physical process of interest and a signal from measurement-dependent artifacts from the sensor or instrument. This secondary signal acts as a confounding factor, limiting our ability to extract information about the physics underlying the phenomena we observe. Furthermore, it complicates the combination of observations in heterogeneous or multi-instrument settings. We propose a deep learning framework that leverages overlapping observations, a dual-encoder architecture, and a counterfactual generation objective to disentangle these factors of variation. The resulting representations explicitly separate intrinsic signals from sensor-specific distortions and noise, and can be used for counterfactual view generation, parameter inference unconfounded by measurement distortions, and instrument-independent similarity search. We demonstrate the effectiveness of our approach on astrophysical galaxy images from the DESI Legacy Imaging Survey (Legacy) and the Hyper Suprime-Cam (HSC) Survey as a representative multi-instrument setting. This framework provides a general recipe for scientific and multi-modal self-supervised pretraining: construct training pairs from overlapping observations of the same physical system, treat sensor- or modality-specific effects as augmentations, and learn invariant representations through counterfactual generation.

Pablo Mercader-Perez, Carolina Cuesta-Lazaro, Daniel Muthukrishna +5

Massachusetts Institute of Technology · Flatiron Institute, Simons Foundation · Institute for Advanced Studies +5

AstrophysicsMultimodal Disentangled Representation LearningScientific MLMulti-View Learning
AskPDF
Apr 7, 2026

Multiscale Physics-Informed Neural Network for Complex Fluid Flows with Long-Range Dependencies

Fluid flows are governed by the nonlinear Navier-Stokes equations, which can manifest multiscale dynamics even from predictable initial conditions. Predicting such phenomena remains a formidable challenge in scientific machine learning, particularly regarding convergence speed, data requirements, and solution accuracy. In complex fluid flows, these challenges are exacerbated by long-range spatial dependencies arising from distant boundary conditions, which typically necessitate extensive supervision data to achieve acceptable results. We propose the Domain-Decomposed and Shifted Physics-Informed Neural Network (DDS-PINN), a framework designed to resolve such multiscale interactions with minimal supervision. By utilizing localized networks with a unified global loss, DDS-PINN captures global dependencies while maintaining local precision. The robustness of the approach is demonstrated across a suite of benchmarks, including a multiscale linear differential equation, the nonlinear Burgers' equation, and data-free Navier-Stokes simulations of flat-plate boundary layers. Finally, DDS-PINN is applied to the computationally challenging backward-facing step (BFS) problem; for laminar regimes (Re = 100), the model yields results comparable to computational fluid dynamics (CFD) without the need for any data, accurately predicting boundary layer thickness, separation, and reattachment lengths. For turbulent BFS flow at Re = 10,000, the framework achieves convergence to O(10^-4) using only 500 random supervision points (< 0.3 % of the total domain), outperforming established methods like Residual-based Attention-PINN in accuracy. This approach demonstrates strong potential for the super-resolution of complex turbulent flows from sparse experimental measurements.

Prashant Kumar, Rajesh Ranjan

Department of Aerospace Engineering, Indian Institute of Technology Kanpur, Kanpur, India 208016

Domain Decomposition MethodsNeural PDE SolversPhysics-Informed MLPDE SolvingScientific MLComputational Fluid Dynamics
AskPDF
Apr 6, 2026

Learning to Unscramble Feynman Loop Integrals with SAILIR

Integration-by-parts (IBP) reduction of Feynman integrals to master integrals is a key computational bottleneck in precision calculations in high-energy physics. Traditional approaches based on the Laporta algorithm require solving large systems of equations, leading to memory consumption that grows rapidly with integral complexity. We present SAILIR (Self-supervised AI for Loop Integral Reduction), a new machine learning approach in which a transformer-based classifier guides the reduction of integrals one step at a time in a fully online fashion. The classifier is trained in an entirely self-supervised manner on synthetic data generated by a scramble/unscramble procedure: known reduction identities are applied in reverse to build expressions of increasing complexity, and the classifier learns to undo these steps. When combined with beam search and a highly parallelized, asynchronous, single-episode reduction strategy, SAILIR can reduce integrals of arbitrarily high weight with bounded memory. We benchmark SAILIR on the two-loop triangle-box topology, comparing against the state-of-the-art IBP reduction code Kira across 16 integrals of varying complexity. While SAILIR is slower in wall-clock time, its per-worker memory consumption remains approximately flat regardless of integral complexity, in contrast to Kira whose memory grows rapidly with complexity. For the most complex integrals considered here, SAILIR uses only 40% of the memory of Kira while achieving comparable reduction times. This demonstrates a fundamentally new paradigm for IBP reduction in which the memory bottleneck of Laporta-based approaches could be entirely overcome, potentially opening the door to precision calculations that are currently intractable.

David Shih

NHETC, Dept. of Physics and Astronomy, Rutgers University, Piscataway, NJ 08854, USA

Scientific MLMemory-Efficient Inference
AskPDF
Mar 31, 2026

Informed Machine Learning with Knowledge Landmarks

Informed Machine Learning has emerged as a viable generalization of Machine Learning (ML) by building a unified conceptual and algorithmic setting for constructing models on a unified basis of knowledge and data. Physics-informed ML involving physics equations is one of the developments within Informed Machine Learning. This study proposes a novel direction of Knowledge-Data ML, referred to as KD-ML, where numeric data are integrated with knowledge tidbits expressed in the form of granular knowledge landmarks. We advocate that data and knowledge are complementary in several fundamental ways: data are precise (numeric) and local, usually confined to some region of the input space, while knowledge is global and formulated at a higher level of abstraction. The knowledge can be represented as information granules and organized as a collection of input-output information granules called knowledge landmarks. In virtue of this evident complementarity, we develop a comprehensive design process of the KD-ML model and formulate an original augmented loss function L, which additively embraces the component responsible for optimizing the model based on available numeric data, while the second component, playing the role of a granular regularizer, so that it adheres to the granular constraints (knowledge landmarks). We show the role of the hyperparameter positioned in the loss function, which balances the contribution and guiding role of data and knowledge, and point to some essential tendencies associated with the quality of data (noise level) and the level of granularity of the knowledge landmarks. Experiments on two physics-governed benchmarks demonstrate that the proposed KD model consistently outperforms data-driven ML models.

Chuyi Dai, Witold Pedrycz, Suping Xu +2

Department of Electrical and Computer Engineering, University of Alberta, Edmonton, T6G 1H9, Canada · School of ElectroMechanical Engineering, Xidian University, Xi’an 710071, China · Hubei Subsurface Multi-Scale Imaging Key Laboratory, Institute of Geophysics and Geomatics, China University of Geosciences, Wuhan 430074, China

Scientific ML
AskPDF
Mar 25, 2026

Symbolic--KAN: Kolmogorov-Arnold Networks with Discrete Symbolic Structure for Interpretable Learning

Symbolic discovery of governing equations is a long-standing goal in scientific machine learning, yet a fundamental trade-off persists between interpretability and scalable learning. Classical symbolic regression methods yield explicit analytic expressions but rely on combinatorial search, whereas neural networks scale efficiently with data and dimensionality but produce opaque representations. In this work, we introduce Symbolic Kolmogorov-Arnold Networks (Symbolic-KANs), a neural architecture that bridges this gap by embedding discrete symbolic structure directly within a trainable deep network. Symbolic-KANs represent multivariate functions as compositions of learned univariate primitives applied to learned scalar projections, guided by a library of analytic primitives, hierarchical gating, and symbolic regularization that progressively sharpens continuous mixtures into one-hot selections. After gated training and discretization, each active unit selects a single primitive and projection direction, yielding compact closed-form expressions without post-hoc symbolic fitting. Symbolic-KANs further act as scalable primitive discovery mechanisms, identifying the most relevant analytic components that can subsequently inform candidate libraries for sparse equation-learning methods. We demonstrate that Symbolic-KAN reliably recovers correct primitive terms and governing structures in data-driven regression and inverse dynamical systems. Moreover, the framework extends to forward and inverse physics-informed learning of partial differential equations, producing accurate solutions directly from governing constraints while constructing compact symbolic representations whose selected primitives reflect the true analytical structure of the underlying equations. These results position Symbolic-KAN as a step toward scalable, interpretable, and mechanistically grounded learning of governing laws.

Salah A Faroughi, Farinaz Mostajeran, Amirhossein Arzani +1

Energy & Intelligence Lab, Department of Chemical Engineering, University of Utah, Salt Lake City, Utah, 84112, USA · Department of Mechanical Engineering, University of Utah, Salt Lake City, Utah, 84112, USA · Scientific Computing and Imaging Institute, University of Utah, Salt Lake City, Utah, 84112, USA +2

Interpretable MLScientific MLKolmogorov-Arnold NetworksSymbolic RegressionPDE DiscoveryPhysics-Informed ML
AskPDF
Mar 21, 2026

CFNN: Continued Fraction Neural Network

Accurately characterizing non-linear functional manifolds with singularities is a fundamental challenge in scientific computing. While Multi-Layer Perceptrons (MLPs) dominate, their spectral bias hinders resolving high-curvature features without excessive parameters. We introduce Continued Fraction Neural Networks (CFNNs), integrating continued fractions with gradient-based optimization to provide a rational inductive bias.'' This enables capturing complex asymptotics and discontinuities with extreme parameter frugality. We provide formal approximation bounds demonstrating exponential convergence and stability guarantees. To address recursive instability, we develop three implementations: CFNN-Boost, CFNN-MoE, and CFNN-Hybrid. Benchmarks show CFNNs consistently outperform MLPs in precision with one to two orders of magnitude fewer parameters, exhibiting up to a 47-fold improvement in noise robustness and physical consistency. By bridging black-box flexibility and white-box transparency, CFNNs establish a reliable grey-box'' paradigm for AI-driven scientific research.

Chao Wang, Xuancheng Zhou, Ruilin Hou +2

School of Future Technology, Shanghai University, Shanghai, 200444, China. · Institute of Artificial Intelligence, Shanghai University, Shanghai, 200444, China. · School of Computer Science and Technology, Shanghai University, Shanghai, 200444, China. +2

Function ApproximationScientific MLNeural Network Approximation Theory
AskPDF
Mar 20, 2026

How Can Machine Learning Emulators Best Support Climate Science?

For decades, physics-based climate models have been used to provide insights for climate decision-making. Their application is, however, constrained by significant computational and technical demands. Machine learning (ML) emulators offer a way to reduce these high computational costs; yet, it remains challenging to use ML emulators effectively in climate research. In practice, climate scientists often bypass emulators altogether, and machine learning researchers frequently develop them as methodological showcases without proving their practical utility. The reasons are diverse, ranging from limited accessibility and a lack of specialized knowledge to broader concerns about the physical grounding of ML methods. Here, we discuss limitations and introduce a framework for guiding emulator development, considering both climate science and machine learning perspectives. We argue that designing easy-to-adopt emulators that address clearly defined tasks and demonstrate their reliability is essential. This offers a promising path towards making machine-learning approaches more relevant and usable for applied climate research.

Luca Schmidt, Nina Effenberger, Vitus Benson +5

Institute for Atmospheric and Climate Science ETH Zurich · Seminar for Statistics ETH Zurich

Surrogate ModelingClimate ScienceScientific ML
AskPDF
Mar 18, 2026

scicode-lint: Detecting Methodology Bugs in Scientific Python Code with LLM-Generated Patterns

Methodology bugs in scientific Python code produce plausible but incorrect results that traditional linters and static analysis tools cannot detect. Several research groups have built ML-specific linters, demonstrating that detection is feasible. Yet these tools share a sustainability problem: dependency on specific pylint or Python versions, limited packaging, and reliance on manual engineering for every new pattern. As AI-generated code increases the volume of scientific software, the need for automated methodology checking (such as detecting data leakage, incorrect cross-validation, and missing random seeds) grows. We present scicode-lint, whose two-tier architecture separates pattern design (frontier models at build time) from execution (small local model at runtime). Patterns are generated, not hand-coded; adapting to new library versions costs tokens, not engineering hours. On Kaggle notebooks with human-labeled ground truth, preprocessing leakage detection reaches 65% precision at 100% recall; on 38 published scientific papers applying AI/ML, precision is 62% (LLM-judged) with substantial variation across pattern categories; on a held-out paper set, precision is 54%. On controlled tests, scicode-lint achieves 97.7% accuracy across 66 patterns.

Sergey V. Samsonau

Authentic Research Partners, Princeton, NJ

Static Code AnalysisScientific ML
AskPDF
Mar 16, 2026

Bridging Theory and Data: Correcting Nuclear Mass Models with Interpretable Machine Learning

Nuclear mass prediction is one of the core issues in nuclear physics research, yet it faces the challenge of small-sample datasets with high complexity. This study introduces the Kolmogorov-Arnold Network (KAN) into the refinement of nuclear mass models, proposing an efficient and interpretable solution. By constructing the KAN-WS4 hybrid model, the prediction accuracy is significantly improved (the root mean square error is reduced from 0.3 MeV to 0.16 MeV). Furthermore, leveraging the intrinsic interpretability of KAN, feature importance analysis reveals that the proton number is the most critical factor influencing residuals, indicating potential systematic biases in proton-related terms within existing theoretical models. The method's generality is demonstrated across five mass models. This study shows that KAN provides a novel approach to small-sample, high-complexity scientific problems. Its interpretability facilitates the data-driven discovery of physical laws, promising broad applicability to key nuclear physics issues.

Yanhua Lu, Tianshuai Shang, Pengxiang Du +2

College of Physics, Jilin University, Changchun 130012, China. · Department of Physics, Graduate School of Science, The University of Tokyo, Tokyo 113-0033, Japan · RIKEN Interdisciplinary Theoretical and Mathematical Sciences Program, Wako 351-0198, Japan

Interpretable MLScientific ML
AskPDF
Mar 16, 2026

Machine learning cannot decarbonise the subsurface until it can be audited

Geological CO2 storage, geothermal energy and underground hydrogen storage depend on irreversible or high-cost decisions made with sparse data and incomplete physical understanding. Machine learning (ML) increasingly informs these decisions, yet methods are usually judged on predictive accuracy rather than on whether their outputs can be trusted, traced and acted upon. We argue that subsurface ML should be designed as decision infrastructure with shared systems that carry calibrated uncertainty and provenance from observation to operating decision. We propose a decision-readiness checklist, an audit-trail workflow for monitoring updates, and benchmark designs that score models by the decisions they support. We identify validation needs shared across the three technologies, and the data-access and governance steps needed for ML to support regulatory and investment decisions.

Hannah P. Menke, Ahmed H. Elsheikh, Lingli Wei +2

Institute of GeoEnergy Engineering, Heriot-Watt University, Edinburgh, UK · Subsurface Energy Transition and Innovation Centre, Heriot-Watt University, Edinburgh, UK · SubsurfaceLabs, Abu Dhabi, UAE +1

Scientific MLPhysics-Informed MLUncertainty Quantification
AskPDF
Mar 16, 2026

A Score Filter Enhanced Data Assimilation Framework for Data-Driven Dynamical Systems

We introduce a score-filter-enhanced data assimilation framework designed to reduce predictive uncertainty in machine learning (ML) models for data-driven dynamical system forecasting. Machine learning serves as an efficient numerical model for predicting dynamical systems. However, even with sufficient data, model uncertainty remains and accumulates over time, causing the long-term performance of ML models to deteriorate. To overcome this difficulty, we integrate data assimilation techniques into the training process to iteratively refine the model predictions by incorporating observational information. Specifically, we apply the Ensemble Score Filter (EnSF), a generative AI-based training-free diffusion model approach, for solving the data assimilation problem in high-dimensional nonlinear complex systems. This leads to a hybrid data assimilation-training framework that combines ML with EnSF to improve long-term predictive performance. We shall demonstrate that EnSF-enhanced ML can effectively reduce predictive uncertainty in ML-based Lorenz-96 system prediction and the Korteweg-De Vries (KdV) equation prediction.

Jingqiao Tang, Ryan Bausback, Feng Bao +2

Data AssimilationScientific MLChaotic Time-Series Prediction
AskPDF
Mar 15, 2026

From Specification to Architecture: A Theory Compiler for Knowledge-Guided Machine Learning

Theory-guided machine learning has demonstrated that including authentic domain knowledge directly into model design improves performance, sample efficiency and out-of-distribution generalisation. Yet the process by which a formal domain theory is translated into architectural constraints remains entirely manual, specific to each domain formalism, and devoid of any formal correctness guarantee. This translation is non-transferable between domains, not verified, and does not scale. We propose the Theory Compiler: a system that accepts a typed, machine-readable domain theory as input and automatically produces an architecture whose function space is provably constrained to be consistent with that theory by construction, not by regularisation. We identify three foundational open problems whose resolution defines our research agenda: (1) designing a universal theory formalisation language with decidable type-checking; (2) constructing a compositionally correct compilation algorithm from theory primitives to architectural modules; and (3) establishing soundness and completeness criteria for formal verification. We further conjecture that compiled architectures match or exceed manually-designed counterparts in generalisation performance while requiring substantially less training data, a claim we ground in classical statistical learning theory. We argue that recent advances in formal machine learning theory, large language models, and the growth of an interdisciplinary research community have made this paradigm achievable for the first time.

Asela Hevapathige, Yu Xia, Sachith Seneviratne +1

AI, Optimization and Pattern Recognition Research Group, Faculty of Engineering and Information Technology, University of Melbourne, Melbourne, Australia

Scientific ML
AskPDF
Mar 13, 2026

An Empirical Investigation of Pre-Trained Deep Learning Model Reuse in the Scientific Process

Deep learning has achieved recognition for its impact within natural sciences, yet the prohibitive financial and technical cost of training models from scratch inhibit adoption. Following software engineering community guidance, natural scientists are reusing pre-trained deep learning models (PTMs) to amortize these costs. While prior works recommend PTM reuse patterns, we present the first empirical study of PTM reuse patterns in the natural sciences, quantifying the utilization and impact of PTM reuse within the scientific process across 17,718 peer reviewed, open access papers. Our results show that "Biochemistry, Genetics and Molecular Biology" has outpaced other natural scientific fields in PTM reuse, "adaptation" reuse is the most prevalent PTM reuse pattern identified across all natural science fields, and the "testing" stage of the scientific process has been most impacted by PTM integration.

Nicholas M. Synovic, Karolina Ryzka, Alessandra V. Vellucci Solari +3

Computer Science Department, Loyola University Chicago, Chicago, IL, USA · Electrical and Computer Engineering Department, Purdue University, West Lafayette, IN, USA

AI-Assisted Scientific ResearchTransfer LearningScientific ML
AskPDF
Mar 13, 2026

Representation Learning for Spatiotemporal Physical Systems

Machine learning approaches to spatiotemporal physical systems have primarily focused on next-frame prediction, with the goal of learning an accurate emulator for the system's evolution in time. However, these emulators are computationally expensive to train and are subject to performance pitfalls, such as compounding errors during autoregressive rollout. In this work, we take a different perspective and look at scientific tasks further downstream of predicting the next frame, such as estimation of a system's governing physical parameters. Accuracy on these tasks offers a uniquely quantifiable glimpse into the physical relevance of the representations of these models. We evaluate the effectiveness of general-purpose self-supervised methods in learning physics-grounded representations that are useful for downstream scientific tasks. Surprisingly, we find that not all methods designed for physical modeling outperform generic self-supervised learning methods on these tasks, and methods that learn in the latent space (e.g., joint embedding predictive architectures, or JEPAs) outperform those optimizing pixel-level prediction objectives. Code is available at https://github.com/helenqu/physical-representation-learning.

Helen Qu, Rudy Morel, Michael McCabe +4

Flatiron Institute · Princeton University · Université Paris-Saclay, Université Paris Cité, CEA, CNRS, AIM +1

Representation LearningScientific MLSelf-Supervised Time Series Representation LearningJoint-Embedding Predictive Architecture
AskPDF
Mar 11, 2026

Differentiable Thermodynamic Phase-Equilibria for Machine Learning

Accurate prediction of phase equilibria remains a central challenge in chemical engineering. Physics-consistent machine learning methods that incorporate thermodynamic structure into neural networks have recently shown strong performance for activity-coefficient modeling. However, extending such approaches to equilibrium data arising from an extremum principle, such as liquid-liquid equilibria, remains difficult. Here we present DISCOMAX, a differentiable algorithm for phase-equilibrium calculation that guarantees thermodynamic consistency at both training and inference, only subject to a user-specified discretization. The method combines discrete enumeration of feasible phase states with masked softmax aggregation in the backward pass, with the propagation of the true equilibrium state in the forward pass, using a straight-through gradient estimator to enable physics-consistent end-to-end learning of neural \gls{gE}-models. We show that this approach bears analogy to statistical thermodynamics, and we evaluate it on binary liquid-liquid equilibrium data where it outperforms existing surrogate-based methods, while offering a general framework for learning from different kinds of equilibrium data.

Karim K. Ben Hicham, Moreno Ascani, Jan G. Rittig +1

RWTH Aachen University, Process Systems Engineering (AVT.SVT), 52074 Aachen, Germany · Forschungszentrum Jülich GmbH, Institute of Climate and Energy Systems ICE-1: Energy Systems Engineering, 52425 Jülich, Germany · JARA-ENERGY, 52056 Aachen, Germany

Scientific MLDifferentiable ProgrammingNeural Surrogate Modeling
AskPDF
Mar 10, 2026

Two Teachers Better Than One: Hardware-Physics Co-Guided Distributed Scientific Machine Learning

Scientific machine learning (SciML) is increasingly applied to in-field processing, controlling, and monitoring; however, wide-area sensing, real-time demands, and strict energy and reliability constraints make centralized SciML implementation impractical. Most SciML models assume raw data aggregation at a central node, incurring prohibitively high communication latency and energy costs; yet, distributing models developed for general-purpose ML often breaks essential physical principles, resulting in degraded performance. To address these challenges, we introduce EPIC, a hardware- and physics-co-guided distributed SciML framework, using full-waveform inversion (FWI) as a representative task. EPIC performs lightweight local encoding on end devices and physics-aware decoding at a central node. By transmitting compact latent features rather than high-volume raw data and by using cross-attention to capture inter-receiver wavefield coupling, EPIC significantly reduces communication cost while preserving physical fidelity. Evaluated on a distributed testbed with five end devices and one central node, and across 10 datasets from OpenFWI, EPIC reduces latency by 8.9×\times and communication energy by 33.8×\times, while even improving reconstruction fidelity on 8 out of 10 datasets.

Yuchen Yuan, Junhuan Yang, Hao Wan +4

George Mason University, Fairfax, VA, USA · University of North Carolina at Chapel Hill, NC, USA

Scientific MLEdge MLEnergy-Efficient MLPhysics-Informed MLInverse Problems
AskPDF
Mar 8, 2026

Partial Differential Equations in the Age of Machine Learning: A Critical Synthesis of Classical, Machine Learning, and Hybrid Methods

Partial differential equations (PDEs) govern physical phenomena across the full range of scientific scales, yet their computational solution remains one of the defining challenges of modern science. This critical review examines two mature but epistemologically distinct paradigms for PDE solution, classical numerical methods and machine learning approaches, through a unified evaluative framework organized around six fundamental computational challenges. Classical methods are assessed for their structure-preserving properties, rigorous convergence theory, and scalable solver design; their persistent limitations in high-dimensional and geometrically complex settings are characterized precisely. Machine learning approaches are introduced under a taxonomy organized by the degree to which physical knowledge is incorporated and subjected to the same critical evaluation applied to classical methods. Classical methods are deductive -- errors are bounded by quantities derivable from PDE structure and discretization parameters -- while machine learning methods are inductive -- accuracy depends on statistical proximity to the training distribution. This epistemological distinction is the primary criterion governing responsible method selection. We identify three genuine complementarities between the paradigms and develop principles for hybrid design, including a framework for the structure inheritance problem that addresses when classical guarantees propagate through hybrid couplings, and an error budget decomposition that separates discretization, neural approximation, and coupling contributions. We further assess emerging frontiers, including foundation models, differentiable programming, quantum algorithms, and exascale co-design, evaluating each against the structural constraints that determine whether current barriers are fundamental or contingent on engineering progress.

Mohammad Nooraiepour, Jakub Wiktor Both, Teeratorn Kadeethum +1

Faculty of Mathematics and Natural Sciences, University of Oslo, 1047 Blindern, 0316 Oslo, Norway · Department of Mathematics, University of Bergen, Bergen, Norway · AI Lab, Siemens Energy, Orlando, FL 32826, USA +1

PDE SolvingPartial Differential EquationsScientific MLPhysics-Informed ML
AskPDF
Mar 4, 2026

LUMINA: Foundation Models for Topology Transferable ACOPF

Foundation models in general promise to accelerate scientific computation by learning reusable representations across problem instances, yet constrained scientific systems, where predictions must satisfy physical laws and safety limits, pose unique challenges that stress conventional training paradigms. We derive design principles for constrained scientific foundation models through systematic investigation of AC optimal power flow (ACOPF), a representative optimization problem in power grid operations where power balance equations and operational constraints are non-negotiable. Through controlled experiments spanning architectures, training objectives, and system diversity, we extract three empirically grounded principles governing scientific foundation model design. These principles characterize three design trade-offs: learning physics-invariant representations while respecting system-specific constraints, optimizing accuracy while ensuring constraint satisfaction, and ensuring reliability in high-impact operating regimes. We present the LUMINA framework, including data processing and training pipelines to support reproducible research on physics-informed, feasibility-aware foundation models across scientific applications.

Yijiang Li, Zeeshan Memon, Hongwei Jin +7

Argonne National Laboratory, Lemont, IL 60439, USA · Emory University, Atlanta, GA 30322, USA · Sogang University, Seoul, South Korea

Physical Foundation ModelsScientific MLPhysics-Informed MLOptimal Power Flow
AskPDF
Mar 4, 2026

Adaptive Sensing of Continuous Physical Systems for Machine Learning

Physical dynamical systems can be viewed as natural information processors: their systems preserve, transform, and disperse input information. This perspective motivates learning not only from data generated by such systems, but also how to measure them in a way that extracts the most useful information for a given task. We propose a general computing framework for adaptive information extraction from dynamical systems, in which a trainable attention module learns both where to probe the system state and how to combine these measurements to optimize prediction performance. As a concrete instantiation, we implement this idea using a spatiotemporal field governed by a partial differential equation as the underlying dynamics, though the framework applies equally to any system whose state can be sampled. Our results show that adaptive spatial sensing significantly improves prediction accuracy on canonical chaotic benchmarks. This work provides a perspective on attention-enhanced reservoir computing as a special case of a broader paradigm: neural networks as trainable measurement devices for extracting information from physical dynamical systems.

Felix Köster, Atsushi Uchida

Department of Information and Computer Sciences, Saitama University, 255 Shimo-Okubo, Sakura-ku, Saitama City, Saitama, 338–8570, Japan

Adaptive SensingActive PerceptionScientific MLChaotic Time-Series Prediction
AskPDF
Mar 1, 2026

MultiPUFFIN: A Multimodal Domain-Constrained Foundation Model for Molecular Property Prediction of Small Molecules

MultiPUFFIN is a domain-informed multimodal foundation model for predicting thermophysical properties of small molecules, addressing a critical gap in chemical engineering, drug discovery, and materials science. Existing molecular foundation models pretrain on millions of molecules to learn general-purpose representations, but their standard MLP output layers impose no physical constraints, vapor pressure predictions may violate monotonic temperature dependence, and viscosity curves may lack the functional form required by process simulators. Domain-informed approaches that guarantee thermodynamic consistency have remained limited to single properties and small datasets, whereas multimodal foundation models have focused on biological activity rather than thermophysical properties. MultiPUFFIN fills this gap by fusing SMILES sequences, 2D molecular graphs, and 3D conformer geometries through bidirectional cross-modal attention and gated fusion, supplemented by auxiliary encoders for experimental conditions and molecular descriptors. The backbone is pretrained on 500,000 unlabelled PubChem molecules using three complementary self-supervised objectives. A condition-aware refinement stack of five conditioners (temperature, pH, pressure, polymorph, and measurement method) routes each property to a four-head tournament that selects the best-performing thermodynamically informed head for that property. MultiPUFFIN achieves a mean test R2 of 0.784 and outperforms fine-tuned ChemBERTa-2 on all nine properties despite training on roughly 2,000x fewer labeled molecules.

Idelfonso B. R. Nogueira, Carine M. Rebello, Mumin Enis Leblebici +1

Department of Chemical Engineering, Norwegian University of Science and Technology (NTNU), Trondheim, 7034, Norway · Faculty of Industrial Engineering, KU Leuven, Diepenbeek, Belgium · University of Surrey, Guildford, United Kingdom

Molecular Property PredictionMultimodal Foundation ModelsMolecular Representation LearningScientific MLMultimodal Pretraining
AskPDF
Feb 26, 2026

Neural Operators Can Discover Functional Clusters

Operator learning is reshaping scientific computing by amortizing inference across infinite families of problems. While neural operators (NOs) are increasingly well understood for regression, far less is known for classification and its unsupervised analogue: clustering. We prove that sample-based neural operators can learn any finite collection of classes in an infinite-dimensional reproducing kernel Hilbert space, even when the classes are neither convex nor connected, under mild kernel sampling assumptions. Our universal clustering theorem shows that any KK closed classes can be approximated to arbitrary precision by NO-parameterized classes in the upper Kuratowski topology on closed sets, a notion that can be interpreted as disallowing false-positive misclassifications. Building on this, we develop an NO-powered clustering pipeline for functional data and apply it to unlabeled families of ordinary differential equation (ODE) trajectories. Discretized trajectories are lifted by a fixed pre-trained encoder into a continuous feature map and mapped to soft assignments by a lightweight trainable head. Experiments on diverse synthetic ODE benchmarks show that the resulting practical SNO recovers latent dynamical structure in regimes where classical methods fail, providing evidence consistent with our universal clustering theory.

Yicen Li, Jose Antonio Lara Benitez, Ruiyang Hong +3

McMaster University and the Vector Institute, Main St., Hamilton, L8S 4L8, Ontario, Canada · Rice University, Main St., Houston, 77005, Texas, United States

ClusteringUnsupervised ClusteringNeural OperatorsScientific ML
AskPDF
Feb 26, 2026

LUMOS: Democratizing SciML Workflows with L0-Regularized Learning for Unified Feature and Parameter Adaptation

The rapid growth of scientific machine learning (SciML) has accelerated discovery across diverse domains, yet designing effective SciML models remains a challenging task. In practice, building such models often requires substantial prior knowledge and manual expertise, particularly in determining which input features to use and how large the model should be. We introduce LUMOS, an end-to-end framework based on L0-regularized learning that unifies feature selection and model pruning to democratize SciML model design. By employing semi-stochastic gating and reparameterization techniques, LUMOS dynamically selects informative features and prunes redundant parameters during training, reducing the reliance on manual tuning while maintaining predictive accuracy. We evaluate LUMOS across 13 diverse SciML workloads, including cosmology and molecular sciences, and demonstrate its effectiveness and generalizability. Experiments on 13 SciML models show that LUMOS achieves 71.45% parameter reduction and a 6.4x inference speedup on average. Furthermore, Distributed Data Parallel (DDP) training on up to eight GPUs confirms the scalability of

Shouwei Gao, Xu Zheng, Dongsheng Luo +2

Oregon State University · Florida International University · Argonne National Laboratory

Scientific MLAutomated Machine LearningFeature SelectionNeural Network Pruning
AskPDF
Feb 23, 2026

Behavior Learning (BL): Learning Hierarchical Optimization Structures from Data

Inspired by behavioral science, we propose Behavior Learning (BL), a novel general-purpose machine learning framework that learns interpretable and identifiable optimization structures from data, ranging from single optimization problems to hierarchical compositions. It unifies predictive performance, intrinsic interpretability, and identifiability, with broad applicability to scientific domains involving optimization. BL parameterizes a compositional utility function built from intrinsically interpretable modular blocks, which induces a data distribution for prediction and generation. Each block represents and can be written in symbolic form as a utility maximization problem (UMP), a foundational paradigm in behavioral science and a universal framework of optimization. BL supports architectures ranging from a single UMP to hierarchical compositions, the latter modeling hierarchical optimization structures. Its smooth and monotone variant (IBL) guarantees identifiability. Theoretically, we establish the universal approximation property of BL, and analyze the M-estimation properties of IBL. Empirically, BL demonstrates strong predictive performance, intrinsic interpretability and scalability to high-dimensional data. Code: https://github.com/MoonYLiang/Behavior-Learning ; install via pip install blnetwork.

Zhenyao Ma, Yue Liang, Dongxu Li

Xiamen University · University of Tübingen · Xi’an Jiaotong University

Interpretable MLLearning to OptimizeScientific ML
AskPDF
Feb 5, 2026

Clifford Kolmogorov-Arnold Networks

We introduce Clifford Kolmogorov-Arnold Network (ClKAN), a flexible and efficient architecture for function approximation in arbitrary Clifford Algebra spaces. We propose the use of Randomized Quasi-Monte Carlo grid generation as a solution to the exponential scaling associated with higher-dimensional algebras. Our ClKAN also introduces new batch normalization strategies to deal with variable domain input. ClKAN finds application in scientific discovery and engineering, and is validated in synthetic and physics-inspired tasks.

Matthias Wolff, Francesco Alesiani, Christof Duhme +1

Dept. of Computer Science University of Münster Münster, Germany · Data Science Div. NEC Labs Europe Heidelberg, Germany

Kolmogorov-Arnold NetworksNeural Network Approximation TheoryScientific ML
AskPDF
Jan 16, 2026

OpFML: Pipeline for ML-based Operational Inference

Machine learning models for climate and Earth science are becoming increasingly capable, yet model deployment into operational use remains a largely unaddressed challenge: general-purpose model-serving tools, such as MLflow and KServe, assume input data availability at the inference node, while data acquisition, failure handling, and preprocessing are trusted to a separate workflow. We present OpFML: Operational Forecasting with Machine Learning - a configurable pipeline integrating the four steps of operational inference into a single TOML-configured workflow: data consumption, contingency handling, preprocessing, and model inference. By consolidating these steps, OpFML removes the significant boilerplate code required for each new deployment. We demonstrate the pipeline on the operational forecasting of daily fire activity over southern Italy.

Shahbaz Alvi, Giusy Fedele, Gabriele Accarino +3

CMCC Foundation - Euro-Mediterranean Center on Climate Change, Lecce, 73100, Italy · Columbia University, Department of Earth and Environmental Engineering, New York, USA · University of Salento,Department of Innovation Engineering, Lecce, 73100, Italy

Wildfire ForecastingScientific ML
AskPDF