Parameter Identifiability

Follow this topic Ask about this topic

Momentum

3 papers in the last four weeks, down 50% on the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 77

Jun 28, 2026cs.RO

On the Identifiability of Aided Inertial Navigation Under Measurement Delays: A Geometric Approach

In aided inertial navigation, measurements from different sensors are often subject to unknown relative time delays. Consider a single aiding sensor whose measurements have an unknown but constant delay relative to the inertial-measurement data stream. We study the identifiability of the delay and the initial navigation state that parameterizes the trajectory. Identifiability depends on both the temporal structure of the aiding measurements and the form of the trajectory itself. Our geometric analysis shows that, for a larger class of uninformative (i.e., degenerate) trajectories than has previously been reported, the delayed measurement model admits a continuous symmetry that prevents unique delay-and-state recovery.

Jonathan Kelly

Parameter Identifiability Inertial Navigation

Jun 26, 2026cs.LG

Disentangling Continuous-Time Latent Dynamics: Identifiability of Latent SDEs via Diffusion Shifts

Causal representation learning for time series has developed strong identifiability results in discrete-time latent causal models, but identifiability in continuous-time latent stochastic differential equation (SDE) models remains largely open. We address this gap using environment-induced shifts in diffusion covariance. We study additive-noise latent SDEs observed through an unknown nonlinear diffeomorphism, with shared drift but environment-specific diffusion covariance. We show that two diagonal diffusion regimes with pairwise distinct coordinate-wise variance ratios identify the latent coordinates up to permutation, coordinate-wise scaling, and a possible constant shift, without any sparsity assumption on the drift. We first prove this result for linear Ornstein-Uhlenbeck systems and then extend it to general additive-noise latent SDEs. Under mild smoothness, the instantaneous drift-Jacobian causal graph is identifiable up to the same permutation. We propose a two-stage estimator for latent disentanglement and optional graph recovery; experiments on synthetic systems confirm the predicted identifiability boundary, and an application to Hardanger Bridge monitoring data illustrates the approach on real sensor trajectories.

Yuanyuan Wang, Wenjie Wang, Haoxuan Li +2

Mohamed bin Zayed University of Artificial Intelligence · The University of Melbourne · Peking University +1

Stochastic Differential Equations Causal Representation Learning Representation Identifiability Parameter Identifiability Causal Structure Learning Causal Identifiability

Jun 25, 2026cs.LG

Recovering Governing Equations from Solution Data: Identifiability Bounds for Linear and Nonlinear ODEs

Learning governing equations from observed solution data is a fundamental challenge in scientific machine learning, yet the theoretical conditions under which a ground-truth ODE can be uniquely and stably identified from multiple solution observations remain largely undeveloped, and no quantitative analysis of the sample complexity of such learning tasks exists in the literature. To address this gap, we introduce the Hausdorff distance on solution sets as the natural metric for comparing differential equations, since it captures the worst-case separation between two equations over all admissible initial conditions and thus encodes the minimax structure of the identification problem. We establish identifiability bounds for governing ODEs across a wide class of structure equations--ranging from linear ODEs to nonlinear classes with Lipschitz (Hölder)-continuous vector fields--characterizing precisely when two distinct equations can be distinguished from solution data. Using this metric, we derive metric entropy estimates for the relevant ODE classes and analyze sample complexity bounds, quantifying how many solution observations are needed to reliably recover the governing equation.

Yang Pan, Helmut Bölcskei

Parameter Identifiability Sample Complexity Nonlinear System Identification Statistical Learning Theory ODE Discovery

Jun 22, 2026cs.LG

The Degeneracy Distillery

When two or more parameters or labels produce similar data, they are degenerate, or hard to distinguish. Degeneracies render both label prediction and inverse problems difficult, since both machine learning algorithms and probabilistic samplers rely on the distinguishability of data and its gradients with respect to parameters. However, identifying degeneracies in physical models or real-world datasets can be elucidating about the choice of model or the underlying process that produces the data. We present the degeneracy distillery, a method that (1) detects and (2) resolves degenerate parameter combinations (a) automatically and (b) symbolically, from parameter-data (or parameter-simulation) pairs alone, through estimation and flattening of the Fisher information matrix. By exploring the information geometry of the likelihood, we characterize degeneracies as an intrinsic property of the physical model, requiring no realised data observation. We demonstrate our approach on a range of synthetic and real-world problems, discovering symbolic coordinate transformations that identify the combinations of parameters of a model which yield independent effects on the data. The resulting coordinates flatten the Fisher information in expectation globally, in contrast to posterior-based methods that flatten only at a single point, and substantially reduce the simulation budget required for downstream neural posterior estimation. In test cases we require up to

10\times

fewer simulations for posterior estimation at matched validation calibration whilst simultaneously gaining physical insight on the system.

T. Lucas Makinen, Deaglan J. Bartlett, Niall Jeffrey +1

Department of Applied Mathematics and Theoretical Physics, University of Cambridge, Wilberforce Rd, Cambridge CB3 0WA, United Kingdom · Imperial Centre for Inference and Cosmology (ICIC), Imperial College London, Prince Consort Road, London SW7 2AZ, United Kingdom · Astrophysics, University of Oxford, Oxford OX1 3RH, United Kingdom +5

Simulation-Based Inference Parameter Estimation Parameter Identifiability

Jun 12, 2026econ.EM

Identification and Inference for Algorithmic Frontiers with Selective Labels

This paper provides identification results to characterize a fairness-accuracy (FA) frontier, and statistical inference tools to test hypotheses and build a confidence set for the FA-frontier, when outcomes are observed only for selected individuals. When the selection process is unrestricted but loss is measured in specific ways, we provide a characterization of the sharp identification region of the FA-frontier. Under an assumption of unconfoundedness conditional on observables (and unrestricted loss functions), we obtain point identification and propose a debiased machine learning estimator, derive its asymptotic distribution, and show how this can be used to carry out inference for the FA-frontier. In work in progress, we extend the partial identification results to a broader class of loss functions.

Yiqi Liu, Francesca Molinari, Amilcar Velez

Department of Economics, Cornell University

Debiased ML Parameter Identifiability Algorithmic Fairness Partial Identification Causal Identifiability Selective Labels

Jun 12, 2026cs.LG

Physics-Informed Discovery of Yield Functions in Plasticity via Convex Neural Representations

Identifying anisotropic yield functions remains challenging since yielding is not directly observed in full-field mechanical measurements, directional calibration can require many loading directions, and selecting an appropriate analytical form is nontrivial. This study proposes a physics-informed framework for discovering yield functions from full-field displacement data and reaction force data, without stress observations, plastic strain measurements, direct yield surface data, or a prescribed parametric yield function. The framework identifies the yield function as a mechanically constrained constitutive component inside elastoplastic stress integration, rather than through direct stress-space supervision. The yield function is represented by a convex neural network that enforces convexity and positive homogeneity of degree one while imposing the assumed tension-compression symmetry, and this neural yield function is trained with a differentiable stress update and a physics-informed force equilibrium loss across multiple loading cases. The proposed framework is validated using finite element (FE) benchmark studies with von Mises, Hill 1948, and Yld2000-2d yield functions, assessing yield contour agreement, displacement-noise sensitivity, identifiability through plastically active stress states, epistemic uncertainty, and polynomial-surrogate deployment. This study provides a mechanics-constrained pathway for discovering anisotropic yield functions from displacement and force data while keeping the identified component within the structure of elastoplastic stress integration.

Hyeonbin Moon, Donghyuk Cho, Jecheon Yu +2

Department of Mechanical Engineering, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, 34141, Republic of Korea · School of Engineering, Deakin University, 75 Pigdons Road, Waurn Ponds, Victoria, 3216, Australia · KAIST InnoCORE PRISM-AI Center, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, 34141, Republic of Korea +1

Parameter Identifiability Physics-Informed ML Inverse Problems

Jun 11, 2026cs.LG

Extracting Governing Equations from Latent Dynamics via Multi-View Contrastive Learning

Identifying latent dynamical systems from noisy, high-dimensional measurements is a central problem at the intersection of representation learning, system identification, and scientific discovery. We present DYSCO, a multi-view temporal contrastive learning algorithm that jointly recovers latent trajectories and the governing dynamics from such observations, by leveraging multiple independent noisy views of the same underlying process to disentangle signal from noise. By parameterizing the dynamics in a structured functional basis, our framework further enables symbolic recovery of the governing equations within an affine gauge. We offer theoretical guarantees for strong identification up to an affine indeterminacy, extending prior identifiability results to the realistic setting of noisy nonlinear observations. Empirically, we demonstrate accurate recovery of both latent trajectories and flow fields across a diverse set of dynamical regimes (e.g., chaotic, oscillatory, and metastable) under both Gaussian and Poisson observation noise, the latter being particularly relevant for neural recordings.

Paolo Muratore, Mackenzie Weygandt Mathis

EPFL

Contrastive Learning Parameter Identifiability Latent Dynamics Modeling Nonlinear System Identification Autonomous Scientific Discovery

Jun 10, 2026cs.LG

Physics-Informed Neural Networks for Chemotherapy Pharmacokinetics: Benchmarking the Clinical Estimator and Exposing Parameter Identifiability

Physics-Informed Neural Networks (PINNs) are an attractive tool for partial-observation problems in biology, where the governing dynamics are known but some compartments cannot be measured. Chemotherapy pharmacokinetics (PK) is a clean instance: drug concentration in plasma is routinely measured, but concentration in tissue -- which determines tumour kill and off-target toxicity -- is not. We benchmark a PINN against the standard clinical baseline (nonlinear least-squares on the analytical biexponential plasma solution, hereafter NLS) and a physics-agnostic neural baseline (a data-only MLP) on two PK problems. On the linear two-compartment problem, NLS is near-optimal; the PINN matches it to within a small constant factor while also producing the tissue curve in a single training pass, whereas the data-only MLP fails on tissue by roughly 10x. On a Michaelis-Menten extension (saturable elimination), the biexponential closed form no longer exists, so NLS is mis-specified and silently returns meaningless rate constants. The PINN instead exposes a deeper fact: the Michaelis-Menten two-compartment model is non-identifiable from plasma alone, and the PINN reports this honestly by converging to a basin with k12 -> 0. Adding two sparse tissue observations largely resolves identifiability: across five seeds the PINN recovers k21 to within 1% of truth and Vmax, Km to within one standard-deviation bar, while k12 moves in the correct direction (0.02 -> 0.82) but remains ~2 sigma below truth -- a recovery the closed-form NLS estimator cannot attempt at all, because its biexponential ansatz describes only plasma. Our claim is not that PINNs beat NLS. It is that PINNs offer a uniform recipe that ties the textbook estimator on the textbook problem, exposes structural identifiability that the textbook estimator hides, and absorbs heterogeneous measurements within a single loss.

Riya Bisht, Dhruv Agarwal

Dynamical Systems Parameter Estimation Parameter Identifiability Physics-Informed ML

Jun 8, 2026cs.LG

Computational Identifiability

Identification conditions describe the computability of a target query or parameter of interest as a function of the type and amount of information available. In causal identification, this information is often expressed in the form of a causal graph, and data are observed or collected for some subset of variables in the graph. Target queries may be for a single effect alone or for a class of effects in a given model. The derivation of an identification algorithm then defines mathematically the process by which the desired causal effect(s) can be uniquely determined, theoretically, in expectation. Identifiability in expectation, or 'theoretical identifiability,' generally assumes asymptotic properties, infinite data, or other mathematically idealized conditions. In this paper, we explore a fundamental distinction between this theoretical, idealized notion of identifiability and a proposed alternative that is computation-bound. The framework we propose - 'computational identifiability' - is to instead define a finite computational search procedure for an empirical estimator. If this process finds an estimator empirically, within a desired error tolerance, then identifiability is satisfied, conditional on the specified assumptions of the search (i.e., a prior distribution over the parameters) and conditional on the search procedure itself. Through several experiments, we demonstrate how this framework allows us to answer fine-grained, practical identification questions, such as identification with small finite samples, with ambiguous graphical criteria, with mixed observational-interventional data, and across counterfactual data and estimands. Code is available at https://github.com/lbynum/metadentify.

Lucius E. J. Bynum, Rajesh Ranganath, Kyunghyun Cho

New York University

Causal Effect Estimation Parameter Identifiability Causal Identifiability Causal Inference

Jun 7, 2026cs.LG

When Are Neural Interaction Discoveries Real? Identifiability, Recoverability, and a Pre-Fit Diagnostic

When a neural time-series model reports that one variable modulates another's effect on a target, is the discovered interaction a property of the data or an artifact of model flexibility? We argue that this is fundamentally a question of identifiability, governed by the geometry of the observed input support rather than by the specific neural architecture. We study the problem in a multiplicative-gating extension of neural additive vector autoregression (GNAVAR), in which source contributions are modulated by other lagged variables. We show that representational capacity is not identifiability: dependent inputs induce leakage between edge-specific interaction terms, and low-dimensional support permits distinct interaction decompositions that agree on the observed data while differing elsewhere. We then prove a population identifiability theorem for normalized minimal GNAVAR decompositions under explicit support conditions, including settings with shared modulators. The theory yields a simple practitioner-facing diagnostic: the effective rank of the joint lag-block covariance predicts, before fitting, whether interaction recovery is feasible for a given candidate set. When the candidate set is unknown, a two-seed stability check provides a practical operational test. The same support condition organizes empirical outcomes into the three states predicted by the theory. Our results show that interaction recoverability depends on support geometry, that effective rank provides a practical pre-fit diagnostic, and that instability across independent fits is a characteristic signature of non-identifiable interaction discovery. The identifiability phenomenon, the support condition, and the instability signature are model-agnostic; GNAVAR is the vehicle that makes them provable.

Valentina Kuskova, Dmitry Zaytsev, Michael Coppedge

Lucy Family Institute for Data & Society · University of Notre Dame · Notre Dame, IN, USA +1

Feature Interaction Modeling Parameter Identifiability Nonlinear System Identification Neural Network Interpretability

Jun 6, 2026stat.ML

Identifiability and Estimation for Unlabeled Finite Mixtures under Marginal Independence

We study component recovery and mixing-matrix estimation from unlabeled finite mixtures whose observable distributions share the same latent components but have unknown mixing weights. The main identifying signal is marginal independence: each component is assumed to be independent on at least one coordinate pair, but no labels, clean component samples, or mixing weights are observed. We first prove a structural result for product components: under linear independence of the univariate marginals, any independent affine combination of the components must coincide with a single component. We then extend this principle to observable mixtures and show that, under full-rank and no-cancellation conditions, marginally independent affine combinations recover the corresponding latent components. When every component is independent on some coordinate pair, all components are identifiable, and the mixing matrix is recoverable under the stated completion conditions. Finally, we propose a Product-Marginal Maximum Mean Discrepancy (PM-MMD) estimator over affine combinations of the observable mixtures and prove uniform convergence and stability under approximate marginal independence. This framework also separates the empirical roles of the assumptions: irreducibility is, in general, not directly testable from the unlabeled mixtures alone, whereas marginal independence yields a candidate-level diagnostic through held-out PM-MMD. Controlled and flow-cytometry experiments show when marginal independence provides a useful recovery signal. In the reported multi-component comparisons, condition-aware representative selection stabilizes PM-MMD and improves recovery relative to clustering, factorization, and pairwise mixture-proportion baselines using the same unlabeled mixtures.

Takafumi Kanamori, Yushi Hirose, Shohei Yamamoto

Department of Mathematical and Computing Science, Institute of Science Tokyo · RIKEN Center for Advanced Intelligence Project

Latent Variable Models Parameter Identifiability

Jun 5, 2026cs.LG

The Geometry of Last-Layer Model Stealing

This paper uses geometry to explain how a machine learning model can be stolen using an already existing well-known method. The author has shown the exact conditions required to perfectly copy the final layer of a transformer network. When looking deeper into the hidden layers the author has explained clear limits. The author has also demonstrated that a hidden network cannot be fully reverse engineered just by looking at the final results. The research clearly maps out what can and cannot be stolen from a model.

Snigdha Chandan Khilar

Independent Researcher.

Parameter Identifiability Model Extraction Attacks

May 31, 2026cs.LG

A Fiber Criterion for Representation Identifiability in Supervised Learning

Supervised learning evaluates predictors through their input-output behavior. When a predictor is implemented as a composition

f=c\circ h

, supervised evidence constrains the composite map

f

but need not determine the representation-head factorization

(h,c)

. This paper formalizes the resulting representation-level identifiability problem: for a class of admissible representation-head pairs, a representation property is identifiable from the induced predictor exactly when it is constant on the fibers of the projection

(h,c)\mapsto c\circ h

, equivalently when it descends to a well-defined property of the predictor. Predictor-preserving augmentation gives a canonical obstruction: auxiliary information can be appended to a representation while the head ignores it, leaving the predictor unchanged but altering properties such as minimality, compression, invariance, equivariance, nuisance information, or semantic accessibility. This construction separates representation identifiability from optimization and finite-sample estimation. Finite-sample diagnostics illustrate, rather than prove, the criterion: exact algebraic witnesses hold the predictor fixed while changing representation diagnostics, and matched-performance Waterbirds models show that different constraints can select different representations at similar supervised performance. The results clarify that representation-level claims require assumptions, objectives, measurements, or inductive biases beyond supervised predictive behavior alone.

Vasileios Sevetlidis

Athena Research Center, Kimmeria Campus, Xanthi, Greece · Democritus University of Thrace, Vas. Sofias Campus, Xanthi, Greece · International Hellenic University, Serres, Greece

Representation Identifiability Parameter Identifiability

May 29, 2026cs.LG

Toward Identifiable Sparse Autoencoders

Recently, sparse autoencoders (SAEs) have emerged as an attractive tool for interpreting and interacting with representations in practical neural networks. While it is common empirical folklore, we also show theoretically that SAEs are highly unstable: different training runs are likely to produce different concept dictionaries and sparse codes. We characterize the model properties that hinder the stability of real-world SAEs, and address each of these problems through minimal changes to the architecture and training procedure. Together, these changes yield two versions of an \textbf{i}dentifiable SAE (iSAE), a variant of the standard TopK SAE with lower reconstruction error and improved stability. We explain this improvement theoretically by connecting SAEs with traditional dictionary learning approaches, and show that the dictionaries learned in practice satisfy an approximate restricted isometry condition, rendering the corresponding sparse codes in those models near-identifiable.

Walter Nelson, Theofanis Karaletsos, Francesco Locatello

Institute of Science and Technology Austria · 2Pyramidal Inc. · and Achira Inc., USA.

Parameter Identifiability Sparse Autoencoders Dictionary Learning Sparse Coding

May 27, 2026cs.CV

Physics from Video: Identifiability of Time-Invariant Second-Order ODEs under Minimal Trajectory Conditions

Bridging the gap between visual realism and physical understanding is a core challenge for video-based world models. We study the structural identifiability of continuous-time physical laws from raw pixels, focusing on whether an encoder-only pipeline can uniquely recover the parameters of second-order linear ODEs. We prove that a level-set slope-coverage condition ensures the learned latent space is locally affine to the true physical state, enabling exact parameter recovery. Our theory provides the first characterization of minimal data requirements across damping regimes, establishing that underdamped systems are identifiable from a single video clip, whereas other regimes require three diverse trajectories. We further introduce a variance-floor regularizer to stabilize the decoder-free objective and prevent latent collapse. Validated on synthetic and real-world data, our approach demonstrates that interpretable physical constants can be reliably estimated from video without the need for compute-intensive pixel reconstruction, ensuring both physical correctness and transparency. Code is available at https://github.com/wenjiewang3/PhysicsFromVideo.

Yuanyuan Wang, Wenjie Wang, Kun Zhang +1

Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, UAE · University of Melbourne, Melbourne, Australia · Carnegie Mellon University, Pittsburgh, USA

Parameter Identifiability Latent Dynamics Modeling System Identification Physical Reasoning in Video ODE Discovery

May 25, 2026stat.ML

When Does LeJEPA Learn a World Model?

A representation that scrambles the true degrees of freedom of the world cannot support reliable planning or compositional generalization. We prove that LeJEPA (alignment plus Gaussian regularization) linearly recovers the world's latent variables from nonlinear observations, a property known as linear identifiability, in a broad class of worlds where latents evolve under stationary, additive-noise transitions. Our main result is that among all such worlds, the Gaussian is the unique latent distribution for which this guarantee holds. The forward direction rests on a spectral decomposition in which each degree of nonlinearity is strictly penalized by alignment, making the linear map the optimum; the converse rules out every non-Gaussian alternative. We further prove an approximate identifiability result where the guarantee degrades gracefully, and show that linear, orthogonal identifiability enables optimal latent-space planning. We validate the theory with experiments ranging from 2D examples to 1024-dimensional latents, including distributional ablations and pixel-based robotic control. Our theory turns an empirically successful recipe into a mathematical guarantee, providing the foundation for building World Models that provably recover the structure of the world.

David Klindt, Yann LeCun, Randall Balestriero

Cold Spring Harbor Laboratory · New York University · Brown University

World Model Learning Representation Identifiability Parameter Identifiability Latent Dynamics Modeling Joint-Embedding Predictive Architecture Model-Based Planning

May 25, 2026stat.ML

From DPPs to $k$ -DPPs: identifiability analysis via spectral decomposition

We study the geometry of determinantal point processes (DPPs) through the spectral decomposition

L=UΛU^{\top}

. The spectrum

Λ

governs the cardinality distribution via elementary symmetric polynomials, while the eigenspace orientation

U

governs the conditional law within each fixed-cardinality stratum. Conditioning on cardinality

k

yields the

k

-DPP, for which the identifiability structure changes fundamentally: the spectral parameter becomes identifiable only up to a common scale, and the eigenspace rotation parameter is identifiable only through squared minors of the eigenvector matrix. We characterize the identifiability gap precisely, via three explicit invariances (scale, sign similarity, and eigenspace rotation) and a dimension-counting theorem showing the existence of additional continuous non-identifiability whenever

\binom{N}{k}<N(N+1)/2

. In contrast, for the full DPP the non-identifiability comes only from the discrete sign similarity.

Hideitsu Hino, Keisuke Yano

The Institute of Statistical Mathematics, 10-3 Midori cho, Tachikawa City, 190-8562, Tokyo, Japan.

Parameter Identifiability Determinantal Point Process

May 24, 2026cs.LG

On the Epistemic Uncertainty of Overparametrized Neural Networks

Epistemic uncertainty is often viewed as a reducible uncertainty that vanishes with increasing data. This perspective implicitly assumes parameter identifiability and equates epistemic uncertainty with predictive variability. In overparametrized neural networks, however, model parameters are typically non-identifiable due to symmetries and redundant representations. As a consequence, substantial parameter uncertainty can persist even when the underlying function is fully identified. In this work, we analyze epistemic uncertainty through the lens of non-identifiability and characterize both discrete and continuous sources of residual uncertainty. Focusing on one-hidden-layer ReLU networks, we thoroughly analyze the resulting posterior structure and validate our theoretical insights through empirical studies.

David Rügamer

Department of Statistics, LMU Munich, Munich, Germany · Munich Center for Machine Learning (MCML), Munich, Germany

Bayesian Neural Networks Parameter Identifiability ReLU Neural Networks Epistemic Uncertainty

May 23, 2026cs.LG

Feature Learning in Wide Neural Networks under $μ$ P: Identifiability and Sparse-Dictionary Decomposition of the Mean-Field Limit

We establish four structural results for feature learning in wide two-layer neural networks under the Maximal Update Parametrization (

μ

P). First, we prove global existence and uniqueness of the mean-field limit of noisy gradient descent under

μ

P, identifying the maximal admissible weight

w^*

on the moment sequence of the initialization as the reciprocal parameter-moment-growth boundary, and hence the largest weighted moment class propagated by the flow. The finite-particle approximation has uniform-in-time squared-Wasserstein rate

O(N^{-1})

. Second, we characterize identifiability of the mean-field limit: two admissible parameter measures induce the same network function in

L^2

exactly when their active components agree modulo the finite-rank realization symmetry of the architecture. The orbit depth

D^*_{\mathrm{orb}}

is separated from the moment-variety depth

D^*_{\mathrm{var}}

. Third, under the Barron-Hermite target condition the active support of the long-time limit measure admits a sparse-dictionary decomposition: it is supported on at most

S^*

atoms modulo finite-rank realization symmetry, with

S^*

bounded by an explicit coefficient-threshold number. Fourth, we derive the total feature-learning-error decomposition into statistical, optimization, propagation-of-chaos, and sparse-residual components, with a target-dependent Hermite/Barron tail replacing any initialization-only residual. The four results are tied together by an architectural identity: the triple

(w^*, D^*_{\mathrm{orb}}, S^*)

-- the maximal admissible weight, the orbit identifiability depth, and the sparse-dictionary depth at which the target is realizable -- is the natural learning cell of the architecture-data pair

(σ, ρ)

. The proofs are self-contained except for standard results from

μ

P and mean-field Langevin theory.

Akmal Xodarev

Independent Researcher · Tashkent, Uzbekistan

Representation Learning Parameter Identifiability Neural Network Optimization Mean-Field Theory

May 22, 2026physics.comp-ph

A differentiable machine learning small-angle X-ray scattering analysis framework for structure elucidation of lipid nanoparticles

Lipid nanoparticles (LNPs) are efficient delivery systems for negatively charged nucleic acids. Their multi-component architecture yields a core-shell structure. Small-angle X-ray scattering (SAXS) is an important characterization technique for LNPs, but recovering internal structure and size distribution from SAXS is an inverse problem with non-unique solutions. Realistic models are often too expensive for systematic exploration. We introduce a machine-learning-accelerated, differentiable framework for SAXS analysis of heterogeneous, polydisperse LNPs. The forward model combines a core-shell particle with a Gaussian random-field interior, a neural surrogate for the monodisperse SAXS map, and a differentiable layer integrating over particle-size distributions. The surrogate reduces prediction cost by four orders of magnitude, while differentiability enables large-scale multi-start fitting and ensemble identifiability analysis. Applied to synthetic and experimental MC3 LNP data, the framework shows that near-identical SAXS fits can arise from distinct parameter modes, with the experimental fits dominated by a trade-off between size-distribution and interior-structure parameters.

Maria Bånkestad, Sandra Barman, Magnus Röding +11

RISE Research Institutes of Sweden, Division Digital Systems, Computer Science, Kista, Sweden · RISE Research Institutes of Sweden, Division Bioeconomy, Food Research and Innovation, Göteborg, Sweden · Sustainable Innovation & Transformational Excellence, Pharmaceutical Technology & Development, Operations, AstraZeneca Gothenburg, Mölndal, Sweden +5

Parameter Identifiability Materials Science Inverse Problems

May 22, 2026cs.LG

Learning Individual Dynamics from Sparse Cross-Sectional Snapshots

Predicting how a dynamical unit evolves over time - how an individual ages, an epidemic spreads, or a physical system degrades - typically requires dense longitudinal tracking. When only extremely sparse or entirely cross-sectional data is available, inferring individualized, continuous-time trajectories is fundamentally ill-posed. Existing methods force a strict compromise: sequence models (e.g. latent ODEs) require dense longitudinal data, while cross-sectional methods (e.g. optimal transport, flow matching-based) map aggregate populations, losing individual dynamics. In this paper, we demonstrate that this dichotomy can be broken. We introduce CADENCE, a principled probabilistic framework that recovers continuous individual trajectories from isolated snapshots by anchoring latent dynamics to static, individual-level contexts. We provide novel identifiability guarantees for single-timepoint trajectory inference. By combining a score-based spatial encoder (bijective Probability Flow ODE) to eliminate diffeomorphic ambiguities with a Soft Mixture-of-Experts (SMoE) router, we show that individual dynamical parameters and routing function are jointly identifiable. Across a suite of benchmarks spanning physical systems to real-world biological data, CADENCE, trained strictly on extremely sparse snapshots with context structure, matches or exceeds the performance of state-of-the-art sequential models trained on dense, full-trajectory data.

Christian Lagemann, Kai Lagemann, Steven L. Brunton +1

Statistics and Machine Learning, German Center for Neurodegenerative Diseases (DZNE), Bonn, Germany · MediaTek Research, London, United Kingdom · Department of Mechanical Engineering & AI Institute in Dynamic Systems, University of Washington, Seattle, United States +1

Parameter Identifiability Latent Dynamics Modeling Trajectory Prediction

May 18, 2026cs.LG

The Symmetries of Three-Layer ReLU Networks

We develop a framework for analyzing parameter symmetries in deep ReLU networks and obtain a complete characterization of the generic parameter fibers for three-layer bottleneck architectures. Our approach provides explicit semi-algebraic descriptions of these fibers and yields a polynomial time algorithm for deciding functional equivalence of two parameters. The symmetries include discrete and continuous transformations arising from layer composition, and depend on whether deeper layers hide or preserve geometric structure from preceding layers. Finally, we show that some of these symmetries induce local conservation laws along gradient flow, while others do not.

Johanna Marie Gegenfurtner, Moritz Grillo, Guido Montúfar

Technical University of Denmark · Max Planck Institute for Mathematics in the Sciences · UCLA and MPI MiS

Parameter Identifiability ReLU Neural Networks

May 15, 2026cs.RO

A Reproducible and Physically Feasible Dynamic Parameter Identification Framework for a Low-Cost Robot Arm

This paper presents a reproducible and physically feasible dynamic parameter identification framework for CRANE-X7, a low-cost robot arm driven by modular smart actuators. To improve practical identifiability, products of inertia are removed according to approximate link symmetry, reducing the rigid-body model from 65 to 39 base parameters. Identification motions are hand-designed from structured single-joint and adjacent-joint primitives under practical joint-range limits. The proposed pipeline combines preprocessing, inverse-dynamics-regressor-based ordinary least squares (OLS), conditional semidefinite-programming (SDP) projection for feasibility recovery, and closed-loop input error (CLIE) refinement. Candidate solutions from 40 structured trajectories are analyzed in a common principal component analysis (PCA) space to select a statistically central representative model. Because statistical centrality alone does not ensure physical acceptability, the selected model is finally screened by an all-pose positive-definiteness audit of the inertia matrix and, when necessary, corrected by a localized post-CLIE SDP rescue step. Experiments show that the parameter cloud becomes progressively more concentrated from OLS to SDP and CLIE, while the final accepted model preserves high predictive accuracy on held-out validation motions. These results demonstrate a practical route to statistically coherent and physically feasible dynamic models for low-cost robot platforms.

Junji Oaki, Koki Yamane, Koki Inami +1

Institute of Systems and Information Engineering, University of Tsukuba, 1-1-1 Tennodai, Tsukuba, Ibaraki 305-8573, Japan

Robot Dynamics Identification Parameter Identifiability Nonlinear System Identification System Identification

May 13, 2026cs.LG

IV-ICL: Bounding Causal Effects with Instrumental Variables via In-Context Learning

The instrumental-variables (IV) setting is standard for partial identification of causal effects when unobserved confounding makes point identification impossible. Existing approaches face methodological bottlenecks: closed-form bound estimands are required -- e.g., Balke-Pearl equations in binary IV -- and even when available, designing accurate estimators requires manual effort tailored to each estimand. While direct Bayesian inference of the causal effects, instead of the bounds, circumvents these challenges, it is often computationally intensive and suffers from high prior sensitivity or under-dispersed posteriors. As a remedy, we introduce IV-ICL, an amortized Bayesian in-context learning method that learns the marginal posterior distribution of the causal effects directly and derives bounds as its quantiles. Unlike standard variational inference that optimizes exclusive KL divergence, amortized Bayesian inference minimizes the expected inclusive KL, a mass-covering objective. We empirically observe that optimizing inclusive KL can recover the entire identified set across diverse data-generating processes, while exclusive-KL (e.g. with variational inference) of the same Bayesian formulation collapses onto a single mode and fails to cover the identified set. We evaluate IV-ICL on synthetic and semi-synthetic IV benchmarks and show it produces intervals that are more reliably valid and more informative compared to efficient semi-parametric, Bayesian, and plug-in baselines, at 20-500x lower inference time. Beyond methodology, we propose a procedure to convert randomized controlled trials into IV benchmarks with provably preserved ground-truth causal effects that enables a more realistic evaluation of partial-identification methods.

Vahid Balazadeh, Hamidreza Kamkari, Medha Barath +2

University of Toronto · Vector Institute · MIT CSAIL +1

Causal Effect Estimation Parameter Identifiability Amortized Inference Partial Identification Causal Identifiability In-Context Learning+2

May 12, 2026cs.LG

Limits of Learning Linear Dynamics from Experiments

Learning governing dynamics from data is a common goal across the sciences, yet it is only well-posed when the underlying mechanisms are identifiable. In practice, many data-driven methods implicitly assume identifiability; when this assumption fails, estimated models can yield spurious predictions and invalid mechanistic conclusions. Classical identifiability guarantees for controlled linear time-invariant (LTI) systems provide sufficient conditions -- controllability and persistent excitation -- but leave open whether identifiability holds when these conditions fail, and which parts of the system remain identifiable without full identifiability. We show that the experimental setup, i.e., the realized initial state and control input, dictates a fundamental limit on the information recoverable from the observed trajectory. We develop a geometric characterization of this limit and derive a closed-form description of all systems consistent with the experimental setup. Crucially, we prove that even when the full system is not identifiable, the restricted dynamics on the subspace reachable by the experiment remain uniquely determined.

Aybüke Ulusarslan, Niki Kilbertus, Nora Schneider

1Technical University of Munich · 2Helmholtz Munich · 3Munich Center for Machine Learning (MCML)

Dynamical Systems Parameter Identifiability System Identification

May 11, 2026eess.SY

Multi-Agent System Identification with Nonlinear Sheaf Diffusion

Local interaction laws governing multi-agent systems can be difficult to recover from trajectory data, even when the dynamics are observed faithfully. In systems governed by a nonlinear sheaf Laplacian -- a generalization of the graph Laplacian accommodating heterogeneous state spaces and asymmetric communication channels -- the coordination law is encoded by edge potential functions whose gradients produce the inter-agent forces. Because trajectory observations record node-state evolution, they expose only the aggregate effect of the edge forces at each node: distinct interaction laws that agree at the node level are indistinguishable from trajectory data alone. We show that the fundamental obstruction to recovery is topological, measured by sheaf cohomology, and that unique recovery from an unconstrained function class is possible if and only if this cohomology vanishes. When the obstruction is nontrivial, we show that recovery within a finite-dimensional parameterized class is possible precisely when a data-dependent information matrix is positive definite. Experiments validate the theory and illustrate that accurate trajectory reproduction need not certify recovery of the underlying interaction law.

Nivar Anwer, Hans Riess, Matthew Hale

Department of Computer Science, Georgia Tech · Department of Electrical & Computer Engineering, Georgia Tech

Parameter Identifiability Nonlinear System Identification Sheaf Theory Multi-Agent Systems

May 11, 2026cs.LG

Identified-Set Geometry of Distributional Model Extraction under Top- $K$ Censored API Access

Modern LLM APIs often reveal only top-

K

logit scores and censor the remaining vocabulary. We study the per-position distribution-recovery limits of this access model. For censoring threshold

τ

, the compatible teacher distributions form an identified set whose total-variation diameter is exactly

U_K=(V-K)\exp(τ)/(Z_A+(V-K)\exp(τ))

, where

Z_A

is the observed partition function. For KL recovery, we give a computable binary-endpoint lower bound and an asymptotically matching small-ambiguity upper bound, with an extension to reference-aware attackers. Experiments on a Qwen3 math-reasoning teacher reveal a layered extraction hierarchy: on-task top-

K

distillation recovers 12% of private capability, full-logit distillation recovers 56% despite 99% KL closure, and generation-based extraction recovers 96%. Top-

K

censoring therefore limits per-position distribution recovery but does not by itself prevent capability extraction, separating fidelity from transfer in prompt-only logit distillation.

Wenhua Nie, ZiCheng Zhu, Jianan Wu +3

National Taiwan University

Parameter Identifiability Partial Identification LLM Security Language Model Distillation

May 5, 2026cs.LG

Model synthesis and identifiability analysis of stiff chemical reaction systems with inVAErt networks

We consider the problem of learning data-driven replicas for stiff systems of ordinary differential equations arising in chemical kinetics that can be evaluated with high computational efficiency. We first focus on training emulators for families of reaction equations under varying reaction rates, using conditional residual networks or long-short term memory architectures. We then apply a recently proposed data-driven framework known as ``inVAErt networks'' to address the ill-posed inverse problem of inferring reaction rates, integration time, and possibly initial conditions from a target set of species concentrations - a problem that has received relatively little attention in the literature. The proposed approach is demonstrated on chemical systems with reversible and irreversible kinetics, spanning 2 to 20 differential equations, 3 to 20 chemical species, and 3 to 25 reaction rate parameters. Relative root mean squared errors produced by the proposed emulators range from

10^{-5}

for lower-dimensional systems to

10^{-4}

and

10^{-3}

for an air pollution model and a hydrogen-air reaction system, respectively. Manifolds of non-identifiable reaction rates recovered by the proposed approach can be analytically verified for simple systems and are consistent with local identifiability analysis in higher dimensions.

Sreejata Dey, Guoxiang Grayson Tong, Jonathan F. MacArt +1

Department of Applied and Computational Mathematics and Statistics, University of Notre Dame · Department of Pediatrics, Stanford University · Department of Aerospace and Mechanical Engineering, University of Notre Dame

Neural Surrogate Modeling Parameter Identifiability Nonlinear System Identification

May 5, 2026cs.LG

Most ReLU Networks Admit Identifiable Parameters

We study the realization map of deep ReLU networks, focusing on when a function determines its parameters up to scaling and permutation. To analyze hidden redundancies beyond these standard symmetries, we introduce a framework based on weighted polyhedral complexes. Our main result shows that for every architecture whose input and hidden layers have width at least two, there exists an open set of identifiable parameters. This implies that the functional dimension of every such architecture is exactly the number of parameters minus the number of hidden neurons. We further show that minimal functional representations can still have non-trivial parameter redundancies. Finally, we establish a generic depth hierarchy, whereby for an open set of parameters the realized function cannot be represented generically by any shallower network.

Moritz Grillo, Guido Montúfar

Max Planck Institute for Mathematics in the Sciences, Leipzig · Departments of Mathematics and Statistics & Data Science, UCLA

Representation Identifiability Parameter Identifiability ReLU Neural Networks Neural Network Expressivity

Apr 27, 2026cs.LG

A Unifying Framework for Unsupervised Concept Extraction

Techniques for concept extraction, such as sparse autoencoders and transcoders, aim to extract high-level symbolic concepts from low-level nonsymbolic representations. When these extracted concepts are used for downstream tasks such as model steering and unlearning, it is essential to understand their guarantees, or lack thereof. In this work, we present a unified theoretical framework for unsupervised concept extraction, in which we frame the task of concept extraction as identifying a generative model. We present a general meta-theorem for identifiability, which reduces the problem of establishing identifiability guarantees to the problem of characterizing the intersection of two sets. As we demonstrate on a range of widely-used approaches, this meta-theorem substantially simplifies the task of proving such guarantees, thus paving the way for the development of new, principled approaches for concept extraction.

Chandler Squires, Pradeep Ravikumar

Machine Learning Department, CMU

Unsupervised Learning Parameter Identifiability Sparse Autoencoders