cs.LGMar 12, 2026
SaveSeparable neural architectures as a primitive for unified predictive and generative intelligence
Organizations: Kevin T. Crofton Department of Aerospace and Ocean Engineering, Virginia Polytechnic Institute and State University, Blacksburg, VA 24060, USA · Department of Mechanical Engineering, Bangladesh University of Engineering and Technology, Dhaka, Bangladesh
Abstract
Intelligent systems across physics, language and perception often exhibit factorisable structure, yet are typically modelled by monolithic neural architectures that do not explicitly exploit this structure. The separable neural architecture (SNA) addresses this by formalising a representational class that unifies additive, quadratic and tensor-decomposed neural models. By constraining interaction order and tensor rank, SNAs impose a structural inductive bias that factorises high-dimensional mappings into low-arity components. Separability need not be a property of the system itself: it often emerges in the coordinates or representations through which the system is expressed. Crucially, this coordinate-aware formulation reveals a structural analogy between chaotic spatiotemporal dynamics and linguistic autoregression. By treating continuous physical states as smooth, separable embeddings, SNAs enable distributional modelling of chaotic systems. This approach mitigates the nonphysical drift characteristics of deterministic operators whilst remaining applicable to discrete sequences. The compositional versatility of this approach is demonstrated across four domains: autonomous waypoint navigation via reinforcement learning, inverse generation of multifunctional microstructures, distributional modelling of turbulent flow and neural language modelling. These results establish the separable neural architecture as a domain-agnostic primitive for predictive and generative intelligence, capable of unifying both deterministic and distributional representations.
Figures & tables
Figure 1: The separable neural architecture (SNA) as a unified primitive for predictive and generative intelligence. Top: The SNA formalises a representational class that constructs high-dimensional mappings by combining lower-arity learnable components (atoms) selected by an interaction tensor. By constraining interaction order and tensor rank, this formalism subsumes generalised additive, quadratic and tensor-decomposed neural models.
Figure 2: Prediction and inversion with a canonical separable neural architecture. a , A schematic of the experimental setup. A laser directed energy deposition machine builds thin-walled structures layer by layer on a stainless steel 304 substrate, whilst an infrared camera records the evolving thermal field during the build. These measurements are subsequently linked to the mechanical response of the material through tensile testing of extracted coupons. b , Predictive performance versus trainable parameters on the Inconel 718 thermal-history dataset. KHRONOS matches or exceeds prior model accuracy on both yield stress (YS) and ultimate tensile strength (UTS) with up to five orders-of-magnitude fewer parameters [ 10 , 11 ] , and three orders fewer than XGBoost [ 12 ] . c , Generative inversion of target mechanical properties to thermal histories. KHRONOS’s lightweight structure enables rapid recovery of entire ensembles of plausible histories consistent with queried YS and UTS targets. Here, 47 plausible thermal histories consistent with a target YS of 399.9MPa were recovered in 47.3ms; 64 consistent with a UTS of 670.4MPa in 39.5ms. The illustrated mean and range of converged trajectories closely match the ground-truth thermal history.
Figure 3: Variational separable neural architectures recover high-dimensional PDE solution manifolds with favourable scaling. a , Spatiotemporal evolution of the field for fixed and . The top and middle rows compare KHRONOS’s predicted solution with the exact, and the bottom shows KHRONOS’s relative error. b , The six-dimensional spatiotemporal-parametric advection-diffusion field learned by KHRONOS. Stacked ( ) spatial slices across time are shown across rotation-diffusivity ( ) parameter space, illustrating the recovery of the entire solution manifold in a single global representation. c , Approximation ( ) error versus trainable parameters for the same system under refinement of rank and resolution . Rank-isolines are connected and colour-coded. Along rank-isolines, errors decrease with resolution at slope before saturating at the rank capacity limit. Across ranks, an efficient frontier emerges (fitted slope in log-log space), sustained across four orders of magnitude in parameter count.
| Feature | Size | Description |
|---|---|---|
| Input | ||
| Voxel Grid | Origin-anchored octant of L-BOM. Binary field. | |
| Outputs | ||
| Normal Stiffness | 3 | Axial ( ). |
| Normal Coupling | 3 | Off-diagonal ( ). |
| Shear Stiffness | 3 | Diagonal ( ). |
Table 1: L-BOM dataset features . Inputs are symmetry-reduced unit cells; outputs comprise the 21-component elastic tensor, volume fraction and permeability.
Figure 4: Bidirectional generative framework and realisation of seamless, multiscale metamaterials. a , Schematic of Janus’s architecture. A three-dimensional convolutional autoencoder encodes a unit cell voxelised microstructure into a 64-dimensional latent space from which it learns to reconstruct them. A separable neural architecture head, similar to KHRONOS, predicts physical properties from the latent. This head is readily inverted to generate a new microstructure given target properties. b , Forward prediction accuracy of Janus on key components of the stress tensor from a held-out test set, demonstrating near-perfect correlation. c , Principal component analysis (PCA) of the latent space coloured by axial stiffness , highlighting the smooth manifold learned by the network. d , Macroscale stiffness targets as prescribed by the cantilever beam model, volumetric rendering of the 40-cell multiscale beam with Janus-designed microstructures, and rendering shaded with local relative errors as determined by FFT homogenisation. e , Beamwise validation of the designed property field. Actual volume fraction exactly tracks the target, and axial stiffness closely agrees across the beam with low relative errors. f , Summary of local stiffness-field and global beam-level metrics. Local errors remain below , whilst global metrics remain below , confirming the intended structural-level response.
Figure 5: Analysis of Leviathan as a foundation model for turbulence across three rollout seeds. a-c , three-dimensional principal components of the embeddings of the entire vocabulary set. a , Leviathan generates a continuous embedding manifold of low intrinsic dimensionality, with the visualised components explaining of the variance. b , A dense Transformer embeds isotropically, explaining only of the variance. c , the isotropic cloud of Leviathan’s embedding space when trained on the unstructured o200k_base tokeniser. Despite the mathematical structure of quantised vorticity, the dense embedding space in b more closely resembles that of an unstructured language tokeniser. d , Quantitative validation of long-horizon – 20 timestep – physical consistency. Leviathan, under four sampling techniques (expectation, top-50, top-5, greedy) outperforms deterministic operators (DeepONet, Fourier neural operator, U-Net) across all metrics (left to right: enstrophy log-ratio error, total spectral energy log-ratio error, spectral slope error, Jensen–Shannon divergence) when controlling for parameters. The dense Transformer is competitive in enstrophy and Jensen-Shannon divergence. e , Evolution of radial energy spectra in time, with Leviathan best maintaining inertial-range statistics. The deterministic operators rapidly fall away from the direct numerical simulation (DNS) ground truth. The Fourier neural operator fades to a constant field in a single step, with flat spectrum. f , Evolution of the probability density function of vorticity. Deterministic models drift catastrophically to a non-physical mean state – a delta distribution – whereas Leviathan preserves the heavy-tailed structure of the chaotic attractor. The dense Transformer retains some structure, avoiding collapse to a mean state.
Figure 6: Autoregressive rollout of turbulent flow over long horizons. A comparative visualisation of two-dimensional incompressible turbulence, with ground truth generated via direct numerical simulation at Mach 0.1 and a Reynolds number of 10 million. The ground truth is compared against state-of-the-art deterministic operators (Fourier neural operator, DeepONet, U-Net), as well as a dense Transformer and Leviathan both sampled via expectation. The operator learners decay to a mean state; the dense Transformer preserves a degree of structure but is fundamentally handicapped by its embedding approach. Leviathan qualitatively tracks the ground truth.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
| Metric | Random Forest [ 50 ] | XGBoost [ 12 ] | MLP | KHRONOS |
| Trainable parameters | 4,261,376 | 84,600 | 5601 | 240 |
| Training time, s | 2.8 | 1.5 | 22 | 0.87 |
| Test MSE | ||||
| Test R 2 | 0.9969 | 0.9990 | 0.9992 | 0.9998 |
Table 2: Regression benchmark on the 8D borehole problem.
| Metric | Random Forest [ 50 ] | XGBoost [ 12 ] | MLP | KHRONOS |
| Trainable parameters | 8,849,208 | 239,564 | 13,569 | 1560 |
| Training time (s) | 5.4 | 5.6 | 66 | 5.1 |
| Test MSE | ||||
| Test | 0.5565 | 0.7312 | 0.8788 | 0.9994 |
Table 3: Regression benchmark on a noisy 20D Sobol-G Function.
Figure 7: Variational solution of the inviscid Burgers’ equation by a CP-class VSNA. The recovered solution (left) and exact solution (centre) show close agreement across the full spatiotemporal domain. Absolute errors remain below 0.02 throughout; the spatially ringing residual structure visible as is consistent with impending shock collapse.
Figure 8: Spline-based adaptive networks (SPAN) applied to closed-loop autonomous navigation. a , Quantitative comparison of goal completion rates on 3 independent trials on each of the 20 routes in the CARLA simulator. SPAN achieves an 83.3% success rate compared to 50% for the MLP. b, c , Representative top-down trajectories illustrating key SPAN successes, ( b , Route 3; c , Route 9). SPAN (blue) tracks the ideal trajectory (dashed grey) whilst the MLP (red) invades lanes early on, terminating the run. d , A highlighted failure mode (Route 12) in which SPAN experiences early lane invasions whilst the MLP navigates to the end.
Figure 9: Generative inversion ablation. a , Comparison of predicted – by the predictive heads – versus actual physical errors. These are as evaluated via fast Fourier transform (FFT) homogenisation for the champion generated unit cells at three representative macroscopic design points, the root ( ), the middle ( ) and the tip ( ). Whilst the unconstrained MLP baseline head predicts low deviations at root and midpoint (hatched orange bars), the FFT solver exposes catastrophic divergence (solid red bars). The MLP faces a more difficult inversion objective, evidenced by the stubbornly large predicted error, almost 5%, at the tip. Janus also hallucinates, but produces significantly higher fidelity unit cells. b , Ensemble statistics of physical errors for the 16 randomly initialised latent seeds per design point, as visualised by a box-and-whiskers plot with overlaid swarm. Because the MLP’s dense and entangled weights create a less convex Jacobian than Janus’s, a higher optimisation floor associated with scattered local minima is established.
Figure 10: Sketch-to-stress forward prediction and response code reconstruction for four representative guitar configurations.Forward prediction (left): a), Input sketch. b) Load application-point map. c) Ground truth normalised stress field. d) Forward Janus prediction. e) Absolute pointwise error. Inverse reconstruction (right): a), Ground truth normalised stress field. b), Stress field reconstructed from the compact 8-dimensional response code . c), Absolute pointwise error. Forward predictions achieve masked MAEs of – , faithfully recovering dominant stress concentrations, hotspot locations, and global field topology across all four configurations. Inverse reconstructions from the response code achieve MAEs of – , preserving principal hotspot location and global stress topology.
More in this topic
Compositional Representation Learning
Compositional representation learning studies how models encode reusable parts and relations to understand combinations they have not seen.
See all 83 papers