Emulating Non-Differentiable Metrics via Knowledge-Guided Learning: Introducing the Minkowski Image Loss

Filippo Quarenghi; Ryan Cotsakis; Tom Beucler

cs.LGApr 13, 2026

Emulating Non-Differentiable Metrics via Knowledge-Guided Learning: Introducing the Minkowski Image Loss

Authors: Filippo Quarenghi, Ryan Cotsakis, Tom Beucler

Organizations: Faculty of Geosciences and Environment, Expertise Center for Climate Extremes (ECCE), University of Lausanne, Lausanne, VD, Switzerland · Expertise Center for Climate Extremes (ECCE), University of Lausanne, Lausanne, VD, Switzerland

Abstract

The differentiability gap'' presents a primary bottleneck in Earth system deep learning: since models cannot be trained directly on non-differentiable scientific metrics and must rely on smooth proxies (e.g., MSE), they often fail to capture high-frequency details, yielding blurry'' outputs. We develop a framework that bridges this gap using two different methods to deal with non-differentiable functions: the first is to analytically approximate the original non-differentiable function into a differentiable equivalent one; the second is to learn differentiable surrogates for scientific functionals. We formulate the analytical approximation by relaxing discrete topological operations using temperature-controlled sigmoids and continuous logical operators. Conversely, our neural emulator uses Lipschitz-convolutional neural networks to stabilize gradient learning via: (1) spectral normalization to bound the Lipschitz constant; and (2) hard architectural constraints enforcing geometric principles. We demonstrate this framework's utility by developing the Minkowski image loss, a differentiable equivalent for the integral-geometric measures of surface precipitation fields (area, perimeter, connected components). Validated on the EUMETNET OPERA dataset, our constrained neural surrogate achieves high emulation accuracy, completely eliminating the geometric violations observed in unconstrained baselines. However, applying these differentiable surrogates to a deterministic super-resolution task reveals a fundamental trade-off: while strict Lipschitz regularization ensures optimization stability, it inherently over-smooths gradient signals, restricting the recovery of highly localized convective textures. This work highlights the necessity of coupling such topological constraints with stochastic generative architectures to achieve full morphological realism.

Figures & tables

Figure 1: Conceptual overview of the differentiable surrogate training framework. The lack of gradient flow (red connection) prevents the direct optimization of SciML models ( $M_{\theta}$ ) using non-differentiable scientific metrics ( $\mathcal{G}$ ), which typically involve discrete operations like thresholding (step function plot). We overcome this limitation by using knowledge and machine learning techniques to create a differentiable surrogate loss ( $\mathcal{L}$ ) that approximates the target metric. This differentiable loss, with a well-behaved optimization landscape, can then be successfully integrated into the model’s training pipeline (green connection), enabling the direct backpropagation of structurally and physically relevant error signals

Figure 2: (Left) Lipschitz-Bound Feature Extractor: The backbone acts as a hierarchical encoder. Stability is enforced via spectral normalization and residual connections. We use global sum pooling to preserve the extensivity of geometric features. (Right) Geometric Constraint Heads: The Area

\hat{A}(u)

is constructed via integration of a predicted probability density to enforce monotonicity. The perimeter

\hat{P}(u)

is conditioned on the area via the isoperimetric inequality, preventing geometric principles’ violations

Architecture	$\mathcal{M}$ ( $\downarrow$ )	$R^{2}$ ( $\uparrow$ )	$R^{2}_{A}$	$R^{2}_{P}$	$R^{2}_{\mathrm{CC}}$	$\nu_{\text{Iso}}$ (%)
PCR	5.36	0.35	0.46	0.39	0.25	27.0
Analytical Approx.	1.03	0.67	0.99	0.92	0.14	0.3
CNN (Unconstrained)	5.77 $\pm$ 0.02	0.95 $\pm$ 0.02	0.94 $\pm$ 0.02	0.94 $\pm$ 0.01	0.96 $\pm$ 0.02	7.8 $\pm$ 0.3
Lip-CNN (Unconstrained)	5.74 $\pm$ 0.01	0.88 $\pm$ 0.03	0.79 $\pm$ 0.03	0.91 $\pm$ 0.01	0.95 $\pm$ 0.03	5.8 $\pm$ 0.2
Lip-CNN (Constrained)	5.73 $\pm$ 0.01	0.99 $\pm$ 0.01	0.99 $\pm$ 0.01	0.99 $\pm$ 0.01	0.97 $\pm$ 0.02	0.00

Table 1: Comparison of analytical approximation and emulation. Performance is measured via the Minkowski image loss

\mathcal{M}

(lower is better), the coefficient of determination

R^{2}

for the full vector and individual components (Area (A), Perimeter (P), Connected Components (CC)), and the rate of isoperimetric violations (

\nu_{\text{Iso}}

Figure 3: Feature Inversion Results. Qualitative comparison of precipitation fields reconstructed by inverting the target Minkowski vector $\boldsymbol{\gamma}_{\mathrm{gt}}$ for a test sample (left). All architectures recover the storm’s magnitude, while only constrained models generates coherent, smooth intensity gradients characteristic of convective cells (bottom left)

Figure 4: Structural fidelity in precipitation downscaling. The unconstrained UNet produces an amorphous shield, missing localized convective peaks. Integral-geometric constraints (analytical and Lip-CNN) yield only marginal improvement. The stochastic DDIM baseline recovers textural realism and multi-scale variance, confirming the limitations of deterministic optimization

Architecture	$\mathcal{M}$ ( $\downarrow$ )	$MAE$ ( $\downarrow$ )	SAL	RAPS ( $\downarrow$ )
Interp.	3.44	0.06	0.37, -0.01 , 0.06	1.04
UNet	3.21	0.05	0.32, 0.09, 0.15	2.88
UNet + $\mathcal{M}_{\text{Ana}}$	2.74	0.05	0.41, -0.20, 0.03	1.37
UNet + $\mathcal{M}_{\text{Emu, Lip-CNN (Constr.)}}$	3.25	0.05	0.31, -0.29, 0.07	1.78
DDIM	2.96	0.09	-0.08 , 0.26, 0.17	0.97

Table 2: Quantitative comparison of super-resolution deterministic architectures. Performance is measured via the Minkowski image loss (

\mathcal{M}

), the mean absolute error (MAE), the structure-amplitude-location (SAL) metric, and the radially averaged power spectral density (RAPS) error.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Qualitative prediction analysis. Comparison of ground truth (solid) and predicted (dashed) Minkowski functionals. (Top) A well-behaved storm where all models converge. (Bottom) A challenging, fragmented case. Unconstrained models (blue/orange) predict geometrically impossible perimeters (violating monotonicity), whereas the Constrained model (magenta) maintains consistency by design

Figure 6: Spectral Validation. (Left) Radially Averaged Power Spectral Density comparing the energy cascade of the ground truth (black) against the reconstructions. (Right) The spectral ratio

S_{\text{model}}(k)/S_{\text{ref}}(k)

. Ideal performance corresponds to

y=1

Figure 7: Mechanistic evaluation of the surrogate’s sensitivity to localized physical perturbations. (Left) A synthetic multi-peak Gaussian precipitation field. The red dashed box denotes the spatial mask applied to a single convective cell, where the intensity is systematically scaled by an amplitude factor $\alpha$ . (Middle) The spatial attribution map representing the surrogate’s gradient envelope ( $\nabla_{\mathbf{x}}f_{\theta}$ ), revealing a fragmented and noisy receptive field that lacks cohesive internal structure. (Right) The one-dimensional amplitude perturbation landscape for area, perimeter, and connected components (Betti 0). As the targeted cell’s amplitude increases, the exact topological metrics (black) exhibit discrete step transitions when crossing the evaluation threshold. In contrast, the predictions from the constrained Lip-CNN (magenta) remain entirely static across the perturbation range, demonstrating that aggressive Lipschitz regularization over-smooths the latent space and renders the emulator blind to localized structural variations.

Table 10

Neural Surrogate Modeling

Neural surrogate modeling uses learned models to approximate costly simulations, enabling faster prediction, design, control, and inference.

See all 410 papers

Jul 7, 2026

Level-Crossing Density as a Mesh-Free High-Frequency Auxiliary Loss for Implicit Neural Representations

The Minkowski functionals of a field's excursion sets -- area, boundary measure, and Euler characteristic -- describe its level-set morphology; the Euler characteristic is the cheapest handle on topology. We derive smooth Monte-Carlo estimators for all three of a continuous neural field, evaluated at scattered points via the co-area formula and Gauss-Bonnet, using only autodiff: no grid, no complex, no persistence. The estimator is accurate to 1-3% against exact topology in 2D and 3D, and costs about 3 ms per iteration where a persistent-homology (PH) loss on a cubical grid costs 650-1000 ms -- a 250x gap. We establish four design rules without which these losses silently fail: a dense level ladder (invariants are flat in the parameters away from transitions), a

C^2

backbone (ReLU nets hide curvature in kinks), the full Minkowski vector (Euler characteristic alone is an alternating sum, gamed by debris-hole cancellation; pricing perimeter closes the channel), and sampling-scale coverage. In 2D the vector-valued cap is the only method in a controlled comparison that both repairs topology (3/3 seeds) and preserves fidelity -- uniform smoothing repairs at 11-17x the fidelity cost, and the Euler term alone repairs nothing. In 3D neural-SDF fitting, however, a failure mode we believe general to any sampled soft topology objective appears: gradient descent adversarially hides topological noise below the sampling density, where the estimator is blind -- spurious-feature counts are invariant to 4x more samples, and closing the window needs cubically many points, erasing the cost advantage. A grid-based PH baseline, whose complex is the evaluation resolution, solves the same benchmark (

4/9

exact; median

b_1

error 1 vs. ours above

10^4

). The 250x cost of persistence is, at present, the price of having no null space. We release estimators, receipts, and benchmarks.

Gunner Levi Howe

Independent Researcher

Implicit Neural Representations

Ask PDF

May 8, 2026

The Propagation Field: A Geometric Substrate Theory of Deep Learning

Modern deep learning treats neural networks primarily as endpoint functions from inputs to outputs. Inspired by the shift from force to geometry in physics, we ask whether a network should instead be understood through the geometry of its internal propagation. We define a neural propagation field as the collection of hidden-state trajectories and local Jacobian operators across depth. Endpoint losses constrain only the boundary behavior of this field, leaving its interior geometry underdetermined. We show that endpoint-equivalent models can differ by orders of magnitude in trajectory and Jacobian structure, and introduce observable field metrics such as path sensitivity, solver consistency, and trajectory/Jacobian retention. In controlled teacher-flow and PDE systems, endpoint fitting fails to recover the underlying propagation law. In real multi-path tasks, field-aware objectives improve unseen-path generalization, OOD robustness, and calibration when aligned with the observation structure, but can collapse when over-constrained. In continual learning, field-preservation regularization complements replay and distillation: on Split CIFAR-100, DER++ with field preservation improves average accuracy, backward transfer, and field-retention metrics. These results identify propagation-field quality as a measurable and trainable property of neural networks beyond endpoint performance.

Xingrui Gu

1EECS, University of California, Berkeley, Berkeley, CA.

Neural Network Generalization Neural Representation Geometry Jacobian Regularization Neural Network Training Dynamics

Ask PDF

Sep 2, 2025

Differentiable Expectation-Maximisation and Applications to Gaussian Mixture Model Optimal Transport

The Expectation-Maximisation (EM) algorithm is a central tool in statistics and machine learning, widely used for latent-variable models such as Gaussian Mixture Models (GMMs). Despite its ubiquity, EM is typically treated as a non-differentiable black box, preventing its integration into modern learning pipelines where end-to-end gradient propagation is essential. In this work, we present and compare several differentiation strategies for EM, from full automatic differentiation to approximate methods, assessing their accuracy and computational efficiency. As a key application, we leverage this differentiable EM in the computation of the Mixture Wasserstein distance

\mathrm{MW}_2

between GMMs, allowing

\mathrm{MW}_2

to be used as a differentiable loss in imaging and machine learning tasks. To complement our practical use of

\mathrm{MW}_2

, we contribute a novel stability result which provides theoretical justification for the use of

\mathrm{MW}_2

with EM, and also introduce a novel unbalanced variant of

\mathrm{MW}_2

. Numerical experiments on barycentre computation, colour and style transfer, image generation, and texture synthesis illustrate the versatility of the proposed approach in different settings.

Samuel Boïté, Eloi Tanguy, Julie Delon +2

Université Paris Cité, CNRS, MAP5, F-75006 Paris, France · Centre Borelli, CNRS and ENS Paris-Saclay, F-91190 Gif-sur-Yvette, France · CMAP, CNRS, Ecole Polytechnique, Institut Polytechnique de Paris

Gaussian Mixture Models Expectation-Maximization Differentiable Optimization Optimal Transport Wasserstein Distance

Ask PDF

Abstract

Neural Surrogate Modeling

Explore similar work

Level-Crossing Density as a Mesh-Free High-Frequency Auxiliary Loss for Implicit Neural Representations

The Propagation Field: A Geometric Substrate Theory of Deep Learning

Differentiable Expectation-Maximisation and Applications to Gaussian Mixture Model Optimal Transport