The Cost of a Physics Prior Is Bounded by the Ablation Gap

Boris Kriuk

cs.AIAug 21, 2026

The Cost of a Physics Prior Is Bounded by the Ablation Gap

Authors: Boris Kriuk

Organizations: Department of Computer Science and Engineering, The Hong Kong University of Science and Technology (HKUST), Clear Water Bay, Kowloon, Hong Kong SAR, China

Abstract

Shape-constrained and physics-informed learning reports an accuracy cost of enforcing a prior and treats it as a property of the prior. We show it is mostly a property of the free features and the validation split. Let P be the excess risk of restricting a hypothesis class to functions with a shape constraint on features S, and D the excess risk of the ablated model that ignores S. Because a function constant in x_j is both non-decreasing and non-increasing in x_j, the ablated class is contained in the constrained class, so 0 <= P <= D for every risk functional, with no convexity, smoothness, or realizability assumption. Empirically the bound is a sign test: a constrained model must never be beaten by its own ablation. We instantiate it on an ordinal wildfire-severity task (N = 26,681, K = 3) with hard monotone constraints on four meteorological drivers, coordinates left free, and a validation ladder from i.i.d. resampling to 2-degree spatial blocking. Coordinates act as a shield: alone they recover 92.9% of the full model's macro-F1 under spatial blocking, collapsing D from 0.1288 to 0.0427; the same prior costs 0.0473 shielded and 0.3470 unshielded, a ratio of 7.3 with identical physics. Because D is protocol-dependent it does not transfer: coarsening blocks from 1 to 10 degrees drives D from 0.0942 to 0.0050, leaving two configurations unidentifiable a priori. Inversions of the certified nesting bound the pipeline's additive resolution: over 318 comparisons they give a self-calibrating floor of 0.0220 macro-F1, below which no reported price is interpretable, including four cells in our own headline grid. Cost and compliance are independent: the unconstrained model violates the prior at rate 0.48-0.49 while enforcing it costs 0.0473. We give a two-fit screen that rejects unidentifiable experiments before a constrained model is trained.

Figures & tables

Panel of Figure 1 — Fig. 1: Geometry behind Theorem 9 , with the model names of our experimental zoo attached and their measured out-of-fold macro- $F_{1}$ under $2^{\circ}$ spatial blocking. The class $\mathcal{H}^{\perp S}$ of hypotheses that ignore the constrained features $S$ lies inside the shape-constrained class $\mathcal{H}^{\sigma}$ , because a function constant in $x_{j}$ satisfies every sign pattern on $x_{j}$ vacuously. Hence the constrained optimum can never be worse than the ablated optimum, and the price of the constraint is bounded by the ablation gap: here $\mathcal{P}=0.0179\leq\Delta=0.0427$ . The bound holds for arbitrary risk functionals.

Model	Features	Constraint	Role
MAJORITY	—	—	$\mathcal{H}^{\perp D}$ for PHYS*
GEO_ONLY	$G$	none	$\mathcal{H}^{\perp D}$ for MONO* , MIB*
WX_ONLY	$D$	none	$\mathcal{H}$ for PHYS*
MC	$D\cup G$	none	$\mathcal{H}$ for MONO*
MONO_naive	$D\cup G$	$\sigma^{\mathrm{nv}}$ on $D$	$\mathcal{H}^{\sigma^{\mathrm{nv}}}$
MONO_sc	$D\cup G$	$\sigma^{\mathrm{sc}}$ on $D$	$\mathcal{H}^{\sigma^{\mathrm{sc}}}$ ; $c$ -blind arm

TABLE I: Correspondence between the theory of Section III and the fitted models.

D=\{\textit{temp},\textit{rh},\textit{wind},\textit{precip}\}

are the meteorological drivers,

G=\{\mathrm{lat},\mathrm{lon}\}

the geographic shield,

c

the residualized auxiliary channel (insolation).

\sigma^{\mathrm{nv}}=(+1,-1,+1,-1)

is the textbook (naive) sign pattern,

\sigma^{\mathrm{sc}}

the fold-wise screened one.

Tag	Setting	Purpose
ctx	headline: $2^{\circ}$ blocks, gate auto/fold	headline numbers
b1	$1^{\circ}$ blocks ( $1080$ )	shield strongest
b2	$2^{\circ}$ blocks ( $429$ )	determinism check ( $\equiv$ ctx )
b5	$5^{\circ}$ blocks ( $115$ )	block sweep
b10	$10^{\circ}$ blocks ( $40$ )	headroom collapse; $\hat{s}$ probe
closed	gate forced closed	MIB* $\equiv$ MONO_sc test

TABLE II: Configuration grid

\mathcal{K}

. Each row is a complete run of the three-protocol ladder over all eleven arms of Table I , with

5

folds, seed

0

and

B=2000

cluster bootstrap replicates. ctx , b2 and glob coincide bytewise; see Section VIII-I .

Fig. 2: Out-of-fold macro- $F_{1}$ across the validation ladder, headline configuration. The gap between MC and GEO_ONLY is the certified ceiling $\hat{\Delta}(D)$ on the cost of any monotone prior on the drivers ( $0.1288\to 0.1165\to 0.0427$ ); the gap between MC and MONO_sc is the measured price ( $0.0473\to 0.0555\to 0.0179$ ). Theorem 9 requires the MONO_sc curve never to fall below the GEO_ONLY curve, which it does not. Note that WX_ONLY crosses below GEO_ONLY at the spatial rung: under blocking, coordinates alone beat all four drivers together by $+0.0572$ $[+0.0252,+0.0814]$ .

Fig. 3: Paired macro- $F_{1}$ differences with $95\%$ cluster bootstrap intervals, by protocol, headline configuration. PHYS* versus WX_ONLY and MONO* versus MC are the only feature-matched comparisons and hence the only ones that measure a constraint cost (cf. Remark 24 for the MIB arms); the panel against GEO_ONLY is the ablation dominance test of Corollary 11 , where every constrained arm must lie to the right of zero. Note how the PHYS* points sit an order of magnitude further left than the MONO* points against their respective unconstrained references — identical physics, different free feature set.

Protocol	Model	$F_{1}$ (OOF)	$F_{1}$ fold $\pm$ sd	$\hat{\Delta}$	$\hat{\mathcal{P}}$	$\hat{\mathcal{P}}/\hat{\Delta}$	dominance	$\mathrm{VR}_{\mathrm{own}}$	$\mathrm{VR}_{\mathrm{nv}}$	$\mathrm{VR}_{\mathrm{sc}}$	$F_{1}$ per class
iid	MAJORITY	0.3132	0.3132 $\pm$ 0.0000	—	—	—	—	n/a	n/a	n/a	0.000 / 0.000 / 0.940
	GEO_ONLY	0.7028	0.7026 $\pm$ 0.0147	—	—	—	—	n/a	n/a	n/a	0.406 / 0.733 / 0.969
	WX_ONLY	0.7358	0.7356 $\pm$ 0.0116	—	—	—	—	n/a	0.4849	0.4953	0.597 / 0.646 / 0.964
	MC	0.8316	0.8312 $\pm$ 0.0097	0.1288	—	—	—	n/a	0.4795	0.4957	0.700 / 0.816 / 0.979
	MONO_naive	0.7815	0.7813 $\pm$ 0.0160	0.1288	0.0501	0.389	$+$ 0.0787	0.0000	0.0000	0.1572	0.605 / 0.766 / 0.973
	MONO_sc	0.7843	0.7843 $\pm$ 0.0095	0.1288	0.0473	0.367	$+$ 0.0815	0.0000	0.2068	0.0000	0.602 / 0.776 / 0.974

TABLE III: Headline configuration ( ctx :

2^{\circ}

blocks, gate auto/fold), all eleven arms, all three protocols.

\hat{\Delta}

\textsf{MC}-\textsf{GEO\_ONLY}

for arms on

D\cup G

and

\textsf{WX\_ONLY}-\textsf{MAJORITY}

for arms on

D

alone (Remark 23 ).

\hat{\mathcal{P}}

is the negated difference against the feature-matched unconstrained reference. “dominance” is the sign test of Corollary 11 against the arm’s own ablation and must be

\geq 0

\dagger

marks

|\hat{\mathcal{P}}|<\hat{s}=0.0220

, i.e. cells that carry no interpretation.

\ddagger

marks the MIB arms, whose reference is not feature-matched (Remark 24 ).

\mathrm{VR}

columns are end-to-end violation rates against the arm’s own declaration, the textbook prior, and the screened prior.

Config	Protocol	MAJORITY	GEO_ONLY	WX_ONLY	MC	MONO_naive	MONO_sc	PHYS	PHYS_sc	MIB	MIB_fp	MIB_fm	$\hat{\Delta}_{G}$ / $\hat{\Delta}_{M}$
ctx	iid	0.3132	0.7028	0.7358	0.8316	0.7815	0.7843	0.3842	0.3888	0.7888	0.7968	0.7809	0.1288 / 0.4226
	point	0.3132	0.7146	0.7476	0.8311	0.7761	0.7755	0.3888	0.3907	0.8070	0.8023	0.7790	0.1165 / 0.4344
	spatial	0.3132	0.5602	0.5030	0.6029	0.5886	0.5850	0.3679	0.3713	0.6125	0.6159	0.5630	0.0427 / 0.1898
b1	iid	0.3132	0.7028	0.7358	0.8316	0.7815	0.7843	0.3842	0.3888	0.7888	0.7968	0.7809	0.1288 / 0.4226
	point	0.3132	0.7146	0.7476	0.8311	0.7761	0.7755	0.3888	0.3907	0.8070	0.8023	0.7790	0.1165 / 0.4344
	spatial	0.3132	0.5592	0.5686	0.6534	0.6061	0.6078	0.3711	0.3996	0.6182	0.6381	0.5982	0.0942 / 0.2554

TABLE IV: Complete out-of-fold macro-

F_{1}

across the grid. ctx is the headline configuration; b2 and glob reproduce it bytewise and are not repeated. Rows marked

\ast

have an empty exogeneity gate, so

\textsf{MIB*}\equiv\textsf{MONO\_sc}

by Lemma 8 (verified bytewise).

\hat{\Delta}_{G}=\textsf{MC}-\textsf{GEO\_ONLY}

is the certified ceiling for the shielded arms and

\hat{\Delta}_{M}=\textsf{WX\_ONLY}-\textsf{MAJORITY}

for the unshielded arms. Because block size also sets the bootstrap clustering, the iid and point rows of b1 / b5 / b10 reproduce ctx exactly in point estimate and differ only in interval width.

Fig. 4: Fold-wise identification of driver signs, headline configuration: mean central-difference partial effect of the unconstrained model with $95\%$ clustered intervals, three protocols $\times$ five folds per driver. Circles mark folds where the interval excludes zero ( $13$ of $60$ ); crosses mark non-identified signs. Temperature and wind sit above zero, relative humidity below; precipitation straddles zero, which is why its screened sign contradicts the textbook prior in $6$ of $15$ folds. A non-identified sign implies little headroom and hence, by Corollary 18 , little accuracy at stake in choosing it.

Config	Driver	agree	ident.	$n_{+}$	eff.
ctx (headline)	temp	15/15	5/15	15/15	$+0.00126$
	rh	15/15	6/15	0/15	$-0.00129$
	wind	14/15	0/15	14/15	$+0.00146$
	precip	9/15	2/15	6/15	$-0.00028$
b1 ( $1^{\circ}$ )	temp	15/15	9/15	15/15	$+0.00129$
	rh	15/15	8/15	0/15	$-0.00130$

TABLE V: Sign screening across

15

training folds (

3

protocols

\times

5

folds) per configuration. “agree” counts folds where the screened sign matches the textbook prior; “ident.” counts folds whose

95\%

clustered interval excludes zero; “

n_{+}

” counts folds with a positive screened sign; “eff.” is the mean partial effect on

E[y]

. Textbook signs are

\textit{temp}{+}

\textit{rh}{-}

\textit{wind}{+}

\textit{precip}{-}

\textit{solar}{+}

Fig. 5: Violation rate of the unconstrained MC model against the textbook and the screened prior, per driver, under $2^{\circ}$ spatial blocking (overall $0.4825$ naive, $0.4968$ screened, over $45\,071$ active evaluations). Every bar sits near the dashed $0.5$ line, indicating a partial effect whose sign is essentially arbitrary in the unconstrained fit. The only visible gap is precip , whose screened rate ( $0.527$ ) exceeds its naive rate ( $0.463$ ) because the screened sign flips in some folds. By Proposition 21 these rates carry no implication for the price of enforcing the prior, which is $0.0179$ in the same cell.

Fig. 6: Certificate versus deployed pipeline, headline configuration ( $c=$ insolation admitted). Left: violations of each model’s own declared sign pattern, evaluated end-to-end through the residualizer. Every bar in this panel is exactly $0.0000$ — the constrained arms are feasible by construction of the hard split constraints and remain feasible after preprocessing — and the four unconstrained arms declare nothing, hence n/a . The panel is therefore flat by design; it is a passed plumbing test, not a missing plot. Right: violations against the common textbook prior, which is comparable across all arms. MONO_naive and PHYS read $0.0000$ there because the textbook prior is their own declaration.

Config	Protocol	spread	bound	excess
ctx	iid	0.0160	0.0045	$\times$ 0.0114
ctx	point	0.0280	0.0315	—
ctx	spatial	0.0529	0.0275	$\times$ 0.0254
b1	iid	0.0160	0.0045	$\times$ 0.0114
b1	point	0.0280	0.0315	—
b1	spatial	0.0399	0.0104	$\times$ 0.0295

TABLE VI: Auxiliary-channel arms in the five gate-open configurations. “spread” is

\max-\min

over

\{\textsf{MIB},\textsf{MIB\_fp},\textsf{MIB\_fm}\}

; “bound” is

F_{1}(\textsf{MIB})-F_{1}(\textsf{MONO\_sc})

as required by ( 7 ). Cells where the spread exceeds the bound are marked

\times

; the excess equals the sum of the R4 and R3 inversions listed in Table VII . Gate-closed configurations ( closed , drv ) have spread

=

bound

=0.0000

exactly and are omitted.

#	Config	Protocol	Comparison (subclass $\preceq$ superclass)	relation	$F_{1}$ sub	$F_{1}$ super	inversion
1	ctx	iid	$\textsf{MIB\_fm}\preceq\textsf{MONO\_sc}$	R3	0.7809	0.7843	0.0034
2	b1	iid	$\textsf{MIB\_fm}\preceq\textsf{MONO\_sc}$	R3	0.7809	0.7843	0.0034
3	b5	iid	$\textsf{MIB\_fm}\preceq\textsf{MONO\_sc}$	R3	0.7809	0.7843	0.0034
4	b10	iid	$\textsf{MIB\_fm}\preceq\textsf{MONO\_sc}$	R3	0.7809	0.7843	0.0034
5	ctx	iid	$\textsf{MIB\_fp}\preceq\textsf{MIB}$	R4	0.7968	0.7888	0.0080
6	b1	iid	$\textsf{MIB\_fp}\preceq\textsf{MIB}$	R4	0.7968	0.7888	0.0080

TABLE VII: Complete slack ledger: all

18

inverted orderings among the

318

certified comparisons of the grid. “relation” refers to Proposition 22 .

\hat{s}=0.0220

is the maximum of the last column. Intervals are the paired

95\%

cluster bootstrap difference of the subclass against the superclass where the harness instruments that reference; R3 and R4 are not instrumented against MIB in the emitted artefacts and are marked —. Configurations b2 and glob reproduce ctx bytewise and would triple rows #1, #5, #9, #10; they are not listed.

Monotonicity Constraints

See all 16 papers

Oct 27, 2025

Complexity Dependent Error Rates for Physics-informed Statistical Learning via the Small-ball Method

Physics-informed statistical learning (PISL) integrates empirical data with physical knowledge to enhance the statistical performance of estimators. While PISL methods are widely used in practice, a comprehensive theoretical understanding of how informed regularization affects statistical properties is still missing. Specifically, two fundamental questions have yet to be fully addressed: (A) what is the trade-off between soft penalties and hard constraints, and (B) what is the statistical gain of incorporating physical knowledge compared to purely data-driven empirical error minimisation. In this paper, we address these questions for PISL in classes of functions whose convex hull satisfies a small-ball condition, in particular convex classes, under physical knowledge expressed as linear equations by developing appropriate complexity dependent error rates based on the small-ball method. We specialise them to linear classes of finite dimension

q

in which the space of solutions of the equation has dimension

k < q

, and show that (A) the physics-informed estimator attains the error bound of the hard-constrained empirical error minimiser whenever the penalty parameter is above an explicit threshold, and that (B) the penalty improves the error rate from

\sqrt{q/n}

\sqrt{k/n}

under a second moment condition on the noise only, having the effect of a dimensionality reduction. We also quantify how these conclusions degrade when the equation is misspecified, in terms of the excess residual of the target function. This work establishes a theoretical framework for evaluating the statistical properties of physics-informed estimators in such classes, contributing to closing the gap between statistical theory and practical PISL.

Diego Marcondes

Physics-Informed ML Statistical Learning Theory Empirical Risk Minimization

Ask PDF

Sep 18, 2026

Classification with Abstention Under Class-Conditional Error Constraints

We study binary classification with abstention under separate class-conditional error constraints, with the objective of minimizing abstention while keeping both errors below prescribed thresholds. We characterize the distribution-free minimax rate of excess abstention risk, up to logarithmic factors, in terms of the complexity of the hypothesis class and the sample size. To make the framework amenable to computation with models such as neural networks, we introduce surrogate-loss formulations and derive finite-sample guarantees for excess surrogate ambiguity risk. We formulate the resulting learning task as a constrained optimization problem and characterize its computational complexity in the convex setting. Finally, we evaluate our approach on various datasets and compare its performance with a competing method for this problem.

Mohammadreza M. Kalan, Yuyang Deng, Sanaz Hamidi

Univ Rennes, Ensai, CNRS, CREST–UMR 9194, F-35000 Rennes, France · Columbia University, Department of Statistics

Binary Classification Selective Prediction Statistical Learning Theory Minimax Estimation

Ask PDF

Jun 22, 2026

SPADE: Structure-Prior Adaptive Decision Estimation

Physical-structure priors such as conservation laws, Hamiltonian forms, and symmetries can improve scientific machine learning when correct, but can degrade predictions when misspecified. Existing methods usually enforce a chosen structure or tune a soft penalty, without a calibrated rule for deciding whether to impose a prior, how strongly to impose it, which prior to use, or which subset of candidate laws holds. We introduce SPADE, Structure-Prior Adaptive Decision Estimation, a closed-form framework that treats this problem as shrinkage of the structure-violating block of an unconstrained estimator. SPADE uses one exact specification test and one estimand: the test decides whether the prior is supported by data, Stein-unbiased James-Stein shrinkage sets the enforcement strength with an

O(σ^2/n)

oracle guarantee, and a gate commits to the hard prior only when the test certifies it. The same test yields consistent nested structure selection and Benjamini-Hochberg control for subset discovery in non-nested constraint families. Across a linear-subspace prior, a reservoir conservation law, and a nonlinear Hamiltonian prior on Duffing dynamics, SPADE tracks the oracle, beats a neural-network baseline, reduces correct-prior regret from

10.3\%

2.6\%

, matches cross-validation with

1/71

of the solves, selects the correct structure with

100\%

accuracy, and recovers partial laws with controlled false relaxation.

Yifan Wang

Department of Mechanical Engineering, McGill University · QC, H3A 2T7, Canada

Model Selection Scientific ML Physics-Informed ML Specification-Based Testing

Ask PDF

Abstract

Monotonicity Constraints

Explore similar work

Complexity Dependent Error Rates for Physics-informed Statistical Learning via the Small-ball Method

Classification with Abstention Under Class-Conditional Error Constraints

SPADE: Structure-Prior Adaptive Decision Estimation