Measuring Learned Monotone Temporal Aggregation at Matched Admissibility

Yew Lee Tan

cs.LGOct 4, 2026

Measuring Learned Monotone Temporal Aggregation at Matched Admissibility

Authors: Yew Lee Tan

Organizations: This work was conducted in a personal capacity; the views expressed are the author’s own.

Abstract

Risk regulation imposes directional constraints on scores; we adopt their strict per-input form -- the score monotone non-decreasing in every exposure input -- as a normative commitment. Deployed pipelines -- monotone hand-crafted aggregates feeding sign-constrained gradient boosting -- already satisfy it by composition, so constrained-versus-unconstrained comparisons price a guarantee the incumbent has for free. We instead hold admissibility fixed on both sides and measure what learning the aggregation is worth. Our instrument is a recurrent network whose state is classical risk statistics (an exponentially weighted moving average and a high-water mark with learned transforms), monotone by construction in every input and per MC-dropout sample. The central finding, by functional regression, is a subsumption boundary: a learned monotone channel reproduces the geometrically weighted separable family of hand-crafted statistics, one channel per member, to Spearman $ρ\ge 0.996$ , approximates window statistics with measurable ceilings, and fails at consecutivity ( $ρ= 0.924$ ) and time localization (0.628), both structural, and at the exposure floor (0.829), a learnability boundary. One explicit admissible basis repairs each failure (rank correlation 1.000). In or near the separable family, learned and engineered aggregation are substitutes, and the learned channel is never statistically behind at full sample size and specified capacity. Its advantages are incumbent-specific: a committed grid pays up to 0.019 AUC in decay regions it leaves uncovered (the learned channel stays within 0.004 of the strongest engineered consumer at every swept point); the highest-dimensional comparator degrades fastest with scarce data; and beyond the training support, grid-fed tree-ensemble scores go flat while a strictly increasing head keeps ranking. No single incumbent is dominated on all three axes.

Figures & tables

Panel of Figure 1 — Figure 1: The monotone exposure RNN. Each exposure $x_{t}$ enters through learned monotone transforms $g_{1},g_{2}$ (non-negative-weight MLPs with monotone activations, or fixed admissible forms); the two-dimensional state carries a recency-weighted level $e_{t}$ and a high-water mark $m_{t}$ ; a monotone head $F$ maps the final state to the score. Every arrow is a monotone non-decreasing map, so $s$ is monotone in every $x_{t}$ by construction—for every parameter setting and every MC-dropout sample.

Regime	Logit mechanism $\eta$	Targets
A (recency)	$a\,e_{T}^{*}+b$	$g_{1}^{*}=\log(1+x)$
B (peak)	$c\,m_{T}^{*}+d$	$g_{2}^{*}=x$
C (mixed)	$a\,e_{T}^{}+c\,m_{T}^{}+b$	EWMA and HWM jointly
D (threshold stress)	$\lambda\,m_{T}^{*}+b$	$g_{2}^{}=(x-\tau^{})^{+}$
E (diminishing returns)	$a\log(1+e_{T}^{})+c\log(1+m_{T}^{})+b$	saturating EWMA/HWM
F (cumulative burden)	$\gamma\sum_{t}g(x_{t})+b$	cumulative sum

Table 1: Label regimes. Top: core regimes (EWMA/HWM-centric). Bottom: basis-adversarial regimes, each engineered around one admissible basis from the library of Section 3.4 (used by E8). Here

E_{T}^{\mathrm{ex}}=\sum_{t}(x_{t}-\tau^{*})^{+}

B_{T}=\sum_{t}\mathbf{1}[x_{t}>\tau^{*}]

(the breach count of Proposition 1 ),

r_{T}^{*}

is the longest run of consecutive indices with

x_{t}>\tau^{*}

; the shared exceedance threshold

\tau^{*}

of the basis regimes and regime D’s hinge threshold are distinct fixed constants (

3.0

and

10.17

; Table 22 ). The mixture indicator

z\sim\mathrm{Bernoulli}(0.5)

(distinct from the latent stress state

z_{t}

of Section 4 ) is drawn per sequence,

t_{0}=\lfloor T/3\rfloor

C

is a fixed cap (a high population quantile of the EWMA distribution), and

k

is a short recent-window length. Regime N is the steep variant of regime D: its threshold

\tau_{N}

is a fixed constant placed high in the HWM distribution (the population

q_{80}

) with a steep slope

\lambda_{N}

, so the label depends almost entirely on the peak and makes the HWM maximally prominent relative to recency and accumulation summaries. Regime O is N’s extremal mirror: its label depends only on the exposure floor

\min_{t}x_{t}

, the low-water mark of Proposition 1 .

Exp	Claim	Regimes	Protocol	Operators	Models
E1	C1	B, D	clean	rand., adv., coord., P1–P5, MC drop.	core + PGL + dropout var.
E2	C2	A (swept), C	clean	coordinate, probes	proposed, mono. aggregate
E3	C3	A–E	clean	predictive only	core + lattice
E4	C4a	C	dist. shift	predictive; rand., coord.	robustness set
E5	C4b	C	missingness	predictive; rand., coord.	robustness set
E6	C5	A, B, D	clean	shape recovery	proposed (learned $g_{1},g_{2},\alpha$ )

Table 2: Claims-to-experiments map. “Core” denotes the six-model core set of Section 5.1 ; “lattice” the calibrated-lattice comparator; “robustness set” the proposed model, the static monotone aggregate, and the unconstrained RNN and LSTM; “grid set” the grid-baseline set of Section 5.1 including the grid lattices. E10’s “

n

” is the training-size subsampling phase (regimes A and C), which carries the

5\%

-prevalence sensitivity; E11’s regime P is its predictive window-price diagnostic (two-state channel, recent-sum rung, and grid comparators against the Bayes ceiling).

Claim	Experiments	Verdict
C1	E1, E9	Supported in full: zero violations on every operator and per MC-dropout sample, identically at every capacity tier.
C2	E2	Supported: inverted-U decay-sweep gap over the static aggregate, peak $+0.044$ ( $+0.034$ under a tanh aggregate head), closing toward both sweep ends.
C3	E3, E9	Supported: worst paired CI lower bound $-0.0017$ across the ten regime–baseline comparisons; capacity-robust from the small tier upward, with the four-parameter tiny tier’s one miss an expressiveness floor.
C4a	E4	Holds as stated: guarantee exact under every ordering-preserving shift; the static aggregate is the more shift-robust scorer, and the learned channel degrades most gracefully among sequence models.
C4b	E5	Supported, with its boundary quantified rather than asserted.
C5	E6	Supported in asymmetric form: $g_{2}$ and $\alpha$ identifiable regardless of parameterization, $g_{1}$ given a non-convex monotone activation.

Table 3: Claim-by-claim verdicts, in the wording of the primary results subsections.

	Regime B				Regime D
Model	Rand.	Adv.	Coord.	Probes	Rand.	Adv.	Coord.	Probes
Monotone exposure RNN	$0$	$0$	$0$	$0$	$0$	$0$	$0$	$0$
Monotone exposure RNN (MC)	$0$	$0$	$0$	$0$	$0$	$0$	$0$	$0$
Generic monotone RNN	$0$	$0$	$0$	$0$	$0$	$0$	$0$	$0$
Monotone aggregate	$0$	$0$	$0$	$0$	$0$	$0$	$0$	$0$
Unconstrained RNN + PGL	$6.3\%$	$30.7\%$	$34.3\%$	$35.7\%$	$28.4\%$	$48.3\%$	$43.8\%$	$49.0\%$

Table 4: E1 monotonicity violation rates by operator family (pooled over five seeds; per-seed budgets:

2{,}500

random,

150

adversarial,

7{,}200

coordinatewise, and

{\sim}3{,}000

structured-probe pairs per regime, the probe counts being data-dependent by construction, Appendix C ). Monotone-by-construction models (top block) record zero violations on every operator; penalty-trained (PGL) baselines reduce but do not eliminate violations; unconstrained models violate on roughly a third to nearly two-thirds of non-random tests. The MC-dropout variant is additionally violation-free for each of its

200

sampled dropout sub-networks.

Figure 2: E2 decay sweep on regime A (five seeds; bands and bars are

\pm 1

std). Left: test AUC of the recurrent monotone model and the static monotone aggregate as the generating decay

\alpha^{*}

varies. Right: the paired AUC gap forms the predicted inverted-U, peaking at

+0.044

\alpha^{*}{=}0.8

and closing toward both endpoints where the generating EWMA degenerates to static summaries (the last value at

\alpha^{*}{\to}0

, a saturating sum at

\alpha^{*}{\to}1

); the dashed line marks the regime-C control gap.

Decay $\alpha^{*}$	Monotone exposure RNN	Monotone aggregate	Gap $\Delta$ AUC
0.10	$0.8482\pm 0.0060$	$0.8464\pm 0.0067$	$0.0018\pm 0.0014$
0.30	$0.8461\pm 0.0060$	$0.8373\pm 0.0064$	$0.0088\pm 0.0024$
0.50	$0.8442\pm 0.0056$	$0.8218\pm 0.0044$	$0.0224\pm 0.0036$
0.60	$0.8438\pm 0.0061$	$0.8116\pm 0.0041$	$0.0322\pm 0.0039$
0.70	$0.8443\pm 0.0045$	$0.8035\pm 0.0023$	$0.0408\pm 0.0046$
0.80	$0.8436\pm 0.0035$	$0.7997\pm 0.0020$	$0.0439\pm 0.0030$

Table 5: E2 AUC and paired gap by generating decay

\alpha^{*}

on regime A, with the regime-C control (mean

\pm

std over five seeds; the gap is the per-seed paired difference). Both models record zero monotonicity violations at every grid point.

Regime	Mono. exp. RNN	Gen. mono. RNN	Mono. agg.	Lattice agg.	Unc. agg.	Unc. RNN	Unc. LSTM
A	$0.8397\pm 0.0043$	$0.8308\pm 0.0042$	$0.8120\pm 0.0056$	$0.8185\pm 0.0047$	$0.8212\pm 0.0054$	$0.8396\pm 0.0040$	$\mathbf{0.8402\pm 0.0041}$
B	$\mathbf{0.7540\pm 0.0087}$	$0.7302\pm 0.0056$	$0.7538\pm 0.0089$	$0.7534\pm 0.0086$	$0.7533\pm 0.0084$	$0.7501\pm 0.0083$	$0.7516\pm 0.0087$
C	$\mathbf{0.8535\pm 0.0045}$	$0.8461\pm 0.0038$	$0.8432\pm 0.0050$	$0.8450\pm 0.0046$	$0.8450\pm 0.0051$	$0.8519\pm 0.0046$	$0.8531\pm 0.0048$
D	$\mathbf{0.7142\pm 0.0109}$	$0.7061\pm 0.0070$	$0.7129\pm 0.0089$	$0.7139\pm 0.0110$	$0.7139\pm 0.0091$	$0.7105\pm 0.0123$	$0.7136\pm 0.0102$
E	$\mathbf{0.8757\pm 0.0033}$	$0.8635\pm 0.0031$	$0.8709\pm 0.0045$	$0.8709\pm 0.0040$	$0.8710\pm 0.0045$	$0.8746\pm 0.0038$	$0.8756\pm 0.0035$

Table 6: E3 test AUC (mean

\pm

std over five seeds) for the core model set plus the calibrated-lattice comparator on regimes A–E; best per regime in bold. Column abbreviations: monotone exposure RNN, generic monotone RNN, monotone aggregate, calibrated lattice over the same aggregates, unconstrained aggregate, unconstrained RNN, unconstrained LSTM. The constrained exposure RNN matches or exceeds the unconstrained sequence baselines in every regime; log loss, Brier, and ECE are reported in Appendix E .

	vs. Unc. RNN		vs. Unc. LSTM
Regime	$\Delta$ AUC	95% CI	$\Delta$ AUC	95% CI
A	$+0.0001$	$[-0.0006,+0.0009]$	$-0.0004$	$[-0.0009,+0.0000]$
B	$+0.0038$	$[+0.0022,+0.0053]$	$+0.0023$	$[+0.0013,+0.0033]$
C	$+0.0016$	$[+0.0008,+0.0026]$	$+0.0003$	$[-0.0002,+0.0009]$
D	$+0.0037$	$[-0.0003,+0.0078]$	$+0.0005$	$[-0.0017,+0.0025]$
E	$+0.0010$	$[+0.0000,+0.0022]$	$+0.0001$	$[-0.0006,+0.0008]$

Table 7: E3 paired AUC deltas: monotone exposure RNN minus each unconstrained sequence baseline, per regime. Deltas are computed within (regime, seed) cells on the shared test split; intervals are two-level paired bootstrap

95\%

CIs over the stored per-example predictions (resampling seeds, then test examples on shared indices). Every lower bound clears the

-0.01

non-inferiority margin prespecified for the capacity study (E9).

Figure 3: E4 test AUC under severity sweeps of the four shift families on regime C (five seeds; bands are

\pm 1

std). The static monotone aggregate is flat under input-displacing shifts (exactly under scale, within

0.003

under upper tail); the constrained recurrent model degrades most gracefully among sequence models; mixture contamination (bottom right) degrades all models together: its selective per-row displacement destroys cross-sequence score comparability (see text).

Model	Baseline	Scale	Upper tail	Spike freq.	Mixture	Total
Monotone exposure RNN	0	0	0	0	0	0
Monotone aggregate	0	0	0	0	0	0
Unconstrained RNN	1,302	23,758	15,960	4,083	6,493	51,596
Unconstrained LSTM	1,182	24,015	17,152	4,408	5,672	52,429

Table 8: E4 monotonicity violations by shift family (counts summed over severities and five seeds, identical test budgets across models). Monotone models remain at zero under every shift; unconstrained violations concentrate where inputs are displaced furthest from the training support.

Figure 4: E5 test AUC under MCAR (top) and contiguous-block (bottom) missingness on regime C, by imputation (five seeds and three mask draws; bands are

\pm 1

std). The constrained model degrades gracefully and keeps the best or tied-best AUC across the grid; under zero imputation, lengthening blocks erode its advantage over the static aggregate—the structural caveat of Section 7 —while forward-fill preserves it.

Mechanism	Imputation	worst $\Delta\Delta$ AUC	worst CI 95 hi	pass	worst $\Delta\Delta$ AUC	worst CI 95 hi	pass
		vs. unconstrained (RNN, LSTM)			vs. monotone aggregate
MCAR	zero	$+0.0001$	$+0.0015$	10/10	$+0.0007$	$+0.0017$	5/5
MCAR	forward fill	$-0.0001$	$+0.0006$	10/10	$+0.0001$	$+0.0006$	5/5
MCAR	mean	$+0.0010$	$+0.0017$	10/10	$+0.0010$	$+0.0017$	5/5
Block	zero	$-0.0009$	$+0.0011$	8/8	$+0.0111$	$+0.0121$	2/4
Block	forward fill	$-0.0004$	$+0.0003$	8/8	$+0.0006$	$+0.0014$	4/4

Table 9: E5 paired non-inferiority summary:

\Delta\Delta

AUC

=

degradation(monotone)

-

degradation(baseline) with two-level paired bootstrap

95\%

CIs (seeds

\times

examples, mask draws averaged); a condition passes when the CI upper bound is at most

0.005

. Cells show the worst (largest) mean and CI upper bound across missingness levels and baselines in each group. All

54

contrasts against the unconstrained baselines pass; the only failures are zero-imputed blocks of length

6

and

8

against the static monotone aggregate.

Figure 5: E6 recovered transforms (seed-averaged) against the generating shapes; curves are min–max normalized to

[0,1]

on the evaluation grid because the monotone head absorbs affine scale, so only the shape is identified. (a) The concave

g_{1}^{*}=\log(1+x)

of regime A is recovered by the non-convex activations (tanh, sigmoid) while the convex activations (softplus, ReLU) collapse onto a near-linear fit. (b, c) The peak transforms of regimes B and D are recovered by all activations on the upper input range; in (b) the recovered curves flatten at low

x

because the

\max

channel constrains

g_{2}

only where per-sequence maxima concentrate, leaving the low-

x

region unidentified—the same affine-scale caveat quantified by the RMSE column of Table 10 .

	$g_{1}$ , regime A ( $\log$ )		$g_{2}$ , regime B (identity)		$g_{2}$ , regime D (hinge)		$\hat{\alpha}$ , regime A
Activation	corr.	RMSE	corr.	RMSE	corr.	RMSE	( $\alpha^{*}=0.90$ )
tanh	$0.997\pm 0.003$	$0.036\pm 0.018$	$0.993\pm 0.003$	$0.086\pm 0.016$	$0.998\pm 0.000$	$0.021\pm 0.002$	$0.899\pm 0.003$
sigmoid	$0.989\pm 0.005$	$0.070\pm 0.017$	$0.992\pm 0.002$	$0.089\pm 0.007$	$0.985\pm 0.002$	$0.088\pm 0.006$	$0.897\pm 0.003$
softplus	$0.939\pm 0.000$	$0.247\pm 0.000$	$0.989\pm 0.002$	$0.115\pm 0.008$	$0.993\pm 0.001$	$0.055\pm 0.005$	$0.894\pm 0.003$
relu	$0.940\pm 0.000$	$0.244\pm 0.000$	$0.990\pm 0.002$	$0.115\pm 0.009$	$0.998\pm 0.003$	$0.025\pm 0.015$	$0.894\pm 0.003$

Table 10: E6 shape-recovery metrics (mean

\pm

std over five seeds): correlation and RMSE between the learned and generating transforms on a common normalized grid, and the learned decay on regime A. All activations recover

g_{2}

and

\alpha

; only the non-convex activations recover the concave

g_{1}

, and correlation alone would mask the failure that RMSE exposes.

Regime	Viol. / tests	min	median	max	spread	seed std
		AUC over the 48 configurations
A	$0$ / $2{,}364{,}000$	$0.8052$	$0.8316$	$0.8404$	$0.0351$	$0.0042$
B	$0$ / $2{,}364{,}000$	$0.7513$	$0.7538$	$0.7540$	$0.0027$	$0.0087$
C	$0$ / $2{,}364{,}000$	$0.8381$	$0.8503$	$0.8537$	$0.0156$	$0.0043$
D	$0$ / $2{,}364{,}000$	$0.7137$	$0.7141$	$0.7144$	$0.0007$	$0.0109$

Table 11: E7 ablation grid: pooled monotonicity violations and the distribution of seed-averaged AUC across the 48 admissible configurations, per regime. Every configuration–regime pair is violation-free. The spread column, read against the mean within-configuration seed std, shows that on the peak-driven regimes B and D the design choice matters less than the random seed.

Knob	Setting	A	B	C	D
$g_{1}$	identity	$0.8231$	$0.7536$	$0.8471$	$0.7141$
	$\log$	$0.8312$	$0.7534$	$0.8495$	$0.7141$
	monotone MLP	$0.8310$	$0.7540$	$0.8494$	$0.7141$
$g_{2}$	identity	$0.8288$	$0.7539$	$0.8491$	$0.7141$
	hinge	$0.8288$	$0.7539$	$0.8491$	$0.7141$
	power	$0.8273$	$0.7530$	$0.8477$	$0.7140$

Table 12: E7 marginal AUC per knob setting, averaged over the rest of the grid and five seeds. The shape of

g_{2}

is immaterial even on the peak-driven regimes B and D; the only material knobs are the recency channel’s

g_{1}

and

\alpha

on the recency-driven regimes A and C, where

\log

and the learned decay emerge as mild optima.

Regime	Target basis	Frozen base	Frozen $+$ target	Gap	Learned base	Residual
F (cumulative burden)	cumulative sum	$0.805\pm 0.007$	$0.832\pm 0.008$	$+0.027$	$0.838\pm 0.008$	$+0.007$
G (persistence)	longest run	$0.769\pm 0.003$	$0.833\pm 0.005$	$+0.064$	$0.806\pm 0.005$	$-0.027$
J (breach count)	breach count	$0.799\pm 0.005$	$0.849\pm 0.006$	$+0.051$	$0.849\pm 0.006$	$+0.000$
N (peak prominence)	HWM	$0.787\pm 0.007$	$0.858\pm 0.009$	$+0.071$	$0.858\pm 0.009$	$-0.000$
O (exposure floor)	LWM	$0.640\pm 0.009$	$0.783\pm 0.008$	$+0.143$	$0.774\pm 0.009$	$-0.009$
K (early warning)	early-window EWMA	$0.667\pm 0.007$	$0.761\pm 0.008$	$+0.094$	$0.689\pm 0.007$	$-0.072$

Table 13: E8 necessity ladder on the boundary-probing regimes (AUC, mean

\pm

std over five seeds). The frozen arm fixes the EWMA/HWM channels to their literal fixed statistics, so adding the engineered target basis measures basis-level necessity (Gap, paired per seed); the base subset is frozen EWMA+HWM (M3f) for F, G, J, O, and K, and frozen EWMA alone (M1f) for N, and the augmented subset is the smallest frozen subset containing the target: F

\to

M6f, G

\to

M5f, J

\to

M4f, N

\to

M3f, O

\to

M8f, and K

\to

M9f (the early-window EWMA rung; the basis is available to the ablation arms but kept out of the Phase I/II pool). Residual is the learned arm over the base subset minus the frozen-plus-target rung:

{\approx}0

means the learnable

(g_{1},\alpha)

channel recovers the basis without being given it (F, J, N), negative means the gap stays open in the learned arm without the explicit basis (G; K, where the learned channel cannot realize the vanishing weight profile; and marginally O). The F and G Gap columns include the breach count’s pass-through increment, and each explicit rung sits at its regime’s data-level ceiling; the decomposition and the paired rung-minus-Bayes deltas are in the text. The derived Gap and Residual columns are computed from unrounded per-seed means, so they can differ from the difference of the rounded operands by one unit in the final digit.

Basis subset	Learned arm	Frozen arm
M1 (EWMA)	$0.777\pm 0.007$	$0.769\pm 0.006$
M2 (HWM)	$0.755\pm 0.007$	$0.755\pm 0.007$
M3 (EWMA+HWM)	$0.776\pm 0.007$	$0.776\pm 0.007$
M4 (M3 + breach count)	$0.777\pm 0.007$	$0.779\pm 0.007$
M5 (M4 + longest run)	$0.777\pm 0.007$	$0.779\pm 0.007$
M6 (M3 + cumulative + breach count)	$0.779\pm 0.006$	$0.779\pm 0.007$

Table 14: E8 Phase IV cross-regime generalization: AUC after training on regime C, averaged over the held-out adversarial regimes evaluated for every subset (nine; mean

\pm

std over five seeds). The full subset holds at most a lean over EWMA-alone (per-seed paired difference

+0.0003

+0.0024

, positive in every seed), HWM-alone transfers worst, and the frozen arm matches the learned arm throughout; the floor subset M8 transfers identically to M3 and the early-window subset M9 within

0.002

of it, as expected when the training label carries no floor or early-window signal.

Size	Regime	params	AUC	viol.	params	AUC	viol.	$\Delta$ AUC	CI 95	verdict
		Monotone exposure RNN			Unconstrained LSTM			paired non-inferiority
tiny	A	4	$0.8309\pm 0.0042$	0	361	$0.8402\pm 0.0042$	170	$-0.0093$	$[-0.0105,-0.0078]$	fail
tiny	C	4	$0.8513\pm 0.0045$	0	361	$0.8525\pm 0.0044$	1182	$-0.0012$	$[-0.0020,-0.0004]$	pass
small	A	84	$0.8403\pm 0.0044$	0	1,233	$0.8401\pm 0.0044$	53	$+0.0002$	$[-0.0002,+0.0005]$	pass
small	C	84	$0.8536\pm 0.0045$	0	1,233	$0.8528\pm 0.0047$	1115	$+0.0008$	$[+0.0001,+0.0014]$	pass
medium	A	548	$0.8397\pm 0.0043$	0	12,961	$0.8402\pm 0.0041$	6	$-0.0004$	$[-0.0009,+0.0000]$	pass

Table 15: E9 capacity sweep (mean

\pm

std over five seeds): parameter count, AUC, and pooled violation totals for the monotone exposure RNN and the unconstrained-LSTM comparator, with the paired

\Delta

AUC (monotone minus LSTM), its two-level bootstrap 95% confidence interval, and the non-inferiority verdict (pass iff the lower bound is at least

-0.01

). The only failure is the four-parameter tiny tier on the concave regime A, an expressiveness floor that a cap-raising diagnostic shows is not an artifact of the training budget.

Engineered comparators
	G (persistence)	K (early warning)	O (exposure floor)
Practice grid (constrained GBM)	$0.800\pm 0.005$	$0.689\pm 0.006$	$0.682\pm 0.010$
Steelman grid (constrained GBM)	$0.800\pm 0.005$	$0.689\pm 0.007$	$0.666\pm 0.010$
Raw lags (constrained GBM)	$0.823\pm 0.005$	$0.760\pm 0.008$	$0.779\pm 0.008$
Generic monotone recurrences
Fixed-decay bank (ReLU)	$0.764\pm 0.003$	$0.617\pm 0.006$	$0.640\pm 0.009$

Table 16: E10 boundary regimes: test AUC (mean

\pm

std over five seeds, shared splits) for every comparator class on the three E8 boundary regimes. The engineered comparators are the constrained-LightGBM arms of Section 5.1 ; the generic monotone recurrences are the fixed-decay accumulator banks (ReLU and tanh components) and the weight-constrained Elman form, at the default tier; the two-state rows restate, on the same splits, the E8 ladder’s learned base and its frozen repair rung—run length for G, early-window EWMA for K, LWM for O (Table 13 ). Each repair rung sits at the per-seed test-split Bayes ceiling, and among the monotone models only the aggregation-free raw-lags tree crosses the separable-aggregation boundary.

Model	A	B	C	D	E	params	feat.
Mono. exp. RNN	$\mathbf{0.8397\pm 0.0043}$	$\mathbf{0.7540\pm 0.0087}$	$\mathbf{0.8535\pm 0.0045}$	$\mathbf{0.7142\pm 0.0109}$	$0.8757\pm 0.0033$	548	–
Mono. agg.	$0.8120\pm 0.0056$	$0.7538\pm 0.0089$	$0.8432\pm 0.0050$	$0.7129\pm 0.0089$	$0.8709\pm 0.0045$	241	5
Practice grid	$0.8345\pm 0.0040$	$0.7538\pm 0.0088$	$0.8510\pm 0.0045$	$0.7141\pm 0.0109$	$0.8758\pm 0.0037$	353	12
Steelman grid	$0.8339\pm 0.0041$	$0.7537\pm 0.0088$	$0.8515\pm 0.0044$	$0.7140\pm 0.0109$	$0.8760\pm 0.0035$	337	11
Practice lattice	$0.8360\pm 0.0040$	$0.7537\pm 0.0086$	$0.8519\pm 0.0046$	$0.7140\pm 0.0110$	$0.8752\pm 0.0036$	4,384	12
Steelman lattice	$0.8352\pm 0.0046$	$0.7536\pm 0.0090$	$0.8513\pm 0.0046$	$\mathbf{0.7142\pm 0.0112}$	$\mathbf{0.8761\pm 0.0037}$	2,312	11

Table 17: E10 parity ladder (AUC, mean

\pm

std over five seeds) on regimes A–E with the parsimony account: trainable parameter count (for the GBMs, total leaf values of the fitted ensemble), engineered-feature count, and pooled monotonicity violations. Bold marks the best mean AUC per regime, with ties at the printed precision bolded jointly. The learned channel posts the best mean AUC on regimes A and C, where the recency channel carries signal, and on the peak regime B, with no engineered features and the smallest parameter-plus-feature pipeline of any sequence-aware comparator (the grid-MLP heads themselves are smaller, at

337

–

353

parameters, but consume

11

–

12

engineered columns); D and E are ties near the grid point, the expected outcome.

	vs. Steelman GBM		vs. Raw-lags GBM		vs. Practice GBM
Regime	$\Delta$ AUC	95% CI	$\Delta$ AUC	95% CI	$\Delta$ AUC	95% CI
A	$+0.0037$	$[+0.0028,+0.0045]$	$+0.0013$	$[+0.0002,+0.0025]$	$+0.0029$	$[+0.0019,+0.0038]$
B	$+0.0011$	$[+0.0005,+0.0017]$	$+0.0065$	$[+0.0040,+0.0092]$	$+0.0010$	$[+0.0004,+0.0017]$
C	$+0.0018$	$[+0.0011,+0.0023]$	$+0.0040$	$[+0.0029,+0.0051]$	$+0.0017$	$[+0.0010,+0.0025]$
D	$-0.0001$	$[-0.0022,+0.0020]$	$+0.0019$	$[-0.0019,+0.0059]$	$+0.0003$	$[-0.0021,+0.0026]$
E	$-0.0003$	$[-0.0009,+0.0003]$	$+0.0036$	$[+0.0022,+0.0050]$	$+0.0002$	$[-0.0004,+0.0009]$

Table 18: E10 paired AUC deltas (exposure RNN minus comparator) against the three constrained-LightGBM incumbents on the shared per-seed test splits, with two-level bootstrap

95\%

confidence intervals. The learned channel is ahead on regimes A, B, and C against all three comparators (intervals exclude zero and the per-seed delta is positive for every seed; the regime-B margins, the regime-C grid-GBM margins, and the regime-A margin against the raw-lags GBM are small enough to read as leans) and statistically tied on D and E against the grid GBMs (the regime-E raw-lags delta clears zero on its own but fails the joint-direction rule of Section 5.3 ); it is nowhere statistically behind.

Figure 6: E10 misspecification-cost curves: paired gap

\Delta

AUC (exposure RNN minus baseline, mean

\pm

std over five seeds) under the recalibrated decay sweep on regime A (top) and threshold sweep on regime D (bottom); left panels plot the gap against the swept generating parameter (dotted verticals mark the grids’ fixed values); right panels plot the steelman-grid consumers only—the models whose grid defines the distance—against the distance from the swept parameter to that grid; one legend serves both rows. The static monotone aggregate is omitted from the panels: its decay-sweep gap duplicates Figure 2 at an order of magnitude above the grid consumers’ typical gaps (peak

+0.044

) and would compress their curves, and on the threshold sweep it sits within the same

{\sim}\pm 0.002

null band as the grid consumers. Reading key: each grid pays where its own coverage runs out, and the threshold sweep is a null for the grid consumers (see text).

Figure 7: E10 sample-size sweep on regimes A and C (AUC, mean

\pm

std over five seeds; log-scale

n

), with all torch models trained to early-stopping convergence and validation/test splits fixed at full size across

n

. The learned channel is best at every

n

on regime A and at every

n

except

1000

on regime C; the raw-lags GBM, the only comparator whose input dimension is not compressed by feature engineering or learned aggregation, degrades fastest as

n

shrinks.

Figure 8: E10 stress extrapolation on regime C: AUC versus severity under the scale (left) and upper-tail amplification (right) ladders for clean-trained models (mean

\pm

std over five seeds). The grid GBMs collapse beyond the training support; the learned channel degrades gracefully; the fixed grids feeding neural monotone heads (monotone MLP or calibrated lattice), whose features are linear in the input scale, extrapolate best.

Saturation rate (fraction of test scores at the maximum)
Model	base	$\times$ 1.5	$\times$ 2.0	$\times$ 3.0	$\times$ 5.0	$\times$ 8.0	$\gamma$ 2	$\gamma$ 3	$\gamma$ 5	$\gamma$ 10
Mono. exp. RNN	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000
Mono. agg.	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000
Practice grid	0.000	0.001	0.001	0.003	0.014	0.049	0.001	0.003	0.012	0.067
Steelman grid	0.000	0.001	0.002	0.008	0.033	0.110	0.002	0.007	0.028	0.133
Practice lattice	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000

Table 19: E10 score-degeneracy diagnostics per model and severity (mean over five seeds;

\times k

denotes the scale multiplier and

\gamma k

the upper-tail amplification at coverage

q=0.5

): fraction of test scores saturated at the model’s maximum attained score, and tie rate (

1-

fraction of distinct scores). The grid GBMs saturate and tie increasingly with severity—piecewise-constant ensembles over few engineered features go exactly flat beyond the training support—while the calibrated lattices and the learned channel never measurably tie; the grid-MLP heads tie moderately at extreme severity (up to

0.245

\times 8

and

0.254

\gamma 10

, from tanh-head saturation) without the GBMs’ rank collapse; the raw-lags GBM avoids degeneracy because its

24

input columns keep leaf assignments distinct.

Figure 9: The subsumption boundary in one panel: held-out Spearman

\rho

of the learned e-channel regressed onto each of the eighteen target statistics (dots; whiskers are

\pm 1

std over five seeds), ordered from separable accumulations to negative controls. The separable family sits at rank equivalence; trailing windows and extremes carry a measurable approximation ceiling; the negative controls fail as designed. Open diamonds mark the explicit matching-basis channels—the

\max

channel for the HWM, the LWM channel for the floor, the early-window channel for the time-localization control, and the run-length channel for consecutivity—each of which restores its statistic exactly.

Separable accumulations
Target statistic	Channel	$R^{2}$	Spearman $\rho$
EWMA ( $\alpha{=}0.50$ )	e-channel	$0.992\pm 0.004$	$1.000\pm 0.000$
EWMA ( $\alpha{=}0.60$ , off-grid)	e-channel	$0.993\pm 0.003$	$1.000\pm 0.000$
EWMA ( $\alpha{=}0.70$ )	e-channel	$0.993\pm 0.004$	$1.000\pm 0.000$
EWMA ( $\alpha{=}0.80$ , off-grid)	e-channel	$0.995\pm 0.005$	$1.000\pm 0.000$
EWMA ( $\alpha{=}0.85$ )	e-channel	$0.995\pm 0.006$	$1.000\pm 0.000$

Table 20: E11 functional-emulation spanning table: held-out

R^{2}

and Spearman

\rho

(mean

\pm

std over five seeds) for single-basis learned channels trained by MSE regression to reproduce each target statistic. The separable family is reproduced to rank equivalence, on- and off-grid; windows and extremes are approximated with a measurable ceiling; the negative controls fail as designed, and the explicit run-length, early-window, and LWM channels restore all three exactly.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Parameter	Range	Parameter	Range
$p_{01}$ (normal $\to$ stress)	$U(0.05,0.15)$	$\sigma_{0}$ (normal log-std)	$U(0.15,0.35)$
$p_{10}$ (stress $\to$ normal)	$U(0.10,0.30)$	$\sigma_{1}-\sigma_{0}$ (std gap)	$U(0.15,0.40)$
$\Pr[z_{1}=1]$ (initial stress)	$U(0.10,0.30)$	$p_{\text{spike}}$ (per-step)	$U(0.01,0.05)$
$\rho_{\mathrm{AR}}$ (AR(1) persistence)	$U(0.50,0.90)$	$\mu_{u}$ (spike log-mean)	$U(1.0,2.0)$
$\mu_{0}$ (normal log-mean)	$U(0.20,0.80)$	$\sigma_{u}$ (spike log-std)	$U(0.30,0.70)$
$\mu_{1}-\mu_{0}$ (mean gap)	$U(0.80,1.60)$

Appendix

Table 21: Exposure-process constants: each parameter is drawn independently per sequence from the stated uniform range. The latent state

z_{t}\in\{0,1\}

(normal/stress) follows a Markov chain; log-exposure follows the state-dependent AR(1) of Section 4 ; spikes add

\exp(\mathcal{N}(\mu_{u},\sigma_{u}^{2}))

with per-step probability

p_{\text{spike}}

Regime	Logit $\eta$
A (recency)	$0.376\,\tilde{E}-4.82$
B (peak)	$0.108\,M-2.65$
C (mixed)	$0.266\,\tilde{E}+0.077\,M-4.84$
D (threshold stress)	$0.117\,(M-10.17)^{+}-1.75$
E (diminishing returns)	$1.438\log(1{+}E)+1.621\log(1{+}M)-10.08$
F (cumulative burden)	$0.164\sum_{t}\log(1{+}x_{t})-5.20$

Appendix

Table 22: Label-regime logits

\eta

with their exact constants (notation defined in the accompanying paragraph). Intercepts are fixed constants pre-calibrated to a class prevalence of approximately

25\%

T=24

; regime I draws a per-sequence mixture indicator

z\sim\mathrm{Bernoulli}(0.5)

. Regime P is the window-price diagnostic of E11, not a member of the fifteen-regime suite: its slope follows the standardized-signal convention anchored to regime A (the E2/E10 recalibration recipe), its window

k=6

matches a practice-grid window, and its Bayes ceiling is

0.747

on the shared test splits.

	Regime A			Regime C
Metric	MC drop.	Ensemble	$\Delta$ (ens $-$ MC)	MC drop.	Ensemble	$\Delta$ (ens $-$ MC)
AUC	$0.8396\pm 0.0043$	$0.8397\pm 0.0043$	$0.0001\pm 0.0002$	$0.8534\pm 0.0045$	$0.8535\pm 0.0045$	$0.0001\pm 0.0002$
Log loss (NLL)	$0.4057\pm 0.0058$	$0.4055\pm 0.0060$	$-0.0002\pm 0.0002$	$0.3845\pm 0.0042$	$0.3842\pm 0.0043$	$-0.0003\pm 0.0002$
Brier	$0.1280\pm 0.0019$	$0.1279\pm 0.0019$	$-0.0000\pm 0.0001$	$0.1199\pm 0.0012$	$0.1198\pm 0.0012$	$-0.0000\pm 0.0000$
ECE	$0.0106\pm 0.0011$	$0.0115\pm 0.0016$	$0.0009\pm 0.0013$	$0.0110\pm 0.0021$	$0.0114\pm 0.0018$	$0.0004\pm 0.0012$
Uncertainty–error Spearman	$0.201\pm 0.015$	$0.752\pm 0.103$	$0.551\pm 0.102$	$0.210\pm 0.015$	$0.766\pm 0.074$	$0.556\pm 0.075$

Appendix

Table 23: S1c: deep ensemble (

N=5

) versus MC dropout (

K=20

) for the monotone exposure RNN on regimes A and C (mean

\pm

std over five seeds;

\Delta

is the paired per-seed difference, ensemble minus MC dropout). Both arms are monotone per sample by construction; the comparison isolates uncertainty quality and its training cost.

Log loss
Regime	Mono. exp. RNN	Gen. mono. RNN	Mono. agg.	Lattice agg.	Unc. agg.	Unc. RNN	Unc. LSTM
A	$0.4055\pm 0.0060$	$0.4238\pm 0.0053$	$0.4445\pm 0.0067$	$0.4283\pm 0.0062$	$0.4264\pm 0.0067$	$0.4053\pm 0.0056$	$\mathbf{0.4048\pm 0.0059}$
B	$0.4605\pm 0.0066$	$0.4901\pm 0.0061$	$\mathbf{0.4603\pm 0.0068}$	$0.4605\pm 0.0065$	$0.4605\pm 0.0067$	$0.4647\pm 0.0064$	$0.4625\pm 0.0069$
C	$\mathbf{0.3843\pm 0.0043}$	$0.3971\pm 0.0030$	$0.3981\pm 0.0039$	$0.3944\pm 0.0038$	$0.3942\pm 0.0046$	$0.3871\pm 0.0036$	$0.3850\pm 0.0047$
D	$0.4743\pm 0.0080$	$0.4968\pm 0.0073$	$\mathbf{0.4733\pm 0.0083}$	$0.4738\pm 0.0083$	$\mathbf{0.4733\pm 0.0082}$	$0.4776\pm 0.0086$	$0.4748\pm 0.0079$
E	$0.3647\pm 0.0053$	$0.3926\pm 0.0054$	$0.3797\pm 0.0071$	$0.3706\pm 0.0060$	$0.3733\pm 0.0065$	$0.3661\pm 0.0053$	$\mathbf{0.3643\pm 0.0054}$

Appendix

Table 24: E3 log loss, Brier score, and ECE (mean

\pm

std over five seeds) for the core model set and the calibrated-lattice comparator on regimes A–E; best per regime and metric in bold (lower is better; ties at the printed precision bolded jointly). Column abbreviations as in Table 6 .

	E9 capacity sweep (per tier and pooled)					E5 grid
Model	tiny	small	medium	large	total	total
Unconstrained RNN	324	926	1,713	4,172	7,135	2,373
Unconstrained LSTM	1,352	1,168	1,188	794	4,502	2,425
Unconstrained aggregate	1,958	1,958	1,958	1,958	7,832	–

Appendix

Table 25: Pooled monotonicity-violation totals for the unconstrained baselines quoted in the E5 and E9 prose: per capacity tier and pooled over the E9 sweep (both regimes, five seeds), and pooled over the E5 missingness grid (regime C). The static unconstrained aggregate has no capacity knob, so its identical per-tier counts restate one fitted model evaluated once per tier; it is not part of the E5 model set. Every monotone-by-construction model records zero violations in every corresponding cell (Table 15 ; Section 6 ).

Explore similar work

Jun 26, 2026cs.LG

Dangerous Liaisons of Convex Learning and Non-Affine Aggregation

Last-iterate convergence and generalization guarantees in first-order convex learning hinge on the monotonicity of the update operator. While linear averaging preserves the monotonicity of gradient updates, this property is often violated when gradients are aggregated non-affinely, as in modern pipelines enforcing constraints like adaptivity, privacy, robustness or fairness. Whether it is possible to design non-affine aggregation rules that maintain monotonicity has remained an open question. We answer this question negatively: we prove that the monotonicity of aggregated gradients is preserved if and only if the aggregation rule is positively affine. Consequently, non-affine aggregation prevents steady convergence and substantially degrade algorithmic stability. We quantify these drawbacks and propose a path forward by identifying sufficient conditions under which monotonicity can be restored. Our results provide a unified theoretical framework explaining the disparate failure modes observed in modern learning systems.

Thomas Boudou, Batiste Le Bars, Nirupam Gupta +1

PreMeDICaL team, Inria, Idesp, Inserm, Universit´e de Montpellier, Montpellier, France · Univ. Lille, Inria, CNRS, Centrale Lille, UMR 9189, CRIStAL, F-59000 Lille, France · Department of Computer Science, University of Copenhagen, Copenhagen, Danemark

Convergence Monotonicity

Ask PDF

Jun 16, 2026cs.LG

Monotonic Kolmogorov-Arnold Networks: A Theoretical and Empirical Study of Monotonicity as an Inductive Bias

Monotonicity has been a long-running architectural inductive bias for neural networks, motivated by tabular, scientific, and economic settings where outputs are known to respond monotonically to certain inputs. Existing approaches are MLP- or flow-based and lack per-edge functional transparency; the only Kolmogorov--Arnold Network (KAN) variant with monotonicity, MonoKAN, enforces the constraint only on a restricted parameter subset and requires a projection-style training procedure. We close this gap with \textbf{MKAN}, a KAN with hard monotonicity guaranteed for \emph{all} parameter values via exponential reparameterization of B-spline coefficients, positive edge weights, and a monotone base activation. Training reduces to standard unconstrained gradient descent. Our headline theoretical contribution is a \emph{representation-cost} theorem: any

C^K, K >0

feature extractor inducing a ball-shaped semantic-neighborhood partition admits a monotone realization of the equivalent neighborhood structure at

N' = N^* + k \le 2N^*

, where

k

is the number of non-monotone coordinates of the original. The bound is architecture-agnostic and gives a principled sizing rule for monotone encoders. Empirically, MKAN is competitive with state-of-the-art monotone NNs on the SMM/ICML-2024 benchmark while being the only method that combines hard unconstrained monotonicity with KAN's per-edge functional transparency; the

2N^*

prediction is validated in a self-supervised feature-size sweep on four real datasets, and on a controlled monotone-generative dataset MKAN recovers ground-truth factors with substantially higher Spearman alignment than KAN, MLP, and linear baselines.

Mikhail Krasnov, Blaž Bertalanič, Carolina Fortuna

Jozef Stefan Institute

Kolmogorov-Arnold Networks Monotonicity Empirical Study Transparency

Ask PDF

May 29, 2026stat.ML

Hedging on the Frontier: Learning New Tasks with Few Samples

When a learner faces a new task with few samples, it must leverage any available side information. In practice, this often comes in the form of model evaluations on related tasks in public benchmarks. A key question then is how to model task relatedness such that it is both realistic and the benchmark evaluations lead to provable gains. Empirically, we observe that weak monotonicity is often approximately satisfied: if a model dominates another on many benchmarks, it also tends to outperform on the new task. We explore the statistical complexity of learning under (approximate) weak monotonicity, leveraging it within two learning paradigms: transfer learning and model selection aggregation. We show that not only can we prune the model class based on monotonicity, but we can also further adapt to the geometry of the available trade-offs by hedging on the frontier.

Tobias Wegel, Federico Di Gennaro, Geelon So +1

Department of Computer Science, ETH Zurich · Department of Computer Science and Engineering, UC San Diego

Learnability Monotonicity Transfer Learning Frontiers Model Selection New Tasks+1

Ask PDF