When Alpha Breaks: Two-Level Uncertainty for Safe Deployment of Cross-Sectional Stock Rankers

Ursina Sanderink

cs.AIFeb 24, 2026

When Alpha Breaks: Two-Level Uncertainty for Safe Deployment of Cross-Sectional Stock Rankers

Authors: Ursina Sanderink

Abstract

Cross-sectional ranking models are often deployed as if point predictions were sufficient: the model outputs scores and the portfolio follows the induced ordering. Under non-stationarity, rankers can fail during regime shifts. In the AI Stock Forecaster, a LightGBM ranker performs well overall at a 20-day horizon, yet the 2024 holdout coincides with an AI thematic rally and sector rotation that breaks the signal at longer horizons and weakens 20d. This motivates treating deployment as two decisions: (i) whether the strategy should trade at all, and (ii) how to control risk within active trades. We adapt Direct Epistemic Uncertainty Prediction (DEUP) to ranking by predicting rank displacement and defining an epistemic uncertainty signal ehat relative to a point-in-time (PIT-safe) baseline. Empirically, ehat is structurally coupled with signal strength (median correlation between ehat and absolute score is about 0.6 across 1,865 dates), so inverse-uncertainty sizing de-levers the strongest signals and degrades performance. To address this, we propose a two-level deployment policy: a strategy-level regime-trust gate G(t) that decides whether to trade (AUROC around 0.72 overall and 0.75 in FINAL) and a position-level epistemic tail-risk cap that reduces exposure only for the most uncertain predictions. The operational policy, trade only when G(t) is at least 0.2, apply volatility sizing on active dates, and cap the top epistemic tail, improves risk-adjusted performance in the 20d policy comparison and indicates DEUP adds value mainly as a tail-risk guard rather than a continuous sizing denominator.

Figures & tables

Panel A: Shadow portfolio (20d, monthly non-overlapping L/S, vol-sized LightGBM; cost model as in text; no UQ)
	ALL	DEV (2016–2023)	FINAL (2024+)
Sharpe (ann.)	2.734	3.121	2.337
Sortino (ann.)	6.058	5.410	9.687
Max drawdown	$-18.1$ %	$-18.1$ %	$-8.7$ %
Calmar (CAGR / $\|\mathrm{MaxDD}\|$ )	6.76	6.07	26.2
Ann. return (arithmetic)	87.0%	79.6%	137.3%

Table 1: Baseline performance without uncertainty controls. Panel A reports the project’s shadow portfolio at the 20-day horizon (top/bottom-10, monthly, non-overlapping) with volatility sizing and the project’s turnover-based cost model. Panel B reports RankIC signal quality of raw LightGBM scores. The base strategy remains strong at 20d in FINAL, while longer horizons (60d/90d) invert during the 2024 regime shift.

Work	Task	UQ method	Action rule	Regime control
Spears et al. (2021)	Futures return	Bayesian NN posterior	Multiplicative sizing	Variance threshold
Liu et al. (2026)	Return prediction	Rolling residual quantiles	Uncertainty-adj. sorting	None
Garlappi et al. (2007)	Portfolio opt.	Ambiguity / model uncertainty	Shrinkage toward safety	None
This paper	Cross-sec. ranking	DEUP on rank displacement	Tail-risk cap ( $\hat{e}$ -Cap)	Binary gate $G(t)$

Table 2: How this work relates to prior uncertainty-aware portfolio methods. In ranking, uncertainty can be structurally coupled to signal strength, motivating discrete tail-risk control plus a model-specific regime gate.

Category	Feature	Purpose
Per-prediction	score , abs_score	Prediction magnitude/strength
	cross_sectional_rank	Position in today’s ranking
Stock-level	vol_20d , vol_60d	Realized volatility
	mom_1m	Recent momentum
	adv_20d	Liquidity proxy
Market regime	vix_percentile_252d	Rolling VIX percentile

Table 3: Features used in the error predictor

g(x)

Signal	20d ALL	20d FINAL	90d ALL	90d FINAL
$\hat{e}(x)$ (DEUP)	0.144	0.192	0.146	0.248
$g(x)$ (raw)	0.192	0.218	0.161	0.262
vol_20d	0.047	0.010	0.035	0.009
VIX percentile	0.018	$-$ 0.022	0.053	$-$ 0.020
$\|\text{score}\|$	0.096	0.065	0.018	0.013

Table 4: Per-stock uncertainty signal comparison: Spearman

\rho

with realized rank loss.

\hat{e}

and

g(x)

dominate heuristic baselines by

3

–

10\times

and maintain predictive power in the FINAL holdout where volatility-based signals collapse.

Variant	ALL	DEV	FINAL	Crisis MaxDD
Ungated Raw (LGB)	2.730	3.107	1.650	-7.4%
1. Gate + Raw	1.928	2.039	-0.322	-7.4%
2. Gate + Vol	1.886	1.971	0.375	-8.4%
4. Gate + Resid- $\hat{e}$	1.987	2.041	0.953	-6.0%
6. Gate + Vol + $\hat{e}$ -Cap	1.877	1.928	0.925	-6.7%

Table 5: Deployment-policy ablation (20d). Sharpe

\times\sqrt{12}

from monthly non-overlapping returns. FINAL

N=14

months.

Predictor	AUROC (ALL)	AUROC (FINAL)
$H(t)$ (combined)	0.721	0.750
$G(t)$ (gate)	0.710	0.743
$H_{\text{real}}$ -only	0.715	—
VIX percentile	0.449	0.504
Market volatility	0.596	0.569
Mean stock volatility	0.590	0.460

Table 6: Regime-trust classification at 20d.

H(t)

and

G(t)

substantially outperform market-regime proxies. In the FINAL holdout, VIX percentile is effectively uninformative (AUROC

\approx 0.5

) and mean stock volatility becomes negatively informative (AUROC

<0.5

Level	Question	Signal	Mechanism
Strategy	Should we trade today?	$G(t)$ (trailing RankIC)	Binary abstention
Position	Which stocks need caution?	$\hat{e}_{\mathrm{PIT}}(x)$ (DEUP)	Percentile cap

Table 7: The two-level deployment architecture addresses two distinct failure modes.

	20d		90d
Quintile	DEV	FINAL	DEV	FINAL
Q1 (low $\hat{e}$ )	0.265	0.253	0.242	0.233
Q2	0.267	0.275	0.258	0.305
Q3	0.297	0.308	0.285	0.315
Q4	0.354	0.366	0.321	0.386
Q5 (high $\hat{e}$ )	0.400	0.427	0.371	0.437

Table 8: Mean realized rank loss by

\hat{e}(x)

quintile. All four DEV/FINAL

\times

horizon conditions exhibit perfect monotonicity (Spearman

\rho=1.0

). FINAL separation is stronger than DEV at both horizons—the opposite of overfitting.

Horizon	ALL	DEV	FINAL
20d	0.144	0.142	0.192
60d	0.106	0.103	0.140
90d	0.146	0.138	0.248

Table 9:

\rho(\hat{e},\ell)

: Spearman correlation of

\hat{e}(x)

with realized rank loss. At every horizon, FINAL exceeds DEV.

Variant	Sharpe (ALL)	Sharpe (DEV)	Sharpe (FINAL)	Crisis MaxDD
Gate + Vol + $\hat{e}$ -Cap, $\hat{e}^{\text{oracle}}$	1.864	1.915	0.906	$-6.7\%$
Gate + Vol + $\hat{e}$ -Cap, $\hat{e}^{\text{PIT}}$ ( $W=60$ )	1.864	1.915	0.906	$-6.7\%$

Table 10: Deployability ablation for the cap policy (20d horizon). Gate + Vol +

\hat{e}

-Cap uses

\hat{e}

only through a cross-sectional percentile threshold (P85). Using a hindsight

\hat{e}^{\text{oracle}}

versus PIT-safe

\hat{e}^{\text{PIT}}

(rolling matured-loss

P_{10}

W=60

) yields identical performance, confirming the policy is deployment-ready.

Predictor	AUROC (ALL)	AUROC (FINAL)
$H(t)$ (combined)	0.721	0.750
$G(t)$ (gate)	0.710	0.743
$H_{\text{real}}$ -only	0.715	—
Market volatility (21d)	0.596	0.569
Mean stock volatility	0.590	0.460
VIX percentile	0.449	0.504

Table 11: Regime-trust classification at 20d (extended).

H(t)

and

G(t)

substantially outperform all market-regime proxies. The advantage widens in FINAL: the gate’s margin over the best baseline grows from 10 pp in DEV to 17 pp in FINAL.

Bucket	Mean $G$	Mean RankIC	% Bad days
0 (lowest)	0.006	$-$ 0.011	51.2%
1	0.236	$+$ 0.065	34.3%
2	0.573	$+$ 0.114	22.0%
3 (highest)	0.939	$+$ 0.153	11.5%

Table 12:

G(t)

bucket analysis: higher gate values correspond to strictly better model outcomes.

Period	Mean $G$	% Abstain	Mean IC	Mean VIX%	$G(t)$	VIX
Crisis episodes
COVID recovery 2020	0.375	47%	$+$ 0.062	52%	✓	✓
Meme mania 2021	0.210	73%	$-$ 0.040	96%	$\times$	✓
Inflation shock 2022	0.077	86%	$-$ 0.024	72%	✓	✓
Late hiking 2023 H2	0.381	40%	$+$ 0.034	94%	✓	$\times$
AI rotation 2024	0.123	76%	$-$ 0.013	95%	✓	✓

Table 13: Multi-crisis diagnostic for

G(t)

vs. VIX gate (20d horizon).

G(t)

achieves 7/8 correct verdicts; VIX achieves 5/8. The VIX gate’s three false alarms (2019, 2023 H1, 2023 H2) occur during the model’s strongest periods (IC

>0.10

), where

G(t)

correctly stays active.

Normalizer	Low- $\hat{e}$	Mid- $\hat{e}$	High- $\hat{e}$	Spread	Mean width
Raw (none)	0.982	0.938	0.781	0.201	0.674
DEUP / $\hat{e}^{\text{oracle}}$	0.895	0.904	0.898	0.009	0.647
DEUP / $\hat{e}^{\text{PIT}}$ ( $W=60$ )	0.921	0.897	0.879	0.042	0.640
DEUP / $\hat{e}^{\text{PIT}}$ ( $W=252$ )	0.924	0.897	0.878	0.046	0.640

Table 14: Conditional coverage by

\hat{e}

tercile (20d, 90% nominal). Raw conformal over-covers low-

\hat{e}

stocks and under-covers high-

\hat{e}

stocks (20.08 pp spread). DEUP-normalized conformal substantially reduces the disparity. Under oracle

\hat{e}^{\text{oracle}}

, the spread is 0.94 pp (a

21.4\times

improvement over raw). Under deployable PIT-safe

\hat{e}^{\text{PIT}}

the improvement persists, with spreads of 4.20–4.61 pp and slightly narrower mean widths than the oracle variant.

Quantitative Finance

Quantitative finance uses mathematical models, statistics, and machine learning to value assets, measure risk, and guide trading and portfolio decisions.

See all 191 papers

May 13, 2026

Algometrics: Forecasting Under Algorithmic Feedback

In algorithmic markets, predictive models become part of the data-generating process they aim to forecast. Once their outputs are converted into trades, allocations, execution schedules, or risk controls, they change the future data on which they are evaluated. I introduce algometrics, a framework for time series whose evolution depends on the predictive algorithms forecasting them. The framework distinguishes historical risk, measured under passive forecasting, from deployment risk, measured when forecasts drive actions. I prove three results. First, deployment risk is not identifiable from passive historical data alone: even in a one-step linear feedback model, infinitely many algorithm-mediated environments induce the same historical law while implying different deployment risks for the same forecaster. Second, historical model rankings can invert under crowding, so a predictor with lower passive error can have higher deployment error once similar algorithms are adopted. Third, randomized or instrumented actions identify short-horizon linear feedback, and I derive a finite-sample bound for deployment-risk estimation. These results suggest that time-series benchmarks in algorithmic markets should report feedback sensitivity alongside predictive accuracy.

Marc Schmitt

University of Oxford

Time Series Forecasting Performative Prediction Algorithmic Trading

Ask PDF

Oct 19, 2026

Regime-Aware Interpretable Ranking for Sector Rotation

Sector rotation is a common decision in benchmark-relative portfolio management, where portfolio managers rank sectors for overweight and underweight positions. We formulate this task as ranking sector actions by expected future risk-adjusted active performance, measured with forward sector-benchmark Sharpe ratios. Such rankings should preserve temporal information, express model outputs through economically familiar signal characteristics, account for changes in signal relevance across macroeconomic environments, and capture dependencies among sectors. We propose RegimeRank, an interpretable sector-ranking model that represents fundamental and technical signals through expert-defined temporal characteristics, conditions them on learned macro factors via cross-attention, and models cross-sector dependencies through self-attention. Expected-gradient attributions identify the contributions of individual signal-transformation pairs, while macroeconomic and cross-sector attention maps provide diagnostics of the learned conditioning and relational structure. We evaluate RegimeRank on weekly S&P 500 data from 2002 to 2024 under combinatorial purged cross-validation, where it outperforms representative ranking and machine-learning baselines across multiple ranking metrics. Ablation studies support the contributions of latent macro states, regime conditioning, and cross-sector modelling, while attribution and attention analyses illustrate the temporal characteristics, macro states, and sector relationships underlying its rankings.

Kelvin J. L. Koa, Ziling Liu, Xinyang Li +7

Asian Institute of Digital Finance, National University of Singapore Singapore · Manulife Toronto, Ontario, Canada

Finance Learning to Rank Financial Forecasting

Ask PDF

Jul 16, 2026

Evaluating Epistemic Uncertainty: Beyond OOD Detection and Active Learning

Current evaluation of epistemic uncertainty relies on tasks such as out-ofdistribution detection and active learning. However, the Bayes-optimal decision strategies for these tasks do not coincide with the scores commonly used to quantify epistemic uncertainty. Building on the epistemic reject-option framework, we evaluate epistemic uncertainty using its ability to identify regret, the reducible error. Formulating selective prediction as a constrained optimization over coverage, expected risk, and regret, we prove the optimal selector is a thresholded convex combination of the ground-truth aleatoric and epistemic uncertainties. This theoretical unification exposes a weakness in recent uncertainty disentanglement literature: we demonstrate that standard correlation metrics between learned components do not necessarily predict their actual operational utility. We instead propose to evaluate the achievable risk, regret, coverage surface of the decomposition as a diagnostic for joint disentanglement and utility. Benchmarking standard methods on datasets with dense human annotations reveals that decision-theoretic rankings can disagree substantially with proxy-task rankings, including pairwise rank inversions between methods that are top-ranked on one criterion and bottom-ranked on other.

Jakub Paplhám, Willem Waegeman, Eyke Hüllermeier +1

Czech Technical University in Prague · Ghent University · LMU Munich, MCML, DFKI

Selective Prediction Decision-Making under Uncertainty Epistemic Uncertainty

Ask PDF

Abstract

Quantitative Finance

Explore similar work

Algometrics: Forecasting Under Algorithmic Feedback

Regime-Aware Interpretable Ranking for Sector Rotation

Evaluating Epistemic Uncertainty: Beyond OOD Detection and Active Learning