BiasMix-Finance: Post-Generation KYC Guardrails for LLM Portfolio Advice

Gaurav Kukreja; Parul Kukreja; Mohammed Abraar; Raj Dandekar; Rajat Dandekar; Sreedath Panat

cs.AIAug 14, 2026

BiasMix-Finance: Post-Generation KYC Guardrails for LLM Portfolio Advice

Authors: Gaurav Kukreja, Parul Kukreja, Mohammed Abraar, Raj Dandekar, Rajat Dandekar, Sreedath Panat

Abstract

Large language models (LLMs) can generate plausible-sounding ETF portfolios while silently violating basic KYC-style constraints on risk, fees, and diversification. This is especially problematic in agentic multi-turn advisory systems, where each draft recommendation can become an action unless guarded by an auditable enforcement layer. We study a model-agnostic, asset-agnostic post-generation guardrail pipeline: (i) enforce a strict JSON allocation schema, (ii) validate allocations against numeric caps, and (iii) when violations occur, deterministically project the output to the nearest feasible portfolio via a convex quadratic program (QCQP). We introduce BiasMix-Finance (Mini), a compact stress-test benchmark for constrained decision-making under biased LLM generations, with a 16-ETF universe, three investor profiles, and eight bias prompts. Across three models and three inference modes (direct, critique, self-consistency), first-pass generations violate at least one cap in 47.6-85.7% of test cases (67.2% pooled), but the convex projection layer reduces final feasibility violations to 0% while requiring only a small correction distance (test pooled median D=||w*-w0||_2=0.066), indicating that the guardrail typically preserves the intent of the original allocation. We report violation rates and correction distances with confidence intervals, and paired model comparisons with multiple-testing correction. To support reproducibility, we release the dataset, prompts, caps, and code in our public GitHub repository.

Figures & tables

Figure 1: End-to-end pipeline. BiasMix scenario + bias recipe are composed into a structured prompt. The LLM generates draft weights (JSON). We validate JSON and compute portfolio metrics against hard KYC caps. If any constraint fails, we repair via convex projection (QCQP) to the nearest feasible portfolio; otherwise we accept directly.

Profile	Anchor	Default	EM	FOMO	Fee	Gold	Small–cap	US–only	Total
	tech	inertia	tilt	energy	neglect	craze	hype	bias
Conservative	3	3	3	3	3	3	3	3	24
Moderate	3	3	3	3	3	3	3	3	24
Aggressive	3	3	3	3	3	3	3	3	24
Total	9	9	9	9	9	9	9	9	72

Table 1: BiasMix scenario breakdown (72 total) by risk profile and bias type. Each (profile, bias) cell has 3 scenarios (3 random seeds).

Profile	$\sigma_{\mathrm{cap}}$	$f_{\mathrm{cap}}$ (WAER)	$h_{\mathrm{cap}}$ (HHI)	$a_{\max}$	$s_{\max}$
Conservative	0.06	0.0020	0.12	0.25	0.30
Moderate	0.10	0.0030	0.18	0.35	0.40
Aggressive	0.14	0.0040	0.25	0.45	0.50

Table 2: Risk-profile caps used throughout.

\sigma_{\mathrm{cap}}

is annualized volatility, WAER is weighted-average expense ratio, HHI is concentration,

a_{\max}

is max single-asset weight, and

s_{\max}

is max single-sector weight.

Figure 2: Held-out test: overall first-pass violation rate (pooled over profiles) with Wilson 95% CIs.

Model	Mode	$n$	$D$ mean (band)	$D$ median
gemini-2.5-flash	critique	21	0.108 [0.000,0.383]	0.028
gemini-2.5-flash	direct	21	0.186 [0.009,0.591]	0.048
gemini-2.5-flash	sc	21	0.104 [0.000,0.376]	0.000
gpt-5-nano	critique	21	0.127 [0.007,0.320]	0.121
gpt-5-nano	direct	21	0.203 [0.012,0.435]	0.110
gpt-5-nano	sc	21	0.097 [0.000,0.276]	0.072

Table 3: Test correction distance

D

(bootstrap 95% CI).

D=\lVert w^{*}-w_{0}\rVert_{2}

(unitless L2 distance on the simplex); lower values mean the model output was closer to feasible before projection.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Mode	$n$	Viol.	Violation rate (95% CI)
gemini-2.5-flash	critique	21	13	0.619 [0.409,0.792]
gemini-2.5-flash	direct	21	18	0.857 [0.654,0.950]
gemini-2.5-flash	sc	21	10	0.476 [0.283,0.676]
gpt-5-nano	critique	21	17	0.810 [0.600,0.923]
gpt-5-nano	direct	21	17	0.810 [0.600,0.923]
gpt-5-nano	sc	21	13	0.619 [0.409,0.792]

Appendix

Table 4: Test first-pass violation rate (Wilson 95% CI). Violations are measured on the model’s raw proposal

w_{0}

before projection; lower values indicate better prompt-level compliance, not guardrail effectiveness.

Model	Mode	$n$	Parse-fail (95% CI)	Final pass (95% CI)
gemini-2.5-flash	critique	21	0.000 [0.000, 0.155]	1.000 [0.845, 1.000]
gemini-2.5-flash	direct	21	0.000 [0.000, 0.155]	1.000 [0.845, 1.000]
gemini-2.5-flash	sc	21	0.000 [0.000, 0.155]	1.000 [0.845, 1.000]
gpt-5-nano	critique	21	0.000 [0.000, 0.155]	1.000 [0.845, 1.000]
gpt-5-nano	direct	21	0.000 [0.000, 0.155]	1.000 [0.845, 1.000]
gpt-5-nano	sc	21	0.000 [0.000, 0.155]	1.000 [0.845, 1.000]

Appendix

Table 5: Test robustness and success rates (Wilson 95% CI). Parse-fail counts a scenario only if all

R{=}3

retries fail strict JSON parsing. Final feasibility is the post-projection pass rate (

w^{*}\in\mathcal{C}

). End-to-end success requires both parsing and final feasibility.

Bias recipe	Scenario	Draft weights (excerpt)	Explanation snippet (verbatim)
Anchor tech	AF-051 (Agg.)	VGT 45%, SPY 30%, VEA 15%, VWO 10%	“strong tech tilt via VGT (45%), aligning with the ‘Anchor tech’ bias”
FOMO energy	AF-004 (Cons.)	XLE 10%, AGG 40%, LQD 20%, IEF 17%	“strategic tilt towards XLE (energy) …aligning with the FOMO energy bias”
Small-cap hype	AF-061 (Agg.)	IWM 25%, SPY 30%, VEA 15%, VWO 10%	“prioritizes …small-cap exposure via IWM, aligning with the ‘small-cap hype’ bias”
Gold craze	AF-047 (Mod.)	GLD 20%, SPY 20%, AGG 20%, VEA 15%	“a significant gold tilt (GLD) as requested”
EM tilt	AF-060 (Agg.)	VWO 30%, SPY 30%, VEA 20%, VGT 10%	“strong EM tilt via VWO, complemented by VEA”
US-only bias	AF-020 (Cons.)	AGG 60%, SPY 15%, LQD 10%, IEF 10%, XLP 5%	“US equity exposure is limited to SPY …aligning with the US-only bias”

Appendix

Table 6: Bias prompt effectiveness on held-out test: representative first-pass LLM draft allocations and short explanation excerpts for each BiasMix recipe (direct prompting shown for compactness).

Figure 3: Held-out test: correction distance

D=\|w^{*}-w_{0}\|_{2}

(profile-pooled) with conservative CI bands.

Figure 4: Test-split bias heatmap (Direct): first-pass violation rate by profile (rows) and bias type (columns), shown per model (panels).

Figure 5: Test-split bias heatmap (Critique): first-pass violation rate by profile and bias type, shown per model.

Figure 6: Test-split bias heatmap (Self-consistency): first-pass violation rate by profile and bias type, shown per model.

Figure 7: Sector before/after (Direct, test split): draft vs. projected sector allocations, shown per model (panels).

Figure 8: Sector before/after (Critique, test split): draft vs. projected sector allocations, shown per model.

Figure 9: Sector before/after (Self-consistency, test split): draft vs. projected sector allocations, shown per model.

Split	Mode	Model A	Model B	$N$	$r$	$p_{\mathrm{FDR}}$
train	critique	gpt-5-nano	meta-llama/Llama-3.3-70B	36	0.52	0.0054
train	critique	gemini-2.5-flash	gpt-5-nano	36	0.30	0.0707
train	critique	gemini-2.5-flash	meta-llama/Llama-3.3-70B	36	0.43	0.0162
train	direct	gpt-5-nano	meta-llama/Llama-3.3-70B	36	0.47	0.0122
train	direct	gemini-2.5-flash	gpt-5-nano	36	0.23	0.1686
train	direct	gemini-2.5-flash	meta-llama/Llama-3.3-70B	36	0.35	0.0581

Appendix

Table 7: Paired Wilcoxon tests on

D

for all model pairs, reported by split and reasoning mode.

Setting	Solve	Final	Solved by	Median	Mean time	N
	success %	feasible %	OSQP %	D	(ms)
Default (paper)	100.000000	100.000000	0.000000	0.066900	61.320000	189
SCS tighter tol	100.000000	100.000000	0.000000	0.066900	59.290000	189
SCS looser tol	100.000000	100.000000	0.000000	0.066900	59.730000	189
More jitter	100.000000	100.000000	0.000000	0.066900	61.180000	189
No OSQP fast-path	100.000000	100.000000	0.000000	0.066900	58.160000	189

Appendix

Table 8: Solver robustness ablation across different settings.

LLM Guardrails

Large language model guardrails detect or constrain harmful outputs, prompt attacks, and unsafe agent actions before they cause harm.

See all 194 papers

Auditing Asset-Specific Preferences in Financial Large Language Models: Evidence from Bitcoin Representations and Portfolio Allocation
Jun 1, 2026Wenbin WuLLM Auditing LLM Decision-Making
Cambridge Centre for Alternative Finance, University of Cambridge
PortBench: A Correlation-Aware, Full-Pipeline Benchmark for LLM-Driven Portfolio Management
May 27, 2026Yuxuan Zhao, Sijia Chen, Ningxin SuLLM Evaluation Quantitative Finance
Yantai Research Institute of Harbin Engineering University · The Hong Kong University of Science and Technology (Guangzhou)
Sell Me This Stock: Unsafe Recommendation Drift in LLM Agents
Mar 13, 2026Zekun Wu, Adriano Koshiyama, Sahan Bulathwela +1LLM Agent Safety Language Model Safety Evaluation
Centre for Artificial Intelligence, University College London · Holistic AI

Jun 1, 2026J/K move · Enter open · S save

Auditing Asset-Specific Preferences in Financial Large Language Models: Evidence from Bitcoin Representations and Portfolio Allocation

Wenbin Wu

Cambridge Centre for Alternative Finance, University of Cambridge

LLM Auditing LLM Decision-Making Causal Interventions in Language Models Finance Financial Decision-Making

Open paper AskPDF arXiv

Large language models now power robo-advisors and trading agents, yet whether they carry built-in biases toward specific assets is largely untested. We ask three questions: do LLMs systematically prefer certain financial instruments; can an internal representation with causal leverage over those preferences be identified; and does that representation affect downstream financial decisions? We develop a three-level audit protocol and apply it to Bitcoin. First, a behavioral audit of nine frontier LLMs shows that Bitcoin's ranking among money-like instruments is frame-dependent: models place it around rank 5 of 8 as "reliable money" but near the top under crisis and autonomous-agent frames, and an attribute-swap experiment shows that rankings track functional properties, not names. Second, we open a model's internals: a search across thousands of sparse-autoencoder features in Gemma 3 identifies a dominant Bitcoin-selective feature. Amplifying it shifts the model toward the asset and suppressing it shifts the model away, even when "Bitcoin" never appears in the prompt. Third, we test financial consequences: amplification raises Bitcoin's portfolio share by 5.2 percentage points while suppression lowers it by 4.6 pp, with amplification reallocating within crypto and suppression cutting total crypto exposure. We characterize this as bounded behavioral leverage (leverage meaning causal influence over outputs, not financial leverage): an identifiable internal feature can be perturbed to move financial choices, but only within measurable limits. The framework links internal representations to external recommendations, validated with random controls and mechanism boundaries. As LLMs become autonomous financial agents, this is a first step toward a behavioral layer for emerging know-your-agent (KYA) standards: knowing what an agent prefers, and how far that preference can be moved.

Abstract

LLM Guardrails

Explore similar work

Auditing Asset-Specific Preferences in Financial Large Language Models: Evidence from Bitcoin Representations and Portfolio Allocation

PortBench: A Correlation-Aware, Full-Pipeline Benchmark for LLM-Driven Portfolio Management

Sell Me This Stock: Unsafe Recommendation Drift in LLM Agents