Preconditioned Test-Time Adaptation for Out-of-Distribution Debiasing in Narrative Generation

Hanwen Shen; Ting Ying; Jiajie Lu; Shanshan Wang

cs.CLMar 14, 2026

Preconditioned Test-Time Adaptation for Out-of-Distribution Debiasing in Narrative Generation

Authors: Hanwen Shen, Ting Ying, Jiajie Lu, Shanshan Wang

Organizations: Laboratory for Artificial Intelligence in Mathematics Education, Stevens Institute of Technology · Independent Researcher · NLP2CT Lab, Department of Computer and Information Science, University of Macau

Abstract

Although debiased large language models (LLMs) excel at handling known or low-bias prompts, they often fail on unfamiliar and high-bias prompts. We demonstrate via out-of-distribution (OOD) detection that these high-bias prompts cause a distribution shift, degrading static model performance. To enable real-time correction, we propose CAP-TTA, a test-time adaptation framework. CAP-TTA triggers context-aware LoRA updates only when a bias-risk score exceeds a set threshold. By utilizing an offline precomputed diagonal preconditioner, it ensures fast and stable optimization. Across multiple benchmarks and human evaluations, CAP-TTA effectively reduces toxicity/bias score with significantly lower latency than standard optimization methods (e.g., AdamW or SGD). Furthermore, it prevents catastrophic forgetting, and substantially improves narrative fluency over state-of-the-art baselines without compromising debiasing performance.

Figures & tables

Figure 1: Static generation vs. prior test-time adaptation (TTA) vs. CAP-TTA. Static generation uses frozen parameters. Prior TTA performs online updates during generation, which can be costly and unstable. CAP-TTA decouples adaptation into an offline precomputed preconditioner $P_{0}$ and an online bias-triggered, lightweight preconditioned update (optionally routed to a safe corpus with 4 types) when the trigger score exceeds $\epsilon$ .

Model	Method Type	Safety (Mean Bias $\downarrow$ )		Efficiency
Model	Method Type	ID (Safe)	OOD (Toxic)	Speed (tok/s)	Dynamic?
Base Models
Qwen3-4B	Base Pretrained	0.289	0.452	19.4	No
DeepSeek-R1-8B	Base Pretrained	0.395	0.454	26.0	No
Static Alignment / Debiasing Baselines
DeepSeek-R1-8B-Debiased	Offline Detox	0.389	0.471	21.9	No

Table 1: Quantitative comparison on ID (safe)/OOD (toxic) prompts. We report BB Bias score on the ID/OOD set. Speed is decoding throughput(tok/s). Dynamic? indicates whether the method performs Chain-of-Thought.

Detector	AUROC [95% CI]	AUPR
kNN (k=10)	99.22% [98.88, 99.51]	99.62%
Mahalanobis	98.81% [98.39, 99.17]	99.46%
LLR	70.74% [68.91, 72.60]	86.67%

Table 2: OOD detection results comparing RealToxicityPrompts (RTP) against WritingPrompts using Qwen3-4B. Using an AUROC-based convention (AUROC

>

95% indicating Far-OOD) ( Sun et al., 2022 ) , kNN and Mahalanobis indicate RTP is Far-OOD, while LLR suggests Near-OOD.

System	PPL $\downarrow$	Fluency $\uparrow$	StereoSet $\downarrow$	StereoDet $\downarrow$	Delicate $\downarrow$	BB Bias $\downarrow$	Trigger rate	Update (s) $\downarrow$
Qwen-3-4B	13.491 $\pm$ 6.994	0.298 $\pm$ 0.065	0.359 $\pm$ 0.249	0.713 $\pm$ 0.427	0.284 $\pm$ 0.244	0.452 $\pm$ 0.191	-	N/A
DeepSeek-8B	21.361 $\pm$ 18.239	0.255 $\pm$ 0.033	0.310 $\pm$ 0.224	0.729 $\pm$ 0.413	0.324 $\pm$ 0.246	0.454 $\pm$ 0.177	-	N/A
Mistral	212.5 $\pm$ 6504.3	0.275 $\pm$ 0.040	0.397 $\pm$ 0.280	0.807 $\pm$ 0.368	0.372 $\pm$ 0.274	0.525 $\pm$ 0.188	-	N/A
Self-correction	22.092 $\pm$ 25.893	0.262 $\pm$ 0.059	0.268 $\pm$ 0.221	0.757 $\pm$ 0.403	0.287 $\pm$ 0.242	0.437 $\pm$ 0.173	-	N/A
DS-8B-debiased	22.894 $\pm$ 71.423	0.256 $\pm$ 0.023	0.340 $\pm$ 0.238	0.744 $\pm$ 0.408	0.330 $\pm$ 0.262	0.471 $\pm$ 0.181	-	N/A
Qwen-SGD	13.498 $\pm$ 6.688	0.298 $\pm$ 0.065	0.364 $\pm$ 0.244	0.723 $\pm$ 0.422	0.292 $\pm$ 0.245	0.460 $\pm$ 0.185	0.262	5.720

Table 3: Main comparison on BiasBench (toxic prompts). We report generation quality (PPL, Fluency), safety (BB Bias), and efficiency (trigger rate and update time) for static baselines, debiased checkpoints, and test-time adaptation methods. Lower is better for PPL, BB Bias, and update time; higher is better for Fluency. (

\dagger

indicates marginal significance,

\dagger\dagger

indicates significance, versus the corresponding baseline under a paired

t

-test.) Qwen-Prec-trig/no-trigger use our standard hyperparameter setting while Qwen-Prec-trig-2 uses another setting.

Backbone	Axis	Setting	PPL $\downarrow$	Fluency $\uparrow$	Trigger rate	BB Bias $\downarrow$	Update (s) $\downarrow$	Test (s) $\downarrow$	$\epsilon$	Seg	Tok/seg
Qwen-3-4B	Baseline	baseline	13.491	0.298	0.000	0.452	0.000	6.589	0.000	4	128
Qwen-3-4B	Epsilon	eps0.2	13.197	0.306	0.871	0.456	0.913	9.901	0.200	4	128
Qwen-3-4B	Epsilon	eps0.25	13.593	0.297	0.494	0.469	1.007	10.231	0.250	4	128
Qwen-3-4B	Segments	nseg2	13.373	0.294	0.335	0.468	0.918	9.934	0.300	2	128
Qwen-3-4B	Segments	nseg8	14.774	0.298	0.216	0.420	0.761	9.745	0.300	8	128
Qwen-3-4B	SegTokens	tok256	10.628	0.341	0.278	0.433	0.845	19.818	0.300	4	256

Table 4: TTA ablations on Qwen-3-4B and DeepSeek-8B (toxic prompts), including the following hyperparameters: precondition epsilon sweep, segments, tokens per segment, and multi-trigger; includes each model’s baseline.

Model	#Items	#Biased Items	Biased(%)
Base	20	3	15
Self-correction	20	3	15
CAP-TTA	20	2	10

Table 5: Item-level Bias score (binary), where a generated response is labeled as biased if the number of Yes votes among 5 annotators satisfies Yes

\geq 3

(i.e.,

\geq 3/5

). We report the percentage of biased items per model, computed on prompts where all three models have valid 5 bias ratings. Lower is better. We collected 20 valid prompt IDs from 31 human annotators.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Text	snlp	u-Unb	u-BERT	$\bar{b}$	Race	Sex	Rel.	Other	Triggered types
Black holes are fascinating objects in astrophysics.	0.0001	0.0302	0.5346	0.1883	0.1883	0.0000	0.0000	0.1883	[]
Asian students are good at math.	0.0014	0.4274	0.5608	0.3299	0.3299	0.0000	0.0000	0.3299	[race, other]
I hate those people. They are disgusting and should be removed.	0.9992	0.9220	0.9526	0.9579	0.9579	0.9579	0.9579	0.9579	[race, sex, religion, other]
The priest gave a sermon about compassion and forgiveness.	0.0000	0.0243	0.5397	0.1880	0.0000	0.0000	0.0000	0.1880	[]

Appendix

Table 8: Examples of BB-trigger gating and 4-type routing. For each text, we compute three raw toxicity/bias scores (snlp/roberta_toxicity_classifier, unitary/unbiased-toxic-roberta, unitary/toxic-bert). We use the committee mean

\bar{b}

as the BB-trigger score : if

\bar{b}>\epsilon

, we trigger test-time adaptation. In parallel, we compute four routing scores over { race , sex , religion , other }; any type whose score exceeds

\epsilon

is included in Triggered types .

Figure 2: This is evaluated using the bias trigger score. Per-prompt bias trajectories over narrative segments on the toxic prompt set. Each polyline corresponds to one prompt and tracks the bias/toxicity score across segments in the long-form generation protocol. This visualization highlights where bias spikes occur during generation and how CAP-TTA suppresses late-emerging bias by selectively triggering updates.

Figure 3: This is evaluated using the bias trigger score. This figure shows triggering trade-off for CAP-TTA. We compare the bias trigger score against epsilon threshold, see if it decreases the level of bias trigger score.

Figure 4: This is evaluated using the bias trigger score. This figure shows comparison of mean trigger-bias score across narrative segments for static baseline models on toxic prompts. Each point represents the average score at a given segment position. The trajectories illustrate how bias-related signals evolve during long generation across different static models.

Figure 5: This is evaluated using the bias final. This figure shows the empirical CDF of the bias metric ( bb_bias_final , lower is better) for Qwen3 under TTA-only and ablation variants.

Figure 6: This is graded by bias final. This figure shows the distribution of bias/toxicity scores across methods on the evaluation prompt set. Boxes summarize median and interquartile range, with whiskers indicating variability across prompts. Lower medians and reduced upper tails indicate better mitigation and fewer extreme harmful generations.

Figure 7: Ablation on test-time update latency (ECDF). This figure shows the empirical CDF of the total parameter-update time per prompt for different inference-time strategies on Qwen3. We compare unpreconditioned TTA-SGD (blue) against preconditioned CAP-TTA variants with different trigger thresholds $\epsilon$ . Curves further left indicate lower update-time overhead (better efficiency); the plot shows that preconditioning consistently reduces the update-time tail, while $\epsilon$ controls how often updates are triggered and thus the total overhead.

Figure 8: LoRA Structure. We only update a fraction of weights during TTA.

Question / Item	Yes	No
Does this submission meet the federal definition of research (HHS §46.102(l))?	X
Does this submission involve human subjects as defined in HHS §46.102(e)(1)?	X
Does this research involve interaction or intervention with subjects?	X
Does this research involve any procedures for which written consent is normally required outside of a research context?		X
If the subject signed or eSigned the consent document, would the signed consent document be the only record linking this subject to this study?	X

Appendix

Table 9: IRB Screening Responses (Survey-Only, Minimal Risk, Adults 18+).

Figure 9: Example of question given to human annotators.

Subset	Model	Fleiss’ $\kappa$	#Items used	#Items excluded ( $<5$ )
All items with $n{=}5$ (any model)	OVERALL	0.294	81	9
	qwen3-4B-base	0.415	29	1
	qwen3-4B-self-correction	0.178	29	1
	qwen3-4B-CAP-TTA	0.304	23	7
Prompt IDs with $n{=}5$ for all models	OVERALL	0.297	60	-
	qwen3-4B-base	0.433	20	-

Appendix

Table 10: Fleiss’

\kappa

for inter-annotator agreement on Bias (Yes/No). We compute

\kappa

only on items with exactly 5 bias ratings (fixed-

n

requirement). We report results for (A) all available items with

n{=}5

per model, and (B) a fair subset of common prompt IDs where all three models have

n{=}5

(20 prompt IDs; 60 items total across 3 models). A separate fluency question was used as an attention check and filter.

Benchmark	Metric	Typical baseline (orig.)
StereoSet	ICAT $\uparrow$	BERT-base: 71.2; GPT2-large: 70.5
CrowS-Pairs	Bias score ( $50$ ideal)	BERT: 60.5; RoBERTa: 64.1
RTP	ExpMax toxicity $\downarrow$	GPT-2: 0.75 (toxic) vs 0.51 (non-toxic)

Appendix

Table 11: Simple reference numbers reported in original benchmark papers (not our re-runs).

Narrative Generation

See all 32 papers

Decoding-Time Debiasing via Process Reward Models: From Controlled Fill-in to Open-Ended Generation
May 4, 2026Muneeb Ur Raheem KhanSocial Bias in Language Models Language Model Decoding
Lahore University of Management Sciences (LUMS)
Adaptive Triggering for Bias Correction in LLM Reasoning
Aug 26, 2026Nayoung Kim, Mickey Mancenido, Huan LiuLanguage Model Debiasing Demographic Bias in Language Models
School of Computing and Augmented Intelligence, Arizona State University, Tempe, AZ, USA · School of Mathematical and Natural Sciences, Arizona State University, Phoenix, AZ, USA
CatRAG: Functor-Guided Structural Debiasing with Retrieval Augmentation for Fair LLMs
Mar 23, 2026Ravi Ranjan, Utkarsh Grover, Mayur Akewar +2Language Model Debiasing Social Bias in Language Models
Florida International University Miami, USA · University of South Florida Tampa, USA

May 4, 2026J/K move · Enter open · S save

Decoding-Time Debiasing via Process Reward Models: From Controlled Fill-in to Open-Ended Generation

Muneeb Ur Raheem Khan

Lahore University of Management Sciences (LUMS)

Social Bias in Language Models Language Model Decoding Process Reward Models Debiased ML Linguistic Bias in Language Models Language Model Bias Evaluation+1

Open paper AskPDF arXiv

Large language models pick up social biases from the data they are trained on and carry those biases into downstream applications, often reinforcing stereotypes around gender, race, religion, disability, age, and socioeconomic status. The standard fixes (retraining on curated data or fine-tuning with human feedback) are expensive, need access to model weights, and risk degrading the model on other tasks. In this paper we take a different route: we debias the model at decoding time, treating bias mitigation as a structured search over candidate tokens without ever touching model weights. A separate Process Reward Model (PRM) acts as a judge, scoring each candidate for both fairness and fluency. We design three schemes of increasing sophistication (Best-of-N selection, Sequential critique-and-revise, and Constitutional self-audit) and evaluate them on four models (GPT-4o-mini, Llama 3.2 3B, Gemma 3 4B, Qwen 2.5 3B) across a 200-prompt bilingual benchmark in English and Urdu covering eight bias categories. Sequential debiasing proves the most effective, raising mean bias scores by up to +0.40 over baseline while preserving (and sometimes improving) fluency. We then extend all three schemes to open-ended generation, where each token is debiased on the fly, and introduce a lightweight Bias Guard gate that fires only on potentially biased words, keeping overhead near 2x for well-calibrated models. A formal overhead metric that separates generator cost from judge cost reveals that Best-of-N is effectively free on the generator side in a native implementation. GPT-4o-mini, included as a strong proprietary anchor, confirms that the framework scales with model capability; the three open-weight models show where current small-scale LLMs still struggle.

Abstract

Narrative Generation

Explore similar work

Decoding-Time Debiasing via Process Reward Models: From Controlled Fill-in to Open-Ended Generation

Adaptive Triggering for Bias Correction in LLM Reasoning

CatRAG: Functor-Guided Structural Debiasing with Retrieval Augmentation for Fair LLMs