Adaptive Triggering for Bias Correction in LLM Reasoning
Authors: Nayoung Kim, Mickey Mancenido, Huan Liu
Organizations: School of Computing and Augmented Intelligence, Arizona State University, Tempe, AZ, USA · School of Mathematical and Natural Sciences, Arizona State University, Phoenix, AZ, USA
Chain-of-thought prompting can expose and amplify demographic stereotypes within an LLM's intermediate reasoning and create a failure mode that final-answer debiasing alone cannot address. Mitigating such bias during generation presents a fundamental timing problem: intervening too late allows biased reasoning to propagate, while unnecessarily intervening can disrupt otherwise correct reasoning. Existing approaches largely avoid this decision by either evaluating completed reasoning chains post hoc or intervening at predetermined steps, leaving open when a developing reasoning trajectory provides sufficient evidence to warrant correction. We formulate this decision as an online change-point detection problem. A per-step bias signal updates a CUSUM statistic and a targeted correction is injected only when accumulated evidence crosses a detector-specific threshold calibrated on held-out data. We instantiate the framework with a white-box signal derived from next-token probabilities and a black-box signal obtained from an LLM judge, enabling deployment with both open-weight and hosted models. On gpt-4o-mini adaptive black-box triggering recovers most of the disambiguated-context accuracy lost under fixed-interval intervention while requiring substantially fewer interventions. That result holds even with an independent judge. Across six open-weight models, the white-box signal improves ambiguous-item accuracy on all six but reduces disambiguated-item accuracy on five because it cannot distinguish unsupported stereotype reliance from correct, stereotype-congruent evidence.
Figures & tables
Fig. 1: The Adaptive Triggering control loop (Section III-C ). White- and black-box variants differ only in how they compute the per-step bias-risk signal bt (Section III-B ); the downstream CUSUM update, trigger logic, cooldown, and intervention are shared. Generation terminates with a final answer or, if the step cap is exhausted, without one.
Fig. 2: Trajectory-level example from the Age category using Mistral-7B-Instruct-v0.3, quoted verbatim. Steps 1 and 2 receive the same maximum judge score, but the trigger fires only at step 2 because evidence accumulates across steps: S1=0.60 does not exceed h=0.60 , whereas S2=1.20 does. The resulting intervention recovers the correct answer. The un-intervened and fixed-schedule conditions instead select the stereotype-consistent answer; the fixed schedule does not fire because the trajectory ends before its predetermined intervention point.
Fig. 3: Accuracy and intervention rate on the 504-item stratified gpt-4o-mini subsample. adaptive_bb recovers most of the disambiguated accuracy lost under fixed_interval . The intervention-rate panel pools both context types; adaptive_bb averages 0.60 interventions per disambiguated item and 0.06 per ambiguous item, compared with 1.00 for fixed_interval on both splits.
Condition
Disambig.
Ambig.
Bias
Interv./item
none
92.1%
94.0%
4.8%
0.000
fixed
82.9%
95.2%
4.4%
1.000
bb , self-judge
90.1%
95.2%
4.0%
0.331
bb , indep. judge
89.7%
93.7%
5.2%
0.163
Comparison
Diff.
95% CI
p
vs. self, disambig.
−0.40
[−3.17,+2.38]
1.00
TABLE II: Independent-judge replication on gpt-4o-mini ( n=252 per split). The upper block reports condition-level outcomes; the lower block reports paired comparisons for the independent-judge condition using stratified bootstrap CIs and McNemar’s exact test.
Fig. 4: Relationship between fixed-schedule exposure and disambiguated-accuracy change. The same N=5 schedule produces different firing rates because models generate traces of different lengths. Marker area denotes mean trace length. DS-Llama-8B is the principal exception to the cross-model pattern. Values are reported in Table III .
Model
Mean steps
Fired items
Δ Acc.
Llama-8B
6.40
65.5%
−10.13 pp
Qwen-1.5B
5.40
46.7%
−4.85 pp
Mistral-7B
3.86
9.5%
+0.30 pp
Qwen-7B
3.56
3.3%
−0.28 pp
Gemma-9B
2.78
0.0%
+0.27 pp
DS-Llama-8B
8.81
99.0%
+1.25 pp
TABLE III: Effective firing rate and disambiguated-accuracy change under the same N=5 fixed schedule. Because firing depends on trace length, a common cadence produces substantially different intervention exposure across models. Δ Acc. is fixed_interval minus none .
Fig. 5: Fixed-interval cadence sweep ( N∈{2,3,5,8,10} ) and adaptive conditions on a 504-item Qwen2.5-1.5B-Instruct subsample, plotted against observed intervention rate. Comparing measured firing rates rather than nominal N separates adaptive selection from simply intervening less often. Accuracy excludes non-completions.
Fig. 6: Accuracy change from none to adaptive_wb across six open-weight models. The white-box condition improves ambiguous-item accuracy on all six models but reduces disambiguated-item accuracy on five. Bars show category-stratified complete-case differences; Table IV reports the underlying accuracies and significance tests.
Disambiguated
Ambiguous
Model
none
wb
Δ
none
wb
Δ
Mistral-7B
81.5%
74.0%
−7.0
73.0%
74.8%
+1.3
Gemma-9B
74.7%
61.1%
−12.6
94.6%
97.3%
+2.8
Qwen-1.5B
71.8%
55.4%
−15.6
57.5%
66.5%
+9.9
Qwen-7B
79.5%
73.0%
−6.7
94.8%
96.6%
+2.1
Llama-8B †
73.5%
71.7%
−1.9
92.5%
93.1%
+1.3∗
TABLE IV: adaptive_wb accuracy relative to none at full nine-category scale for models with verified results across all categories. Δ is the category-stratified complete-case estimate ( wb − none ). Accuracy columns pool items, whereas Δ averages category-level differences; the two conventions differ by at most 1.0pp on any row.
Configuration
Acc.
Bias
Score
Interv./item
wb , selected ( k=.45,h=.30 )
75.4%
2.4%
+73.0
0.532
wb , default ( k=.30,h=.45 )
75.4%
2.4%
+73.0
0.532
bb , selected ( k=.25,h=.75 )
81.7%
4.8%
+77.0
0.135
bb , default ( k=.40,h=.60 )
81.0%
4.8%
+76.2
0.143
TABLE V: Held-out calibration results on Qwen2.5-7B-Instruct ( n=252 ). Accuracy is measured on disambiguated items and bias on ambiguous items. “Score” is the calibration objective defined in Algorithm 1 .
Comparison
Split
Acc. (A)
Acc. (B)
p
bb vs. wb
disambig.
64.6%
55.4%
<0.0001
bb vs. wb
ambig.
69.2%
66.5%
<0.0001
bb vs. fixed
disambig.
64.7%
67.0%
<0.0001
bb vs. fixed
ambig.
69.2%
62.8%
<0.0001
TABLE VI: Item-paired accuracy comparisons on Qwen2.5-1.5B-Instruct at full nine-category scale. Comparisons use the complete-case item set; p -values are from McNemar’s exact test.