Surprise Alone Learns Lies: Evidence-Gated Continual Learning for LLMs from Streaming Claims

Abdelkader Metales; Miriam Grabenheinrich

cs.LGOct 29, 2026

Surprise Alone Learns Lies: Evidence-Gated Continual Learning for LLMs from Streaming Claims

Authors: Abdelkader Metales, Miriam Grabenheinrich

Organizations: caaise GmbH

Abstract

Continual learning methods for large language models increasingly decide what to learn by surprise, keeping the inputs the model finds least predictable. This rule is unsafe when the stream contains lies. A planted falsehood about something the model knows has low probability under the model's belief, so it is highly surprising by construction, and a surprise-driven learner absorbs nearly every such lie, at a rate similar to random selection with the same budget. We study the problem with LoRA-adapted students (Qwen2.5-1.5B, Qwen2.5-3B and Llama-3.2-3B) that learn from a stream of true and false claims about invented people and real CounterFact facts. We report four findings. (1) A verifier key, a judge that must approve each claim, removes lies to the extent that its verifier is accurate; real LLM judges are cautious and reject unsourced truths and even source-backed updates. (2) Calibrated regression tests on unrelated knowledge slow learning down without filtering out lies. (3) The original model's own prior flags claims that contradict what it knows, without a judge. It separates lies about known facts from true new facts (AUC 0.78-0.83 across three models) but not from legitimate updates of the same kind of fact (AUC 0.53). It saves 43% of judge calls, but an attacker with access to the model can choose lies that pass it. (4) If trusted sources exist that an attacker cannot plant, routing contradicted claims to evidence recovers legitimate updates and the combined system has the lowest poison rate among learned claims; when an attacker plants matching sources for 30% of the lies, the system ends up worse than the judge alone. The acceptance signals answer different questions, and none of them decides on its own whether a claim is true.

Figures & tables

Panel of Figure 1 — Figure 1: The acceptance decision for one claim ( sura/runner.py ). A claim is learned only if it is new, uncontradicted and verified. Claims are processed in chunks of 16; each key can be switched off for the ablations.

Set	Source	Size	Role
New facts	400 of the 600 facts generated for 120 invented people $\times$ 5 relations (birthplace, profession, language, instrument, sport)	400	True claims the model cannot know; should be learned
Lies about invented people	The other 200 generated facts, each given a wrong value from its relation’s pool	up to 200	Lies a source would contradict
Known-fact lies	CounterFact ( Meng et al., 2022 ) counterfactuals on facts the student knows	up to 200	Plausible lies about known facts
Known facts	CounterFact records the student ranks correctly (2,443 of 3,000 screened)	200 / 300 / 400	Regression suite / KL anchor / held-out retention set

Table 1: Claim sets of one world.

System	Learns from	Safety check
Naive	every claim	none
Surprise-only (SuRe-lite)	surprising claims, replay, fast/slow EMA	none
Stability-only (STABLE-lite)	every claim	rollback if suite exact-match drops $>0.02$
Verifier key	surprising and verified claims	none
Verifier key + test	surprising and verified claims	SPRT vs. original base, commit / rollback
Test only	surprising claims	SPRT (ablation: no verifier key)

Table 2: Systems compared. Every system uses the same student, stream and training loop.

Figure 2: Share of all streamed lies asserted after learning, by poison rate (oracle verifier; Qwen2.5-1.5B; 6 systems

\times

3 poison rates

\times

3 seeds; results/main ).

System (30% poison)	New facts learned	Old facts kept
Before learning	0.06	1.00
Naive	0.19	0.92
Surprise-only	0.80	0.93
Stability-only	0.17	0.90
Surprise + test, no verifier key	0.74	0.92
Verifier key	0.86	0.98

Table 3: Learning and retention at 30% poison (oracle verifier). Means over 3 seeds; seed standard deviations are at most 0.05, except for the verifier key with the uncalibrated test (0.18 at 30%, 0.35 at 50% poison), whose rollbacks made learning unstable.

Figure 3: Lies absorbed by the verifier key without test, against the share of lies the simulated judge approves (4 accuracies

\times

3 seeds; results/judge_sweep ).

	Rollbacks	Poison share of learned claims		New facts learned
Judge accuracy	per run (with test)	with test	without test	with test	without test
0.6	26.3	22.0%	22.2%	0.46	0.60
0.7	38.3	14.7%	15.3%	0.40	0.67
0.8	15.3	10.2%	9.8%	0.63	0.71
0.9	2.0	5.3%	5.3%	0.79	0.80

Table 4: The calibrated test with a simulated judge at 30% poison (3 seeds; results/judge_sweep ).

Figure 4: Known-fact lies asserted after learning, with and without the prior key (

g\geq 6

), across judge accuracies (3 seeds; results/prior_sweep ).

Figure 5: Trade-off curve of the prior gap on the fresh seed-7 world under each original model (200 claims per type). Each point is a threshold; the diagonal is chance.

Gap threshold	Known-fact lies held back	True new facts held back	Lies about invented people held back	Precision: lies among held-back claims
4	0.92 / 0.92 / 0.91	0.42 / 0.46 / 0.50	0.46 / 0.48 / 0.49	0.69 / 0.67 / 0.65
6 (used)	0.76 / 0.81 / 0.71	0.27 / 0.33 / 0.32	0.29 / 0.34 / 0.29	0.74 / 0.71 / 0.69
8	0.54 / 0.56 / 0.44	0.14 / 0.18 / 0.17	0.14 / 0.13 / 0.16	0.79 / 0.76 / 0.72

Table 5: Safety–learning operating points of the prior key on the fresh seed-7 world with 200 claims per type. Each cell gives Qwen2.5-1.5B / Qwen2.5-3B / Llama-3.2-3B (the larger models are introduced in Section 5.7 ). The fourth column tracks the third at every threshold, so for lies about unknown entities the key holds back lies and truths at the same rate. Precision compares known-fact lies with true new facts in this balanced sample (200 of each); with fewer lies in the stream it would be lower. Person-clustered bootstrap intervals for the AUC are 0.78–0.86 (Qwen2.5-3B) and 0.74–0.82 (Llama-3.2-3B), matching the claim-level ones.

Sources	System	Updates learned	New facts learned	Known-fact lies believed	Old facts kept	Lies per 1,000 learned claims
All claims	Judge only	0.90	0.79	0.18	0.92	42
All claims	Judge + prior	0.24	0.60	0.07	0.97	33
All claims	+ evidence route	0.98	0.79	0.11	0.91	18
Half of claims	Judge only	0.90	0.81	0.18	0.93	44
Half of claims	Judge + prior	0.24	0.60	0.07	0.97	32
Half of claims	+ evidence route	0.62	0.72	0.09	0.94	24

Table 6: Legitimate updates and the evidence route. Judge accuracy 0.9, means over 5 held-out seeds; seed standard deviations are at most 0.07. At judge accuracy 0.7 the pattern is the same (updates learned 0.70 / 0.19 / 0.93 with all sources and 0.59 with half; lies per 1,000 learned claims 129–132 / 112–114 / 60 and 79).

System	Known-fact lies believed	Lies about invented people	Updates learned	New facts learned	Old facts kept	Lies per 1,000 learned claims
Surprise-only (no judge)	1.00 / 1.00 / 1.00	0.79 / 0.81 / 0.87	0.99 / 0.99 / 0.99	0.77 / 0.81 / 0.84	0.88 / 0.89 / 0.88	257 / 251 / 254
Judge only	0.18 / 0.16 / 0.19	0.16 / 0.16 / 0.15	0.90 / 0.91 / 0.90	0.81 / 0.81 / 0.83	0.93 / 0.93 / 0.92	44 / 37 / 38
Judge + prior	0.07 / 0.04 / 0.05	0.12 / 0.11 / 0.12	0.24 / 0.17 / 0.20	0.60 / 0.57 / 0.57	0.97 / 0.97 / 0.97	32 / 25 / 29
+ evidence route	0.09 / 0.07 / 0.08	0.14 / 0.14 / 0.13	0.62 / 0.57 / 0.59	0.72 / 0.72 / 0.73	0.94 / 0.95 / 0.94	24 / 18 / 22

Table 7: Replication with three students. Each cell gives Qwen2.5-1.5B / Qwen2.5-3B / Llama-3.2-3B; judge accuracy 0.9; sources for half the claims; means over 5 held-out seeds, seed standard deviations at most 0.06. Judge accuracy 0.7 shows the same pattern for all three models.

Claim type	phi4 (14B)	Llama 3.2 (3B)
Known-fact lies	5.9%	4.0%
Lies about invented people	0.5%	0.0%
True new facts with a source	99%	65%
True new facts without a source	0%	0%
Legitimate updates with a source	30%	40%
Legitimate updates without a source	4%	1%

Table 8: Share of claims each real judge accepted, pooled over 5 seeds.

System	Updates learned	New facts learned	Known-fact lies believed	Old facts kept	Lies per 1,000 learned claims
Judge only	0.19 / 0.23	0.55 / 0.38	0.09 / 0.09	0.96 / 0.96	23 / 21
Judge + prior	0.07 / 0.08	0.40 / 0.28	0.06 / 0.04	0.98 / 0.98	16 / 10
+ evidence route	0.47 / 0.46	0.54 / 0.41	0.07 / 0.07	0.94 / 0.94	9 / 5

Table 9: Full systems with real judges (phi4 / Llama 3.2). Means over 5 held-out seeds; seed standard deviations at most 0.05. The evidence route rescued no lie.

Signal (original model)	Qwen2.5-1.5B	Qwen2.5-3B	Llama-3.2-3B
Prior gap	0.83	0.82	0.78
Claim NLL (surprise under the original model)	0.78	0.79	0.74
Confidence in its own answer (ignores the claim)	0.84	0.80	0.88
Prior gap, lies about invented people instead	0.49	0.50	0.49

Table 10: AUC for separating known-fact lies from true new facts on the fresh seed-7 world, by signal (200 claims per type; 95% bootstrap intervals are about

\pm 0.04

Signal	Question it answers	Lie about a known fact	Lie about an unknown entity	True new fact	True update of a known fact
Surprise	Is it new to me?	selects it	selects it	selects it	selects it
Prior conflict	Does it contradict what I believed?	holds most back (AUC 0.78–0.83)	blind (AUC 0.49)	passes about 70%	holds most back
Judge	Does a verifier approve it?	rejects 94–96% (real judges)	rejects nearly all	accepts mainly if sourced	rejects 60–70% even with a source
Evidence route	Does a trusted source state the same?	never matches ∗	never matches ∗	accepts if sourced ∗	accepts if sourced ∗
Regression test	Did unrelated knowledge degrade?	blind	blind	(no effect)	(no effect)

Table 11: What each signal can and cannot see, summarising Section 5. ∗ Under the assumption of trusted sources that an attacker cannot plant or copy.

Continual Learning for LLMs

Continual learning for large language models studies how models absorb new knowledge and skills while retaining previously learned capabilities.

See all 236 papers

Censored LLMs as a Natural Testbed for Secret Knowledge Elicitation
Mar 5, 2026Helena Casademunt, Bartosz Cywiński, Khoi Tran +3LLM Honesty Language Model Probing
Harvard University · Warsaw University of Technology · IDEAS Research Institute +2
Unlearners Can Lie: Evaluating and Improving Honesty in LLM Unlearning
May 9, 2026Renjie Gu, Jiazhen Du, Yihua Zhang +1LLM Unlearning LLM Honesty
Fudan University · Central South University · Michigan State University
Knowing but Not Correcting: Routine Task Requests Suppress Factual Correction in LLMs
May 7, 2026Zixuan Chen, Hao Lin, Zizhe Chen +6LLM Self-Correction LLM Reliability
1Shanghai Jiao Tong University · 2The Chinese University of Hong Kong · 3Ant Group

Mar 5, 2026J/K move · Enter open · S save

Censored LLMs as a Natural Testbed for Secret Knowledge Elicitation

Helena Casademunt, Bartosz Cywiński, Khoi Tran, Arya Jakkli+2

Harvard University · Warsaw University of Technology · IDEAS Research Institute+2

LLM Honesty Language Model Probing

Open paper AskPDF arXiv

Large language models sometimes produce false or misleading responses. Two approaches to this problem are honesty elicitation -- modifying prompts or weights so that the model answers truthfully -- and lie detection -- classifying whether a given response is false. Prior work evaluates such methods on models specifically trained to lie or conceal information, but these artificial constructions may not resemble naturally-occurring dishonesty. We instead study open-weights LLMs from Chinese developers, which are trained to censor politically sensitive topics: Qwen3 models frequently produce falsehoods about subjects like Falun Gong or the Tiananmen protests while occasionally answering correctly, indicating they possess knowledge they are trained to suppress. Using this as a testbed, we evaluate a suite of elicitation and lie detection techniques. For honesty elicitation, sampling without a chat template, few-shot prompting, and fine-tuning on generic honesty data most reliably increase truthful responses. For lie detection, prompting the censored model to classify its own responses performs near an uncensored-model upper bound, and linear probes trained on unrelated data offer a cheaper alternative. The strongest honesty elicitation techniques also transfer to frontier open-weights models including DeepSeek R1. Notably, no technique fully eliminates false responses. We release all prompts, code, and transcripts.

Abstract

Continual Learning for LLMs

Explore similar work

Censored LLMs as a Natural Testbed for Secret Knowledge Elicitation

Unlearners Can Lie: Evaluating and Improving Honesty in LLM Unlearning

Knowing but Not Correcting: Routine Task Requests Suppress Factual Correction in LLMs