Organizations: City University of Hong Kong · Tencent Jarvis Lab · Xi’an Jiaotong University · Renmin University of China · University of Science and Technology of China · Westlake University
Post-training updates change what large language models (LLMs) can answer, but may leave their answerability judgments misaligned. We study this knowledge-judgment desynchronization through functional feeling of knowing (FoK), which supports selective generation, retrieval triggering, and model routing without implying subjective experience. We propose two trainable channels: a knowledge channel that changes answerability and a FoK channel that judges the current knowledge state. Three stages train them: on-policy supervised fine-tuning (OP-SFT) updates knowledge from corrected on-policy answers; knowledge-delta FoK training (KΔ-FT) adapts judgments across intermediate knowledge states; and FoK-guided policy optimization (FGPO) trains the subsequent answer or explanation. Factual-QA experiments, cross-dataset evaluation, and three sequential updates show that explicit FoK alignment improves judgment quality on updated knowledge. MMLU and ARC-Easy separately assess capability preservation.
Figures & tables
Figure 1: FoK signal localizes late under causal restoration. Panel (a) shows a 7B example across tokens and layers; panel (b) aggregates FoK recovery across Qwen2.5 sizes over normalized depth.
Base
K-SFT
J-SFT
K → J SFT
Model
Acc.
AUROC
Δ Acc.
Δ AUC
Δ Acc.
Δ AUC
Δ Acc.
Δ AUC
Qwen2.5-0.5B
20%
0.519
+2
-0.016
-14
+0.213
+2
+0.211
Qwen2.5-1.5B
35%
0.529
+6
-0.035
+2
+0.255
+5
+0.248
Qwen2.5-3B
39%
0.739
+12
-0.220
+3
+0.038
+11
+0.051
Qwen2.5-7B
53%
0.724
+7
-0.042
+2
+0.088
+7
+0.100
Qwen2.5-14B
66%
0.762
+6
+0.004
+1
+0.030
+5
+0.068
Table 1: Knowledge and FoK under separate or sequential SFT updates. Base gives absolute values; the three SFT blocks give changes from Base. Accuracy changes are percentage points (pp); AUROC changes are unitless. Δ AUC denotes the change in AUROC. Higher accuracy and AUROC are better. Full illusion and omission rates are reported in Appendix I.1 .
Figure 2: FoK score and AUROC over pre-judgment reasoning tokens. Green/red curves show mean FoK score for examples whose final FoK token is Yes/No. J=0.5 denotes FoK uncertainty, while J≈1 or J≈0 denotes high certainty. The black curve reports AUROC of prefix-level J for predicting whether the final FoK token is Yes or No.
Train
Test
General
Method
Acc. ↑
AUROC ↑
Bias →0
Acc. ↑
AUROC ↑
Bias →0
MMLU ↑
ARC ↑
Base
0.680
0.702
0.381
0.672
0.688
0.402
0.687
0.811
+ Prob
0.680
0.723
0.409
0.672
0.730
0.407
0.687
0.811
+ N-Prob
0.680
0.723
0.409
0.672
0.730
0.407
0.687
0.811
+ EliCal
0.637
0.808
0.263
0.622
0.810
0.254
0.709
0.504
Vanilla K-SFT
0.874
0.458
0.500
0.546
0.519
0.500
0.713
0.840
Table 2: Overall performance after one TriviaQA knowledge update on Qwen2.5-7B. Train columns use questions seen during the update; Test columns use held-out TriviaQA questions; General reports MMLU and ARC-Easy. ↑ : higher is better; ↓ : lower is better; →0 : closer to zero is better. Bold and underline indicate the best and second-best results.
Figure 3: FoK error decomposition for Table 2 . Red circles show illusion, the conditional rate of false “can answer” judgments; blue squares show omission, the conditional rate of false “cannot answer” judgments. The horizontal gap shows which error source determines the signed Bias value.
Figure 4: Coverage–accuracy trade-off on Qwen3-8B (thinking mode). Each panel varies the FoK threshold and plots coverage (solid, the fraction of questions the model judges itself able to answer), conditional accuracy on that answered subset (dashed), and overall accuracy with unanswered examples counted as incorrect (dotted).
Table 3: Additional evaluation on Qwen2.5-7B. Left: all methods start from the same OP-SFT knowledge checkpoint; K Δ -FT adds the FoK adapter, so its combined answering state differs. Right: φ1 is frozen after round 1; φi is retrained over each round’s knowledge trajectory. Each round adds a disjoint shard from 5,000 update questions; all test columns use the same 1,000 held-out questions.
Stage
Acc. ↑
AUROC ↑
Illu. ↓
Om. ↓
Bias →0
Base
0.672
0.688
0.683
0.122
0.402
Stage 1: + OP-SFT
0.679
0.603
0.601
0.283
0.442
Stage 2: + K Δ -FT
0.699
0.770
0.478
0.187
0.333
Stage 3: + FGPO
0.692
0.809
0.432
0.184
0.308
Table 4: Held-out stage-wise ablation on Qwen2.5-7B. OP-SFT updates knowledge, K Δ -FT adapts FoK, and FGPO optimizes judgment-guided generation. Higher accuracy and AUROC and lower error rates are better; Bias closer to zero is better. Update-set results are in Table 26 .
Appendix figures & tables27 assets
Supplementary material from the paper’s appendix.
Appendix
Parameter
Value
Backbone
Qwen2.5-7B-Instruct
Training framework
LLaMA Factory
Adapter type
LoRA, rank 16, alpha 32, dropout 0
Parallelism
Multi-GPU DDP with DeepSpeed ZeRO-2
Precision
bfloat16
Vanilla K-SFT LR / epochs
10−4 / 5
Appendix
Table 5: Training hyperparameters for the main experiments.
Y
j
Response outcome
Base reward
1
yes
correct answer
+2.0
1
yes
wrong answer
−1.5
1
no
—
−1.0
0
no
valid explanation
+0.5
0
no
invalid explanation
−0.5
0
yes
correct answer
−0.5
Appendix
Table 6: Base FGPO reward RFoK+Rans by answerability target Y , judgment j , and response outcome.
Asset
Use in this paper
License or terms
TriviaQA
Knowledge-update corpus and in-domain QA benchmark
Apache-2.0 ( source )
MMLU
General capability benchmark
MIT License ( source )
ARC-Easy
General capability benchmark
CC BY-SA 4.0 ( source )
Qwen2.5-7B-Instruct and Qwen2.5-32B-Instruct
Main answer model and LLM judge
Apache-2.0 (Qwen 7B and 32B model cards)
Qwen3-8B
Thinking-mode backbone for the selective-prediction evaluation
Apache-2.0 (Qwen model card )
LLaMA-3.1 and LLaMA-2 checkpoints
Supplementary FoK synchronization results in Appendix G
Meta Llama 3.1 Community License and Llama 2 Community License ( 3.1 , 2 )
Appendix
Table 7: External assets used in the experiments and their publisher-specified licenses or terms of use.
Figure 5: Per-layer indirect effect on true-positive (TP) samples for the instruction-first (IF) prompt across six Qwen2.5 sizes. The solid orange curve shows the question-region effect, the dashed purple curve shows the suffix-region effect, and the red dashed line marks two-thirds of model depth. The question-region effect concentrates in early and middle layers, whereas the suffix-region effect rises mainly after this boundary at every tested scale.
Figure 6: Layer-wise weight delta ∥ΔWℓ∥F after one epoch of K-SFT (red) vs. J-SFT (blue) on Qwen2.5-7B (full SFT from base, same data budget). K-SFT changes concentrate in shallow/mid layers, while J-SFT changes remain small until the deep layers. This weight-side pattern is consistent with the depth separation observed in causal tracing.
Figure 7: ROME-style causal tracing on Qwen2.5-14B, broken down by outcome group. Rows: TP (answers correctly and judges yes), FP (answers incorrectly but judges yes), TN (answers incorrectly and judges no), and FN (answers correctly but judges no). Each row shows three samples, except FN, which has only one sample with p>0.4 . Each cell is the layer × token restoration heatmap; tokens are grouped into Sys (system), K-Q (knowledge question), and Hdr (response header). The red dashed line at layer 29 marks the depth boundary used in our analysis. The suffix-region (Hdr) signal appears mainly in deep layers across all four outcome groups, suggesting that the late FoK signal does not simply track answer correctness.
Figure 8: FoK dependence on updated knowledge. Left: In this evaluation, K → J training improves FoK but does not reach direct J-SFT performance. Right: J-SFT alone barely shifts answer accuracy.
Figure 9: In a separate small-scale study, we evaluate a fixed, trained K Δ -FT FoK channel across successive OP-SFT knowledge checkpoints. We report answer accuracy, FoK AUROC, and thresholded illusion and omission errors.
Base
K-SFT
J-SFT
K → J SFT
Model
EM
AUROC
EM
AUROC
Illu.
Om.
AUROC
EM
AUROC
Illu.
0.5B
4.2%
0.523
16.9%
0.502
0%
99%
0.801
13.1%
0.763
3%
1.5B
30.9%
0.527
35.5%
0.500
66%
48%
0.824
34.8%
0.815
28%
3B
35.4%
0.724
45.7%
0.519
24%
80%
0.830
45.4%
0.822
40%
7B
47.2%
0.735
55.8%
0.592
28%
51%
0.855
55.1%
0.833
82%
14B
59.8%
0.756
65.2%
0.767
73%
9%
0.850
66.1%
0.814
85%
Appendix
Table 8: Supplementary Qwen2.5 size-scaling experiments using a dedicated setup with 5,000 update questions and 1,000 held-out questions.
Model
Base AUROC
K-SFT AUROC
J-SFT AUROC
K → J SFT AUROC
K → J SFT EM
Note
Qwen2.5-7B
0.735
0.592
0.855
0.833
55.1%
Reference
LLaMA-3.1-8B (LR =2×10−6 )
0.646
0.503
0.786
0.836
8.6%
K → J forgetting
LLaMA-3.1-8B (LR =10−6 )
0.646
0.526
0.782
0.749
73.5%
LR corrected
LLaMA-2-7B
0.520
0.500
0.500
0.500
63.5%
No FoK improvement
LLaMA-2-13B
0.525
0.500
0.500
0.500
70.1%
No FoK improvement
Appendix
Table 9: Cross-family FoK synchronization results with 5,000 update questions and 1,000 held-out questions.
Answer model
FoK estimator
Test Acc.
Test AUROC
Base
Direct judgment
0.672
0.688
Base
Prob
0.672
0.730
Base
EliCal
0.622
0.810
OP-SFT
Direct judgment
0.679
0.603
OP-SFT
Prob
0.679
0.571
OP-SFT
EliCal
0.679
0.618
Appendix
Table 10: FoK evaluation before and after OP-SFT on the 1,000-question held-out TriviaQA split. All OP-SFT rows start from the same knowledge checkpoint.
New-shard
Test
φ1 (frozen)
φi (adaptive)
State
Acc. before → after
Acc.
AUROC
Illusion
AUROC
Illusion
Base
—
0.672
—
—
—
—
Round 1
0.575→0.972
0.659
0.798
0.455
0.798
0.455
Round 2
0.676→0.938
0.644
0.776
0.688
0.800
0.202
Round 3
0.631→0.938
0.661
0.754
0.752
0.797
0.195
Appendix
Table 11: Three sequential TriviaQA updates. Illusion is the conditional rate of predicting Yes on questions the current answering model gets wrong.
Table 13: SelfAware detail for the same zero-shot systems. TPR is the rate of predicting Yes on correctly answered questions; TNR is the rate of predicting No on incorrectly answered questions.
System
Held-out Acc.
FoK AUROC
Base
0.319
0.629
Vanilla K-SFT
0.199
0.585
Ours
0.366
0.647
Appendix
Table 14: Full-pipeline replication on Natural Questions. This held-out split differs from the zero-shot population in Table 12 .
Train Acc.
Train AUROC
Test Acc.
Test AUROC
Before edit
0.035
0.594
0.633
0.726
After edit
0.640
0.553
0.627
0.709
Appendix
Table 15: Knowledge and functional FoK before and after a MEMIT edit. Train denotes 200 edited zsRE facts; Test denotes 1,000 held-out TriviaQA questions. These states are compared only within this experiment.
Vanilla K-SFT
OP-SFT + K Δ -FT
Seed
Acc.
AUROC
Acc.
AUROC
42
—
—
0.699
0.770
43
0.522
0.635
0.665
0.770
44
0.522
0.627
0.670
0.790
0
0.520
0.619
0.665
0.778
Mean ± SD
0.521±0.001
0.627±0.008
0.675±0.016
0.777±0.009
Appendix
Table 16: Training-seed variation of Vanilla K-SFT and OP-SFT + K Δ -FT on held-out TriviaQA.
State
Test Acc.
Test AUROC
Shared Stage-1+2 initialization
0.699
0.770
FGPO, seed 1
0.692
0.809
FGPO, seed 43
0.677
0.810
FGPO, seed 44
0.672
0.815
FGPO mean ± SD
0.680±0.010
0.811±0.003
Appendix
Table 17: FGPO seed variation from one shared Stage-1+2 initialization, evaluated end to end on held-out TriviaQA. Seed 1 is the original full-method run.
Answer consistency
Questions
Mean J
0–0.2
840
0.413
0.2–0.4
178
0.486
0.4–0.6
71
0.571
0.6–0.8
29
0.591
0.8–1.0
31
0.808
Appendix
Table 18: Functional FoK among 1,149 questions with 16/16 incorrect sampled answers. Bin labels follow the evaluation grouping.
Reasoning shown
AUROC
Mean- J gap
Question only
0.60
0.05
25%
0.66
0.09
50%
0.67
0.10
75%
0.74
0.15
Full reasoning
0.91
0.32
Appendix
Table 19: FoK discrimination with progressively revealed reasoning on 440 MATH problems.
Format
Condition
J
ΔJ
Acc.
Δ Acc.
—
None
0.599
—
0.408
—
Words
Unrelated
0.184
−0.415
0.424
+0.016
Words
Related
0.674
+0.075
0.478
+0.070
Sentences
Unrelated
0.298
−0.301
0.436
+0.028
Sentences
Correct related
0.677
+0.078
0.510
+0.102
Sentences
Incorrect related
0.772
+0.173
0.397
−0.011
Appendix
Table 20: Context interventions on 1,460 questions after filtering gold-answer-containing contexts.
Template
Peak layer
Sensitive range
Original
20
16–22
A: rephrased
21
17–22
B: third-person elicitation
21
16–21
C: numeric labels 1/2
21
18–23
D: letter labels A/B
21
18–21
E: labels can/cannot
20
18–20
Appendix
Table 21: Causal-tracing sensitivity to judgment templates on 40 shared samples. Layers are reported out of 28.
Strategy
Update Acc.
Held-out Acc.
GSM8K
Average
Base
0.685
0.603
0.798
0.695
Vanilla SFT
0.992
0.452
0.368
0.604
Reptile meta-update
0.558
0.584
0.640
0.594
SDFT
0.529
0.565
0.367
0.487
Unlikelihood on wrong span
0.500
0.528
—
—
One-step lookahead KL
0.738
0.619
0.646
0.668
Appendix
Table 22: Alternative knowledge-update strategies on a separate TriviaQA split. Unlikelihood GSM8K was not reported, so its average is omitted.
Asset
Role
License or terms
SelfAware
Zero-shot QA
CC BY-SA 4.0 ( source )
Natural Questions
Transfer and replication
CC BY-SA 3.0 ( data source )
WebQuestions
Zero-shot QA
CC BY 4.0 ( source )
MATH
Reasoning evaluation
MIT ( repository )
GSM8K
Capability evaluation
MIT ( repository )
MEMIT
Editing implementation
MIT ( repository )
Appendix
Table 23: Additional assets and publisher-specified terms.
Model
Acc. (%)
AUROC
Illusion (%)
Omission (%)
Qwen2.5-0.5B
20
0.519
28
85
Qwen2.5-1.5B
35
0.529
56
38
Qwen2.5-3B
39
0.739
8
68
Qwen2.5-7B
53
0.724
51
20
Qwen2.5-14B
66
0.762
50
19
Qwen2.5-32B
68
0.819
62
8
Appendix
Table 24: Base values for the preliminary SFT comparison.
K-SFT
J-SFT
K → J SFT
Model
Acc.
AUC
Illu.
Om.
Acc.
AUC
Illu.
Om.
Acc.
AUC
Illu.
Om.
Qwen2.5-0.5B
+2
-0.016
-27
+13
-14
+0.213
-28
+15
+2
+0.211
-21
-10
Qwen2.5-1.5B
+6
-0.035
+6
-2
+2
+0.255
-28
-6
+5
+0.248
-21
-12
Qwen2.5-3B
+12
-0.220
+10
+18
+3
+0.038
+11
-29
+11
+0.051
+20
-41
Qwen2.5-7B
+7
-0.042
-15
+19
+2
+0.088
-9
-4
+7
+0.100
+6
-12
Qwen2.5-14B
+6
+0.004
+9
-4
+1
+0.030
+9
-9
+5
+0.068
+29
-17
Appendix
Table 25: Changes from Base under K-SFT, J-SFT, and sequential K → J SFT. Positive accuracy and AUROC changes indicate improvement; negative illusion and omission changes indicate improvement. All source values are retained.
Stage
Acc. ↑
AUROC ↑
Illu. ↓
Om. ↓
Bias →0
Base
0.680
0.702
0.653
0.109
0.381
Stage 1: + OP-SFT
0.966
0.575
0.589
0.319
0.454
Stage 2: + K Δ -FT
0.967
0.803
0.350
0.190
0.270
Stage 3: + FGPO
0.961
0.817
0.250
0.240
0.245
Appendix
Table 26: Update-set results for Base and the three cumulative training stages.
While large language models (LLMs) demonstrate strong capabilities across diverse user queries, they still suffer from hallucinations, often arising from knowledge misalignment between pre-training and fine-tuning. To address this misalignment, we reliably estimate a fine-grained, instance-level knowledge score via multi-sampled inference. Using the knowledge score, we scale the learning signal according to the model's existing knowledge, while encouraging explicit "I don't know" responses for out-of-scope queries. Experimental results show that this approach allows the model to explicitly express uncertainty when it lacks knowledge, while maintaining accuracy on questions it can answer. Furthermore, we propose evaluation metrics for uncertainty, showing that accurate discrimination between known and unknown instances consistently improves performance.
Detecting whether a model's internal knowledge is sufficient to correctly answer a given question is a fundamental challenge in deploying responsible LLMs. In addition to verbalising the confidence by LLM self-report, more recent methods explore the model internals, such as the hidden states of the response tokens, to capture how much knowledge is activated. We argue that such activated knowledge may not align with what the query requires, e.g., capturing the stylistic and length-related features that are uninformative for answering the query. To fill the gap, we propose GRADE (GRAdient Dynamics for knowlEdge gap detection), which quantifies the knowledge gap via the cross-layer rank ratio of the gradient to that of the corresponding hidden state subspace. This is motivated by the property of gradients as estimators of the required knowledge updates for a given target. We validate GRADE on six benchmarks, demonstrating its effectiveness and robustness to input perturbations. In addition, we present a case study demonstrating how the gradient chain can generate interpretable explanations of knowledge gaps in long-form answers. Code is available at https://github.com/yjEugenia/llm-aware
Yujing Wang, Yuanbang Liang, Yukun Lai +2
Beihang University · Cardiff University · King’s College London
Parametric knowledge in large language models (LLMs) is a cornerstone of their success, yet remains poorly understood. Existing knowledge benchmarks typically rely on predefined questions (e.g., "What is the birth date of M.L. King?"), evaluating only knowledge that benchmark designers explicitly choose to query, a problematic availability bias. In this paper, we introduce open knowledge evaluation, a new paradigm for benchmarking the stated knowledge of LLMs. Instead of asking narrow questions, it evaluates models on the knowledge they choose to surface in response to open-ended elicitation prompts (e.g., "Tell me everything you know about M.L. King"). This shifts the focus from predefined answer retrieval toward characterizing the knowledge models naturally express. We instantiate this paradigm with BeQu (Beyond Questions), a benchmark of 10,000 entities paired with reference corpora for statement verification. Using BeQu, we evaluate a broad range of language models and analyze the effects of reasoning effort, model scale, prompt format, and knowledge domain. Data and leaderboard are available on this work's GitHub repository and at bequ.scads.ai, the benchmark website.