Understanding the internal `thinking' process of Large Language Models (LLMs) and %how this relates their connections to hallucinations remains a key challenge. To this end, we introduce latent debate, a structured surrogate framework for interpreting model outputs on True/False prediction tasks through the lens of internal latent arguments and interactions amongst them. Unlike human debates, latent debate captures the hidden supporting and attacking signals that arise within a model during a single inference. We first present a model- and task-agnostic conceptual framework, and then instantiate it symbolically to approximate the thinking process of LLMs towards binary decisions. Empirical studies demonstrate that our latent debate is a faithful structured surrogate model that has highly consistent predictions with the original LLM, while providing a form of interpretability. We also demonstrate that our latent debate provides a strong baseline for hallucination detection. Specifically, we identify strong correlations between debate patterns and hallucinations, such as a high degree of disagreement in the middle layers of the latent debate surrogate is linked to a higher risk of hallucinations. Our findings suggest that latent debate provides a new framework for understanding the internal processes of LLMs.
Figures & tables
Figure 1: Visualizations of latent debate for two claims (both predicted as True) using selected tokens of Llama-8B. Red/blue cells represent attacking/supporting arguments, respectively. High controversy indicates hallucination.
Figure 2: The conceptual framework of latent debate. Given an input (e.g., a claim ), a set of latent arguments are identified , i.e., model components (raw latent signals) that convey the model’s opinions toward the input . These arguments are then processed by the argument interpreter , identifying the arguments’ supporting or attacking stance towards the input . The resulting attacking and supporting arguments are fed into the thinking module , which reaches the final True/False decision .
Figure 3: QBAFs and LLMs. (a) An example QBAF showing how initial strengths ( τ ) of arguments n1,…,n4 are transformed through gradual semantics based on attacking (-) and supporting (+) relations to produce final strengths ( σ ). (b) Skeleton of a QBAF drawn from an LLM architecture, where each node represents a specific token at a specific layer. To obtain a QBAF, the directed edges need to become attacks or support and the nodes need to be equipped with an initial strength.
common_claim
counterfact
company
TriviaQA
MuSiQue
TruthfulQA
Avg
500
500
500
500
500
500
–
Llama-8B (%)
Random
76.8
64.2
67.4
66.4
68.8
79.2
70.5
Average
92.2
67.4
80.4
73.0
77.0
90.2
80.0
Majority Voting
92.4
67.0
80.4
73.0
77.0
90.2
80.0
Latent Debate
92.4
78.2
89.2
74.0
77.0
90.6
83.6
Table 1: Consistency scores on binary and open-ended question-answering datasets with three LLMs. Each dataset contains 500 examples. The first three datasets use the last six prompt tokens, while TriviaQA, MuSiQue, and TruthfulQA use the last five. Avg is the arithmetic mean across the six datasets. The best scores within each model are shown in bold.
Method
Common Claim
CounterFact
Company
TriviaQA
MuSiQue
TruthfulQA
Avg
Hallucination Ratio
40.8%
15.8%
30.2%
25.0%
35.2%
45.8%
32.1%
AvgInit
0.75
0.66
0.86
0.91
0.71
0.58
0.74
AvgFin
0.58
0.54
0.68
0.82
0.62
0.62
0.64
VarInit
0.82
0.70
0.89
0.91
0.78
0.54
0.77
VarFin
0.78
0.63
0.78
0.90
0.65
0.50
0.71
NumAtk
0.55
0.53
0.68
0.65
0.51
0.55
0.58
Table 2: AUROC scores for the identified features in isolation (AvgInit, …, NumAtk), hallucination detection baselines, and Latent Debate Detector. Results are reported as the average score over three random seeds where available. The best results are highlighted in bold.
Figure 4: Feature importance across layer regions (Lower, Middle, Upper) and feature types. NumAtk: number of attacks; VarFin: variance of final strength; AvgFin: average of final strength; VarInit: variance of initial strength; AvgInit: average of initial strength.
Metric
Interpreter
Common Claim
CounterFact
Company
Average
Prediction Consistency
Logit Lens
92.40
78.20
89.20
86.60
Tuned Lens
99.80
96.40
83.40
93.20
Hallucination Detection (AUROC)
Logit Lens
83.68±0.31
82.50±0.31
93.44±0.41
86.54
Tuned Lens
59.24±0.29
57.33±0.67
84.54±0.20
67.03
Table 3: Comparison of Logit Lens and Tuned Lens as argument interpreters in latent debate. Consistency measures agreement with the original LLM’s predictions, while AUROC evaluates hallucination detection. The best results are highlighted in bold.
Connection Topology
Common Claim
CounterFact
Company
TriviaQA
MuSiQue
TruthfulQA
Avg
Original
92.40
78.20
89.20
76.80
77.80
91.20
84.27
Adjacent Grid
79.00
43.80
64.00
52.20
55.80
56.80
58.60
Original Quadratic
91.80
64.00
80.00
71.60
75.00
88.60
78.50
Fully Quadratic
56.20
38.80
43.40
49.80
48.60
50.00
47.80
Table 4: Comparison of different graph topologies in latent debate. Consistency scores (%) measure agreement with the original LLM predictions. The best results are highlighted in bold.
Figure A1: 3D surface plot of the semantics function σ .
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
common_claim
counterfact
company
TriviaQA
MuSiQue
TruthfulQA
Avg
500
500
500
500
500
500
Llama-8B (%)
Top Right Argument
92.4
84.2
85.6
88.2
80.6
91.2
87.03
Latent Debate
92.4
78.2
89.2
74.0
77.0
90.6
83.57
Mistral-7B (%)
Top Right Argument
96.2
95.8
99.8
98.6
96.8
95.8
97.17
Appendix
Table A1: Consistency scores. Each entry shows the proportion of consistent predictions (out of 500).
common_claim
counterfact
company
TriviaQA
MuSiQue
TruthfulQA
Avg
500
500
500
500
500
500
Llama-8B (%)
Top Right Argument
0.73
0.49
0.69
0.59
0.60
0.36
0.58
Latent Debate Detector
0.82
0.72
0.88
0.93
0.76
0.60
0.79
Appendix
Table A2: AUROC for hallucination detection. We compare our latent debate detector with a single-argument baseline.
Method
Common Claim
Counterfact
Company
Avg
SAPLMA-L1
0.79
0.66
0.88
0.78
SAPLMA-L2
0.80
0.67
0.89
0.79
SAPLMA-L3
0.80
0.68
0.87
0.78
SAPLMA-L4
0.79
0.69
0.91
0.80
Latent Debate Detector
0.84
0.83
0.93
0.87
Appendix
Table A3: Hallucination detection performance of SAPLMA across different layers, compared with the latent debate-based detector. Latent debate consistently achieves the best performance across all datasets.
Setup
Common Claim
CounterFact
TriviaQA
TruthfulQA
Original
81.50
72.44
92.60
60.49
Remove first 3 layers
↑1.10
↓2.55
↑0.23
↑0.25
Remove first 6 layers
↓0.64
↑0.18
↑0.25
↓1.15
Remove first 9 layers
↓0.51
↑3.18
↑1.58
↓1.86
Remove first 12 layers
↑0.11
↑2.58
↑1.41
↓2.07
Appendix
Table A4: Impact of removing lower layers on hallucination detection.
Method
CounterFact
Company
TriviaQA
MuSiQue
TruthfulQA
Avg
SAPLMA
67.91±2.04
80.13±1.75
80.21±0.85
67.21±2.80
49.02±1.13
68.89
Latent Debate Detector
80.94±0.19
85.26±0.23
89.29±1.00
68.02±1.07
52.81±0.33
75.26
Appendix
Table A5: Cross-task hallucination detection performance (AUROC). Detectors are trained on Common Claim and evaluated on five other datasets without retraining.
Model
Common Claim
CounterFact
Company
Avg
Llama-8B
92.4
78.2
89.2
86.6
Llama-13B
98.4
95.2
99.6
97.7
Qwen-8B
87.6
89.0
99.0
91.9
Qwen-14B
89.2
85.0
98.8
91.0
Appendix
Table A6: Prediction consistency of latent debate across different model families and scales.
Large language model (LLM) interpretability studies how internal representations and computations produce outputs, and how they can be explained or controlled.