Lingtai: What Concept Geometry Reveals--and Does Not Reveal--About LLM Inference

Jiangang Chen

cs.CLSep 30, 2026

Lingtai: What Concept Geometry Reveals--and Does Not Reveal--About LLM Inference

Authors: Jiangang Chen

Organizations: Chengdu Beiluoshimen Technology Co., Ltd.

Abstract

Observing what a large language model computes during autoregressive inference--online and without training probes--remains difficult. We introduce Lingtai, a training-free concept telemetry layer: at each generation step, residual states are projected onto a domain-specific bank of named concept anchors, constructed without labeled concept examples, outcome labels, gradient fitting, or activation-space optimization, producing a structured per-step concept-coordinate signal. Across code generation and grade-school mathematical reasoning, this signal exhibits a robust association with predictive uncertainty: the association survives problem-identity and token-position controls and is not attributable to a single token type, is not explained by a simple correct/incorrect mixture on GSM8K, and is not reproduced by matched random anchors; it is markedly weaker or direction-inconsistent in K-means and PCA projections. Two structures emerge: a recurring uncertainty-linked activity signal whose functional geometry is task-conditioned (distinct activity-entropy shapes on HumanEval, MBPP, and GSM8K), and an execution-specific trajectory identity with strong local inertia but weak re-instantiation invariance--under completion-only elastic alignment, corruption at k=32 (approximately a median quarter of the completion) on the matched re-execution subset still retrieves the archived episode at 62.0%, while a fresh execution retrieves it only 11.7-16.0% of the time. Finally, a matched audit finds no evidence that the scalar concept-activity signal used here supplies a stable correctness coordinate under the tested protocol; we therefore treat correctness as externally supplied. Telemetry adds 0.7-1.6% per-token decode overhead for the 161-anchor code implementation, with unchanged generated tokens.

Figures & tables

Figure 1: Lingtai concept telemetry: residual stream at a chosen layer

\rightarrow

RBF projection onto a domain-specific bank of named concept anchors

\rightarrow

per-step concept-activity signal; anchors constructed without labeled concept examples or activation-space fitting; most are corpus-discovered, with fixed arithmetic primitives added only in the math bank.

Representation	Dim	Training	Named coordinates	Token $\rho$	AUROC	Mean problem $\rho$ [CI]
Concept telemetry (ours)	161	none	yes	0.297	0.722	0.284 [0.238, 0.329]
Raw hidden	3584	none	no	0.384	0.736	0.387 [0.356, 0.417]
K-means	161	unsup.	no	0.102	0.630	0.140 [0.085, 0.192]
PCA top-10	10	unsup.	no	$-0.070$	0.551	0.026 [ $-0.028$ , 0.073]
PCA top-20	20	unsup.	no	$-0.015$	0.573	0.078 [0.031, 0.122]
PCA top-161	161	unsup.	no	0.033	0.570	0.118 [0.071, 0.164]

Table 1: External-validity audit, HumanEval / Qwen2.5-Coder-7B-Instruct (50 problems); criterion: predictive entropy; rank-based metrics. K-means and PCA are fitted transductively (in-sample, favorable to the baselines); K-means uses

k

-means++ (

n_{\mathrm{init}}=10

) on all evaluated tokens. AUROC uses the top entropy decile over pooled tokens as the positive class.

Figure 2: The activity–entropy relation is task-conditioned, not universal. Within-problem deciles of mean predictive entropy by concept-activity decile (50 problems/panel; error bars = bootstrap 95% CI).

Figure 3: Execution fingerprints are locally robust but re-instantiation brittle (OpenCoder-1.5B-Base, centered self-margin, 402-episode bank). Left: top-1 self-retrieval under a contiguous corruption block (

k=1..128

; shaded range across four position bands;

k=128

replaces a median 25% of completion tokens). Right: exact replay vs fresh sample, paraphrase, and permutation null. Fresh-sample queries are the bank’s 94 assertion/runtime episodes, re-sampled at temperature 0.7 under seeds 42 and 7 (left panel: 402-episode bank; dashed line: fresh re-execution rate on the 94-task subset, seed 42; right panel bars: seed 42); paraphrase queries are 30 tasks per seed.

Figure 4: Position-matched correctness audit (OpenCoder-1.5B-Base, 163 replayed HumanEval problems). Left: pre-output AUROC for

K_{\mathrm{var}}

and predictive entropy at the prompt end (

k=0

), the pre-output divider (

d-1

), and the synchronization point (

d_{0}

); the paired

\Delta

AUROC for entropy against the prompt-average-entropy baseline has CIs including 0 at all three points. Right: no monotone rise along generation progress. The earlier unmatched F1 = 0.81/0.84 claim is invalidated.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Benchmark	Model (precision), layer	$n$	Trajectory source
HumanEval, MBPP	Qwen2.5-Coder-7B-Instruct (NF4), L23	50 each	recorded greedy completion
HumanEval	Qwen2.5-Coder-14B (INT8), L40	50	recorded greedy completion
GSM8K	Phi-2 (FP16), L28	50	stored own completion (replay)
MBPP fingerprints	OpenCoder-1.5B-Base (FP16), L10	402; 94 $\times$ 2; 30 $\times$ 2	Panel A teacher-forced; B re-sampled
Matched audit	OpenCoder-1.5B-Base (FP16), L23	163	recorded + re-generation check

Appendix

Table 2: Configurations and extraction protocol. For the four external-validity configurations, trajectory telemetry is extracted by teacher-forcing a recorded greedy completion; no model is re-sampled during extraction. Process selection: the 50-problem audit subsets are deterministic ID-ordered prefixes fixed before analysis, with no selection on telemetry or outcome (GSM8K: first 50 of the test split). Panel B fingerprint queries are separately re-sampled at temperature 0.7 (seeds 42/7); all other trajectories are greedy. Fingerprint row: 402-episode bank; 94 fresh-sample and 30 paraphrase queries per seed.

Setting	Problems with positive paired $d_{\log}$	Token $\rho$
HumanEval / Qwen2.5-Coder-7B-Instruct	47/50	0.297
MBPP / Qwen2.5-Coder-7B-Instruct	47/50	0.236
HumanEval / Qwen2.5-Coder-14B	40/50	0.235
GSM8K / Phi-2	50/50	0.443

Appendix

Table 3: Direction replication of the concept-telemetry row: problems with a positive paired

d_{\log}

against random anchors, and token-level Spearman

\rho

, per configuration. Configurations differ in model, quantization, layer, and anchor bank; we compare direction only, never absolute values.

$\gamma$	Token $\rho$	Mean problem $\rho$	Mean $d_{\log}$
0.25	0.295	0.282	0.874
0.5	0.297	0.283	0.867
1.0	0.297	0.284	0.865
2.0	0.297	0.284	0.855
4.0	0.296	0.284	0.844

Appendix

Table 4: Kernel-bandwidth sensitivity (HumanEval / Qwen2.5-Coder-7B-Instruct).

Layer	Token $\rho$	Mean problem $\rho$ [CI]
5	$-0.110$	$-0.084$ [ $-0.114$ , $-0.053$ ]
13	0.115	0.131 [0.102, 0.160]
20	0.262	0.249 [0.206, 0.286]
23	0.297	0.284 [0.238, 0.328]
26	0.306	0.307 [0.268, 0.347]

Appendix

Table 5: Layer sweep with per-layer anchor centers (last-token title residuals re-encoded at each layer).

Concept	Deg.	Source snippet $\rightarrow$ nearest anchors
typer app name	100	import typer; app = typer.Typer() $\rightarrow$ order data list (0.69); import license module (0.68)
int float list	78	class FocalNetConfig(...) $\rightarrow$ order data list (0.83); node none list (0.82)
parser args add_argument	38	parser = argparse.ArgumentParser() $\rightarrow$ args parser add_argument (0.98)
license import lightning	32	# Copyright 2025 the HuggingFace Team $\rightarrow$ torch import license (0.54)
tensor self torch	32	def _quantize_tensor(...) $\rightarrow$ model self torch (0.94)
00 hours minutes	8	“Josh has soccer practice on Monday…” $\rightarrow$ hours hour minutes (0.83)

Appendix

Table 6: Five code and five math concepts (co-occurrence degree, source snippet, nearest anchors with cosine).

	Learned probe	SAE	TCAV/CAV	Lingtai
Outcome/property labels	usually yes	no	concept exemplars	no
Activation-space fitting	yes	yes	yes	no
Model/layer-specific learned representation	yes	yes	yes	anchors re-embedded
Named coordinates	property-specific	post-hoc	yes	by construction
Goal	prediction	sparse decomposition	attribution	runtime telemetry

Appendix

Table 7: Positioning relative to learned-feature alternatives.

Explore similar work

Confidence Geometry Reveals Trace-Level Correctness in Large Language Model Reasoning
May 16, 2026Shuo Liu, Ding Liu, Shi-Ju RanLLM Reasoning Strategies Confidence Estimation
School of Computer Science and Technology, Tiangong University, Tianjin, China. · Center for Quantum Physics and Intelligent Sciences, Department of Physics, Capital Normal University, Beijing, China.
Tyler: Typed Latent Reasoning for Language Models -- When to Think, What to Compute, and How Much to Allocate
Jun 15, 2026Hanyu Lin, Min Cai, Jiawei Wen +1Chain-of-Thought Reasoning
1Shenzhen University · University of Alberta
Laguerre Geometry for Interpreting Large Language Models
Jul 12, 2026Chunwei Ma, Russell WolfingerAutoencoder Architectures Invertible Flows
JMP Statistical Discovery, Cary, NC, USA

May 16, 2026, cs.LGJ/K move · Enter open · S save

Confidence Geometry Reveals Trace-Level Correctness in Large Language Model Reasoning

Shuo Liu, Ding Liu, Shi-Ju Ran

School of Computer Science and Technology, Tiangong University, Tianjin, China. · Center for Quantum Physics and Intelligent Sciences, Department of Physics, Capital Normal University, Beijing, China.

LLM Reasoning Strategies Confidence Estimation Reasoning Errors Traces Receiver Operating Characteristic Area Under The Curve

Open paper AskPDF arXiv

Large language models (LLMs) generate not only reasoning text, but also token-level confidence trajectories that record how uncertainty evolves during inference. Whether these trajectories are relevant to reasoning correctness remains unclear. Here we show that confidence trajectories encode a content-agnostic confidence geometry associated with trace-level final-answer correctness. Using only token-level confidence values, without access to the input question, reasoning text, hidden states, or external verifiers, we find that low-dimensional representations of confidence trajectories separate correct from incorrect reasoning traces. Across GSM8K, MATH, and MMLU, this geometric separation is quantitatively linked to downstream predictability: stronger clustering of correct and incorrect traces, measured by the Davies--Bouldin index, consistently corresponds to higher correctness-discrimination AUC. We further show that correctness-related information is enriched in the tail of reasoning, suggesting that late-stage confidence dynamics carry key correctness signals. We propose NeuralConf, a lightweight estimator that learns from confidence trajectories for correctness evaluation. Under a fixed trace budget, NeuralConf-derived scores improve confidence-weighted answer aggregation over majority voting, tail confidence, and other static baselines. These results reveal that LLMs expose trace-intrinsic statistical signals of correctness through their own confidence dynamics, offering a route to improve inference using information already present within generation.

Abstract

Explore similar work

Confidence Geometry Reveals Trace-Level Correctness in Large Language Model Reasoning

Tyler: Typed Latent Reasoning for Language Models -- When to Think, What to Compute, and How Much to Allocate

Laguerre Geometry for Interpreting Large Language Models