cs.LGOct 16, 2026
SaveDo Knowledge and Reasoning Oriented Benchmarks Induce Stable Representation Signatures in LLMs?
Organizations: EPFL, Idiap Research Institute · Independent Researcher
Abstract
Internal signatures could support monitoring, adaptive computation, and mechanistic analysis, but benchmark prompts can be separable without any shared computational distinction. We test whether a prespecified knowledge/reasoning label defines a reusable internal signature in five contemporary 7B to 31B models. We treat the task as the unit of inference probes hold out complete tasks, labels are permuted across tasks, and the best layer and stream are reselected for every permutation. Raw probes recover an eight-versus-eight-subject MMLU partition in all five models (macro-F1 --), but prompt text predicts it nearly as well: under an identical probe configuration, the activation advantage is only -- macro-F1 and is not significant. At the prespecified level, lexical adjustment removes significance while the adjusted states still identify individual subjects, and the selected site changes across task panels and selection rules. Additionally, two released base-training trajectories show no common direction. These pooled layerwise measures recover the task label but identify no transferable knowledge/reasoning signature at the tested sensitivity, which cannot exclude moderate effects. The resulting protocol gives future internal monitors and intervention studies a sequence of tests to pass. Project page: https://parsa-ra.github.io/lmbehave/.
Figures & tables
Figure 1: Study design and results. Panel 1: token-mixing and feed-forward updates are read before residual addition, and the residual state after it, all at the final valid prompt token (shading is schematic). Panel 2: held-out-task inference exhaustively permutes task labels ( : two permuted labelings) and reselects the layer–stream site. Panel 3: raw probes recover the partition in every model, but the matched gain over prompt text is not significant, sites do not transfer consistently across the three task panels (P1 MMLU-Pro, P2 MMLU, P3 heterogeneous), and the two training trajectories disagree.
Figure 2: Two released base-training trajectories give different endpoint directions. Accuracy rises in both models. OLMo’s depth-averaged mean-shift ratios end below their earliest values, whereas Apertus’s end above them. Neither path is monotonic. Error bars are sample standard deviations across three paired example blocks (example-sampling variation only). Dashed lines separate documented training stages. OLMo labels use E/M/L for the early, middle, and late sampled checkpoint within each stage; Apertus labels give trillions of tokens seen.
| Model (checkpoints) | |||||
|---|---|---|---|---|---|
| OLMo 3 7B (7) | |||||
| Apertus 1.0 8B (11) | |||||
| Reference | chance | chance | : equal projected means | ||
Table 1: Means across three paired example blocks at the first and final checkpoints. Arrows point from the first to the final checkpoint. Figure 2 shows every checkpoint with sample standard deviations.
| Model | MMLU site / mean-shift ratio | max | raw macro-F1 / | matched macro-F1 / | lexical-64 macro-F1 / |
|---|---|---|---|---|---|
| OLMo 3 7B | 3 / FFN / 0.221 | 0.0721 | |||
| Apertus 1.5 8B | 0 / FFN / 0.284 | 0.00435 | |||
| Qwen 3.5 9B | 2 / FFN / 0.616 | 0.00109 | |||
| Qwen 3.5 27B | 1 / token mixing / 0.667 | 0.000777 | |||
| Gemma 4 31B | 23 / FFN / 0.499 | 0.0145 |
Table 2: Primary MMLU inference and cross-panel localization. Shading marks mean-shift maxima with . Matched uses identical 64-feature estimators and reselects the site for every permutation; lexical-64 refits the lexical adjustment within each fold. Sites are zero-indexed layer / stream. The smallest attainable p-value is for MMLU and for MMLU-Pro. Heterogeneous-panel sites are descriptive, and all sites come from the mean-shift selector.
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 3: OLMo 3 normalized-response pilot (four tasks per group). The depth-averaged mean-shift ratio falls across response bins, although the reselected local maximum (not plotted) rises ( , the smallest attainable value). Effective-rank fractions rise. Bins are fractions of the response length rather than semantic reasoning steps.
| Model | Revision | Layers | Storage | Batch |
|---|---|---|---|---|
| OLMo 3 7B Instruct | 6e5971d | 32 | float32 | 32 |
| Apertus 1.5 8B | a411d83 | 32 | float32 | 24 |
| Qwen 3.5 9B | c202236 | 32 | float32 | 20 |
| Qwen 3.5 27B | fc05dae | 64 | float32 | 8 |
| Gemma 4 31B IT | 842da37 | 60 | float32 | 2 |
Table 3: Evaluated conditions on one 94.5-GiB GH200. All use deterministic prefill extraction, final-valid-token pooling, and 100 examples per task. Batch is the extraction batch size, not the number of analyzed examples.
| Model | Mean-shift maximum | Raw probe | Surface+modality | Lexical-64 | Combined |
|---|---|---|---|---|---|
| Qwen 3.5 9B | , L0-T ( ) | ( ) | ( ) | ( ) | ( ) |
| Qwen 3.5 27B | , L5-T ( ) | ( ) | ( ) | ( ) | ( ) |
| Gemma 4 31B | , L36-R ( ) | ( ) | ( ) | ( ) | ( ) |
Table 4: Supplementary heterogeneous multimodal analysis. Mean-shift cells report the selection-aware maximum, zero-indexed layer and stream (T: token-mixing update; R: residual state), and exact p-value. Probe cells report held-out-task macro-F1 and exact p-value in parentheses. No result is significant under its exact task-label null at .
| Measurement at selected site | Qwen 9B | Qwen 27B | Gemma 31B | Reference | Role |
| RBF probe macro-F1 | Nonlinear prediction; descriptive here | ||||
| Cosine silhouette | Higher means greater distance-based separation | ||||
| Covariance alignment | Lower means less similar unpaired geometry | ||||
| Within-group effective-rank fraction | Lower means a more concentrated spectrum | ||||
| Within-group stable rank | — | Scale-dependent spectral concentration diagnostic | |||
| Paired linear CKA | — | — | — | Undefined: zero declared cross-group pairs |
Table 5: Complementary measurements at the observed mean-shift-selected site. These quantities were not given task-randomization tests in this supplementary analysis and are descriptive. Paired CKA is undefined, rather than zero, because the two benchmark groups have no declared example pairing; covariance alignment supplies an order-invariant comparison.
| Family | Reference value | Scientific role |
|---|---|---|
| Mean shift | : equal projected means; : squared mean shift equals summed within-category variance | Within-model mean separation; numerator and denominator remain visible. |
| Probe | : balanced chance | Prediction for tasks excluded from probe fitting; task-label permutations define the null. |
| Geometry | Silhouette : no distance advantage; alignment : identical normalized geometry | Cohesion and similarity; paired CKA is reported only for actual pairs. |
| Rank | : maximally even spectrum | Contraction diagnostic, not separation evidence by itself. |
Table 6: Four complementary measurement families. Reference values aid interpretation; they are not universal quality thresholds. Colors are redundant with the family labels.
Figure 4: Synthetic metric guide computed directly in two dimensions; no embedding method is used. Blue circles denote , orange squares denote , crosses mark class means, and gray hollow points in (d) show the state before contraction. Displaced means raise the mean-shift ratio and both probes (b), whereas equal-mean concentric groups are detected only by the nonlinear probe (c). Holding the means fixed while reducing within-group variance sharply raises the mean-shift ratio (d), illustrating why we report its numerator, denominator, and effective rank separately. Probe values use five complete-group folds with fold-local preprocessing. The examples illustrate what each statistic responds to and define no empirical cutoffs.
| Model | Matched-64 /exact | Null mean SD; q95 | PCA-95 /exact |
|---|---|---|---|
| OLMo 3 7B | |||
| Apertus 1.5 8B | |||
| Qwen 3.5 9B | |||
| Qwen 3.5 27B | |||
| Gemma 4 31B |
Table 7: Matched activation-minus-prompt inference and width sensitivity. The null column summarizes 6,434 non-observed label-complement pairs. PCA-95 replaces activation PCA-64 while retaining 64 prompt components; it tests compression rather than capacity matching. Every observed lies within its null distribution.
| Model | Mean-shift site | Exact range | |||
|---|---|---|---|---|---|
| OLMo 3 7B | 3 / feed-forward | 21 / residual | 20 / residual | 6 / residual | – |
| Apertus 1.5 8B | 0 / feed-forward | 18 / residual | 4 / residual | 4 / residual | – |
| Qwen 3.5 9B | 2 / feed-forward | 30 / residual | 30 / residual | 30 / residual | – |
| Qwen 3.5 27B | 1 / token mixing | 25 / token mixing | 25 / token mixing | 25 / token mixing | – |
| Gemma 4 31B | 23 / feed-forward | 31 / token mixing | 31 / token mixing | 22 / token mixing | – |
Table 8: Selector sensitivity on the primary panel. Each entry gives the zero-indexed layer and stream selected by the observed partition; the exact test reselects over all layers and streams for each of 12,870 task-label permutations. These sites are neither causal claims nor a cross-panel transfer test.
| Standardized mean difference | Directions | Exact | Macro-F1 mean [range] |
|---|---|---|---|
| 1 | 0 | ||
| 10 | 0 | ||
| 10 | 0 | ||
| 10 | 0 | ||
| 10 | 5 |
Table 9: Fixed-site detection calibration after fold-local lexical adjustment. The injected mean difference is measured in standard deviations of the unadjusted activation projected onto each seeded unit direction. Every condition uses the complete 12,870-permutation null; the zero-shift row is evaluated once because direction is immaterial. Direction seeds are calibration conditions, not model, data, or training replicates.
More in this topic
LLM Interpretability
Large language model (LLM) interpretability studies how internal representations and computations produce outputs, and how they can be explained or controlled.
See all 781 papers