A Bag of Concepts? Testing Role-Filler Binding in the Verbalizable Workspace
Organizations: University of Texas at Dallas · MiraCosta College · University of Waterloo · University of California, Irvine ·
Abstract
Runtime alignment monitors built on a verbalizable workspace have been proposed as a way to surface a model's silent cognition before it produces output. Their value rests on the workspace carrying enough structure to distinguish safety-relevant propositions, most of which are relational: "the model plans to deceive the user" and "the user plans to deceive the model" share a concept set and differ only in role assignment. We ask whether the workspace carries the role-filler bindings (which entity fills which role) such a monitor would need, on three frontier language models (Qwen3.6-27B, Gemma-3-12B, Gemma-3-27B-IT), under two lens families (J-lens and R-lens), and with two methods for fitting the role direction (difference-of-means and LRE-gradient). We find a clean dissociation: the workspace carries directions that steer what the model outputs about roles but not the representations the model uses to compute them. Linear probes recover role at near-ceiling from the residual stream but score below a rank-matched random control when reading the workspace, and steering the model along the fitted role direction changes which entity it binds to no more than strength-matched control edits do. Ablating the workspace (zeroing the residual stream's projection onto its ten most active lens directions) damages binding on one of the three models (Gemma-3-27B-IT, holding across three lens variants) and shows no binding-specific damage on the other two, with a rank-matched random subspace as the control; this confirms our design detects binding when present, though scale and instruction tuning are confounded there. A workspace-lens monitor built on these directions would therefore miss a role assignment the model itself represents.
Figures & tables
| Decoded from | Qwen3.6-27B | Gemma-3-12B | Gemma-3-27B-IT |
| Full residual stream | 0.989 | 0.975 | 0.978 |
| Orthogonal remainder | 0.984 | 0.949 | 0.974 |
| Random subspace (matched rank) | 0.643 | 0.589 | 0.590 |
| J-space | 0.587 | 0.573 | 0.574 |
| Displacement | Pooled mean | Non-participant control | Ratio | |
| 64 | 0.328 | 0.83 | ||
| 128 | 0.645 | 1.01 | ||
| 256 | 1.255 | 0.84 |
| Model | Site | Role acc. no edit | Role acc. abl. J | Deficit (J) | Same, random | Edit mag. ratio |
| Qwen3.6-27B | entity | 0.790 | 0.768 | 5.39 | ||
| Qwen3.6-27B † | final | 0.790 | 0.792 | 4.97 | ||
| Gemma-3-12B | final | 0.883 | 0.883 | 0.79 | ||
| Gemma-3-12B † | entity | 0.883 | 0.736 | 0.94 | ||
| Gemma-3-27B-IT | final | 0.990 | 0.991 | 0.88 | ||
| Gemma-3-27B-IT | entity | 0.990 | 0.740 | 0.94 |
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
| Decoded from | Qwen3.6-27B | Gemma-3-12B | Gemma-3-27B-IT |
| Full residual stream | 0.594 | 0.644 | 0.601 |
| Orthogonal remainder | 0.626 | 0.603 | 0.605 |
| Random subspace (matched rank) | 0.713 | 0.613 | 0.575 |
| J-space | 0.524 | 0.514 | 0.510 |
LLM Interpretability
Large language model (LLM) interpretability studies how internal representations and computations produce outputs, and how they can be explained or controlled.
See all 829 papers