Same Function, Different Mechanism: Implementation-Local Geometry Beyond Accuracy
Organizations: Mereology Labs
Abstract
Two models can make exactly the same predictions yet respond differently when their implementations are perturbed. A lookup table can change one case independently; a shared rule couples changes across cases. We compare these mechanisms through an implementation chart: a specified local family of logit perturbations around a fixed predictive distribution. Our main result counts the independent prediction changes allowed by exact finite-group sharing: the active dimension is the number of input-output orbits minus the number of input orbits. For softened modular addition, lookup and shared cyclic charts therefore have ranks and despite identical predictions. The classical Fisher pullback connects these dimensions to local KL curvature; mechanism-preserving reparameterizations leave the rank unchanged. We also bound the spectral tail under approximate sharing and separate quadratic rank from higher-order learning coefficients. Trained modular-addition models verify that chart rank is measurable, while controlled compiled transformers distinguish compiler-supported intermediate charts from matched alternatives. Negative controls show where this discrimination fails. The framework evaluates specified mechanism hypotheses; it does not infer a unique mechanism from predictions or predict training dynamics.
Figures & tables
| Seeds | Unrestricted MLP | Shared projection | Shared architecture | |
|---|---|---|---|---|
| 5 | 3 | 100 | 4 | 4 |
| 7 | 3 | 294 | 6 | 6 |
| 11 | 2 | 1210 | 10 | 10 |
| Theory |
| Intermediate | Compiler | Named best | Random best | Random median |
|---|---|---|---|---|
| Direct: previous token | 0.691 | 0.057 | 0.601 | |
| Decomposed: previous token | 0.691 | 0.030 | 0.606 | |
| Decomposed: current-is- b | 0.674 | 0.058 | 0.674 | |
| Decomposed: previous-is- a | 0.691 | 0.009 | 0.586 |
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
| influence ranks | |||||
|---|---|---|---|---|---|
| 3 | 18 | 2 | 9 | 1 | |
| 5 | 100 | 4 | 50 | 2 | |
| 7 | 294 | 6 | 147 | 3 | |
| 11 | 1210 | 10 | 605 | 5 |
Transformer Interpretability
Transformer interpretability studies how attention, feed-forward layers, and hidden representations support model predictions and behavior.
See all 356 papers