cs.LGOct 31, 2026

Same Function, Different Mechanism: Implementation-Local Geometry Beyond Accuracy

Authors: Miroslav Lžičař

Organizations: Mereology Labs

Abstract

Two models can make exactly the same predictions yet respond differently when their implementations are perturbed. A lookup table can change one case independently; a shared rule couples changes across cases. We compare these mechanisms through an implementation chart: a specified local family of logit perturbations around a fixed predictive distribution. Our main result counts the independent prediction changes allowed by exact finite-group sharing: the active dimension is the number of input-output orbits minus the number of input orbits. For softened modular addition, lookup and shared cyclic charts therefore have ranks p2(p−1)p^2(p-1) and p−1p-1 despite identical predictions. The classical Fisher pullback connects these dimensions to local KL curvature; mechanism-preserving reparameterizations leave the rank unchanged. We also bound the spectral tail under approximate sharing and separate quadratic rank from higher-order learning coefficients. Trained modular-addition models verify that chart rank is measurable, while controlled compiled transformers distinguish compiler-supported intermediate charts from matched alternatives. Negative controls show where this discrimination fails. The framework evaluates specified mechanism hypotheses; it does not infer a unique mechanism from predictions or predict training dynamics.

Figures & tables

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. LoMETab: Beyond Rank-1 Ensembles for Tabular Deep Learning

    May 14, 2026Changryeol Choi, Hyewon Park, Yujin Kwon +1Ensemble LearningLow-Rank Adaptation

  2. Multiplication Beyond Groups: Stratified Fourier Mechanisms in Transformer Circuits

    Jul 8, 2026Zitong Andrew Chen, Junaid Hasan, Akhil Srinivasan +2Transformer InterpretabilityTransformer Attention