cs.AIOct 27, 2026

Beyond the Readout: A Latent Numerical Frame Shared Across Modality and Operation

Authors: Oriol Juan Sabater, Alex Ferrando de las Morenas, Pau Rodríguez, Xavier Suau

Organizations: Universitat Autònoma de Barcelona · Apple

Abstract

When a multimodal model computes 17+35 from text or 91-39 from a rendered image, it produces the same answer, 52. Does it use one internal concept of number for both, or separate machinery for each path? Similarity metrics such as RSA or CKA cannot settle this question. They give the same score to identical and to rotated directions, and two representations can look alike only because both prepare the same output digits. We therefore use causal interventions. For four conditions (text or image, addition or subtraction), we learn a small subspace of Gemma4-12B-it's residual stream in which swapping coordinates swaps the model's answer. The subspaces answer three questions. First, do the conditions write numbers along the same directions? They do not. Reusing a subspace from text in an image on the same operation recovers only 53.2% of its causal effect. A single rotation and rescaling maps one subspace onto the other and raises recovery to 91.4%, so the conditions share one geometry expressed in different coordinates. Second, does this shared geometry only reflect the digit the model is about to print? To test this, we split each subspace into 9 directions that the output layer reads to choose the next digit and 13 directions that it cannot see. The 13 hidden directions still carry the number, yet changing them barely alters the digit being written. Third, what are these hidden directions for? A model must emit a multi-digit answer one digit at a time, so it can compute the full result early and keep it in reserve. When two numbers share their first digit, swapping the number in the hidden directions leaves the digit being written intact and redirects the next digit in 77.6% of cases. The model thus forms an abstract numerical concept upstream and unfolds it across time: multimodal representations share meaning through relational geometry across distinct coordinate frames.

Figures & tables

Appendix figures & tables27 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Multiplication in Multimodal LLMs: Computation with Text, Image, and Audio Inputs

    Apr 20, 2026Samuel G. Balter, Ethan Jerzak, Connor T. JerzakMultimodal Large Language ModelsMultimodal Reasoning

  2. What MLLMs Learn about When they Learn about Multimodal Reasoning

    Oct 2, 2025Jiwan Chung, Neel Joshi, Pratyusha Sharma +2Multimodal ReasoningMultimodal Model Evaluation

  3. Reading, Not Thinking: Understanding and Bridging the Modality Gap When Text Becomes Pixels in Multimodal LLMs

    Mar 10, 2026Kaiser Sun, Xiaochuang Yuan, Hongjun Liu +4Multimodal Large Language ModelsModality Gap in VLMs