Machine learning learns functions: prompt to response, image to caption. What these functions are mathematically remains hard to say. We present a method to approximate these kinds of transformations using techniques from dynamical systems that fall under the umbrella of Koopmanism. We introduce the use of composition operators, which generalize the Koopman operator and, crucially, can map between distinct spaces, motivating the perspective that LLM transformations are rectangular infinite-dimensional operators. This formalism reveals useful structure: under natural assumptions on the prompt and response distributions, the LLM operator is an isometry, and misalignment between learned representations manifests as spectral pollution of its finite sections. We then outline a method of constructing finite-dimensional approximations of an LLM operator, and demonstrate how the singular value spectrum can be used to compare tasks and models.
Figures & tables
Figure 1: (A) Singular value spectra of the finite sections for each task and responder; (B) singular value spectra after subtracting the mean spectrum across all sections.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Task
s1
sˉ
s2
Summarization
1.046±0.014
0.783±0.025
0.673±0.034
Brainstorming
1.177±0.101
0.710±0.038
0.584±0.051
Creative writing
1.169±0.076
0.686±0.033
0.561±0.041
Null (random pairings)
0.819±0.059
0.317±0.005
0.148±0.008
Appendix
Table 1: Summary statistics of the operator singular values per task category, mean ± sd across the five responders (human, Qwen3-0.6B, SmolLM2-360M, Qwen2.5-0.5B, TinyLlama-1.1B). Each operator is the rank-truncated section K for (n=200,r=25) ; every statistic is averaged over 20 stimulus subsamples. s1 : largest singular value; sˉ : mean singular value; s2 : mean squared singular value.
Task
s1
sˉ
s2
Summarization
1.039±0.009
0.782±0.024
0.663±0.035
Brainstorming
1.163±0.101
0.707±0.049
0.581±0.064
Creative writing
1.141±0.073
0.706±0.038
0.577±0.050
Null (random pairings)
0.785±0.069
0.317±0.009
0.147±0.011
Appendix
Table 2: Summary statistics on the MiniLM dictionary (all-MiniLM-L6-v2, 384-d), unwhitened rank-truncated sections. Identical setup to Table 1 .
Figure 2: MiniLM results, analogous to Figure 1 .
Task
s1
sˉ
s2
Summarization
0.979±0.005
0.787±0.026
0.677±0.034
Brainstorming
0.964±0.010
0.706±0.038
0.562±0.050
Creative writing
0.959±0.012
0.680±0.035
0.531±0.045
Null (random pairings)
0.635±0.023
0.303±0.009
0.126±0.007
Appendix
Table 3: Summary statistics on whitened rank-truncated sections (canonical correlations) of BGE embedding model. Analagous to previous experiments.
Figure 3: BGE Whitened results, analogous to Figure 1 .
Figure 4: (A) Empirical null spectra vs Theoretical distribution p in ( 17 ) for β=0.125 , (B) Q-Q comparison of the empirical and theoretical singular value distributions.