The prevailing approach to improving large language model (LLM) reasoning has centered on expanding context windows, implicitly assuming that more tokens yield better performance. However, empirical evidence - including the "lost in the middle" effect and long-distance relational degradation - demonstrates that contextual space exhibits structural gradients, salience asymmetries, and entropy accumulation under transformer architectures. We introduce Context Cartography, a formal framework for the deliberate governance of contextual space. We define a tripartite zonal model partitioning the informational universe into black fog (unobserved), gray fog (stored memory), and the visible field (active reasoning surface), and formalize seven cartographic operators - reconnaissance, selection, simplification, aggregation, projection, displacement, and layering - as transformations governing information transitions between and within zones. The operators are derived from a systematic coverage analysis of all non-trivial zone transformations and are organized by transformation type (what the operator does) and zone scope (where it applies). We ground the framework in the salience geometry of transformer attention, characterizing cartographic operators as necessary compensations for linear prefix memory, append-only state, and entropy accumulation under expanding context. An analysis of four contemporary systems (Claude Code, Letta, MemOS, and OpenViking) provides interpretive evidence that these operators are converging independently across the industry. We derive testable predictions from the framework - including operator-specific ablation hypotheses - and propose a diagnostic benchmark for empirical validation.
Figures & tables
Figure 1: The tripartite zonal architecture. Solid arrows show the four named transitions; the dashed arc shows the degenerate B→V path that Context Cartography recommends mediating through G or operator guards ( ϕ , π+ ). Zone-level failure modes appear below each zone.
Zone
AI Counterpart
Cartographic Function
Failure Mode
Black Fog B
Tool use, search, code execution ( Yao et al., 2023 ; Wang et al., 2023a )
Reconnaissance, surveying
Hallucination ( Huang et al., 2023 )
Gray Fog G
Structured memory, vector DBs ( Jing et al., 2024 ) , persisted state
Archival mapping, state tracking
Drift, staleness, bloat ( Xiong et al., 2025 )
Visible Field V
Active context window, prompt assembly ( Khattab et al., 2024 )
Tactical projection surface
Overload, attention dilution ( Liu et al., 2024 )
Table 1: The tripartite zonal architecture of contextual space. Zone-level failure modes are listed here; transition-level failures (contamination, destructive compaction, cross-agent incoherence, compound iterative failure) are characterized in the text.
Cartographic Op.
Classical Function
Context Op.
Corr.
Notes
Selection / elimination
Filter features by relevance
σ
Iso.
Both filter known inventory by task criteria
Simplification
Reduce geometric complexity
ϕ
Ana.
Cartographic: vertex count; Context: token count
Aggregation
Merge similar features
α
Iso.
Both fuse proximate items into composites
Typification
Replace dense set with representatives
σ (mode)
Sub.
Pattern-preserving sampling; a selection strategy
Displacement
Resolve visual conflicts
δ
Ana.
Cartographic: 2D overlap; Context: 1D salience
Collapse
Reduce dimensionality
π (param.)
Sub.
Dimensionality reduction within projection schema
Table 2: Mapping from classical cartographic generalization operators to context cartography operators. “Isomorphism” indicates shared formal structure; “Analogy” indicates shared purpose with different mechanism; “Subsumed” indicates the cartographic operator maps to a mode or parameter of a context operator rather than a standalone counterpart.
Figure 2: Cartographic pipelines. The inbound pipeline λ∘δ∘ϕ∘π+∘σ transforms content from G to V ; the outbound pipeline π−∘σ archives from V to G ; the dashed loop represents asynchronous maintenance within G .
Operator
Formal Role
Zone Scope
Cart. Corr.
Reconnaissance ρ
Determine what to explore
B→G
—
Selection σ
Filter known inventory by relevance
G↔V
Isomorphism
Simplification ϕ
Reduce tokens, preserve structure
V or G
Analogy
Aggregation α
Fuse repeated signals
V or G
Isomorphism
Projection π
Transform representation across boundary
G↔V
Isomorphism
Displacement δ
Reposition for salience
V only
Analogy
Table 3: Cartographic operators for contextual space transformation.
Concept
Zone / Transition
Operators
Tool / function call
sense : B→G
ρ (decide to call); ϕ∘π+ (mediate result before entering V )
RAG retrieval
recall : G→V
σ (query indexed corpus); π+ (format chunks for V )
MCP server ( Hou et al., 2025 )
Standardized sense interface
ρ (select server/tool); π (common projection schema across providers)
System prompt ( Mu et al., 2025 )
Pinned region of V
λ (typed namespace); δ (constraint pinning at primacy position)
Conversation history ( Wang et al., 2023b )
V (live); G (after compaction)
π− (compaction cycle); σrecall (re-project from archive)
Memory system ( Sumers et al., 2023 ; Pink et al., 2025 )
Table 4: Common agent engineering concepts mapped to the cartographic framework.
Operator
Claude Code
Letta
MemOS
OpenViking
Mean
Reconnaissance ρ
5
1
1
1
2.00
Selection σ
2
2
5
5
3.50
Simplification ϕ
4
4
2
1
2.75
Aggregation α
1
2
4
1
2.00
Projection π
2
5
5
5
4.25
Displacement δ
2
1
1
1
1.25
Table 5: Operator implementation depth across four systems. Each cell scores five binary criteria (Present, Explicit, Configurable, Automated, Documented; max = 5). Row means indicate per-operator adoption; column means indicate per-system coverage.
Recent benchmarks reveal that despite strong reasoning capabilities, large language models (LLMs) still struggle to faithfully apply complex contextual knowledge. These failures are often not wholesale reasoning collapses: in context-rich tasks, models may follow the central reasoning path while missing peripheral, persistent, or format-sensitive requirements.
We introduce methods to quantify how Large Language Models (LLMs) encode and store contextual information, revealing that tokens often seen as minor (e.g., determiners, punctuation) carry surprisingly high context. Notably, removing these tokens -- especially stopwords, articles, and commas -- consistently degrades performance on MMLU and BABILong-4k, even if removing only irrelevant tokens. Our analysis also shows a strong correlation between contextualization and linearity, where linearity measures how closely the transformation from one layer's embeddings to the next can be approximated by a single linear mapping. These findings underscore the hidden importance of filler tokens in maintaining context. For further exploration, we present LLM-Microscope, an open-source toolkit that assesses token-level nonlinearity, evaluates contextual memory, visualizes intermediate layer contributions (via an adapted Logit Lens), and measures the intrinsic dimensionality of representations. This toolkit illuminates how seemingly trivial tokens can be critical for long-range understanding.
Anton Razzhigaev, Matvey Mikhalchuk, Temurbek Rahmatullaev +4
AIRI · Skoltech · Lomonosov Moscow State University +1
Spatial intelligence is a key frontier for multimodal large language models (MLLMs), enabling them to reason about the physical world from visual experience. Inspired by human spatial cognition, recent approaches construct grid-based cognitive maps from multi-frame visual inputs to maintain coherent spatial representations over time. However, limited context lengths still challenge spatial understanding, while existing methods, such as long-context modeling and external memory, often require architectural changes, memory modules, or finetuning, limiting their applicability to off-the-shelf pretrained MLLMs. This motivates a lightweight, model-agnostic method for preserving spatial information beyond the native context window. To this end, we propose a plug-and-play multi-agent framework that collaboratively constructs cognitive maps as structured spatial memory, enhancing the spatial understanding of arbitrary pretrained MLLMs without architectural modification or additional training. Our framework features local-global agent coordination, cognitive map construction with atomic commits, and cross-agent verification. Extensive experiments demonstrate that our method achieves superior performance on spatial understanding tasks while remaining fully training-free. Code will be released.
Yiming Zhang, Ruoxuan Cao, Zhihang Zhong
2Cornell University · 1Shanghai Jiao Tong University