Machine-Interpretable Information: Compiling Documents into Searchable and Readable Protocol States

Yifan Wang; Dejing Dou

cs.CLSep 20, 2026

Machine-Interpretable Information: Compiling Documents into Searchable and Readable Protocol States

Authors: Yifan Wang, Dejing Dou

Organizations: NexusLumenLabs · Fudan University

Abstract

Long-context language models interface with external knowledge through raw natural language. In retrieval-augmented systems, this creates a persistent index-payload schism: dense vectors enable searchable routing, but models must re-ingest lengthy text payloads for reasoning at O(N^2) attention cost. Existing compression methods further produce private states tied to specific architectures. We introduce Machine-Interpretable Information (MII), the first agent-to-agent (A2A) document-to-state protocol. A dual-timescale state-space Writer compiles documents into a canonical, fixed-bandwidth state (56 tokens), and a lightweight Translator maps it into any frozen Reader's embedding space, reducing query-time cost to O(K). The resulting .mii artifact unifies Retrieval (searchable geometry), Reasoning (global memory), and Reconstruction (grounded details) in a single transferable medium. We demonstrate strong cross-model interoperability across heterogeneous LLMs (e.g., Llama, Qwen, Mistral) -- despite the Writer using a legacy GPT-2 vocabulary, forcing genuine semantic translation rather than token-level memorization. Mechanistic probes reveal modular latent structure: entity representations can be causally traced and zero-shot transplanted between unrelated document states while remaining decodable. To address lexical reconstruction under fixed bandwidth, we propose Residual-MII, a cache hierarchy combining compiled global memory with sparse local evidence. On HotpotQA (7,405 queries), Residual-MII exceeds full-context Exact Match at approximately 7% of the attention FLOPs, suggesting a paradigm shift toward compiled, transferable neural document formats.

Figures & tables

Figure 1: The MII Protocol resolves the Index-Payload Schism and establishes a new Pareto frontier, thus redefining the architecture of long-document comprehension. Left Panel: Traditional RAG (A) suffers from a schism between a searchable index and a readable text payload. The MII A2A Protocol (B) compiles documents into a single, interoperable artifact that is natively Searchable, Readable, and Persistent, enabling a "Write Once, Read Many" architecture. Top-Right: On HotpotQA, Residual-MII achieves F1 comparable to FullContext at orders-of-magnitude lower query-time FLOPs. Bottom-Right: On a 128K-token document, MII rapidly amortizes its one-time compilation cost to near-zero marginal latency.MII can be used as a standalone index or synergized with sparse text (Residual-MII) to achieve state-of-the-art efficiency.

Method	EM	F1	Avg Input Tokens	Avg Attn FLOPs	FLOPs Ratio (vs. Blind)	FLOPs Ratio (vs. FC)
Baselines
Blind (No Context)	0.1905	0.2745	62 ∗	3.91G	1.00 $\times$	0.14%
FullContext (12K $\rightarrow$ 7K)	0.4639	0.6114	$\sim$ 7168	2.73T	698.2 $\times$	100%
Oracle (Gold Evidence)	0.5901	0.7441	$\sim$ 450	123.52G	31.6 $\times$	4.5%
Retrieval-Augmented Methods
BM25 (Sparse)	0.4177	0.5478	$\sim$ 480	154.07G	39.4 $\times$	5.6%

Table 1: Main results on the full HotpotQA development set (7,405 queries). We evaluate MII against raw text baselines and GPT-5.5 generated full context natural language summaries. Residual-MII achieves a higher Exact Match (EM) than the FullContext baseline while consuming only

\approx

7% of the FLOPs. Notably, the 56-token continuous protocol state encodes a much denser cognitive manifold than a 56-token text summary, performing competitively with a 512-token text summary.

Dataset	Res-MII F1	FC Toks	MII/FC FLOPs
Qasper	0.36	4.9K	0.81%
2WikiMQA	0.43	7.1K	0.51%
LB-HotpotQA	0.53	12.8K	0.16%
MuSiQue	0.31	15.6K	0.12%
NarrativeQA	0.17	28.4K	0.03%

Table 2: True long-context regime. Residual-MII uses an increasingly minuscule fraction of full-context FLOPs as document length scales.

Figure 3: Validation of Semantic Isomorphism against an Unseen External Space. When clustered via an unseen E5 space, the MII manifold exhibits clearer semantic boundaries and smoother interpolation than the BGE teacher space, confirming it learns a generalized cognitive topology.

Pure MII Cond.	EM	F1	EM $\Delta$	Residual-MII Cond.	EM	F1	EM $\Delta$
Pure MII (Full)	0.2120	0.2836	-	Res-MII (Full)	0.4220	0.5390	-
Pure MII (No Global)	0.0020	0.0148	-99.1%	Res-MII (No Global)	0.0040	0.0210	-99.0%
Pure MII (No Event)	0.2340	0.3073	+10.4%	Res-MII (No Event)	0.4060	0.5213	-3.8%
Pure MII (No Cover)	0.2180	0.2958	+2.8%	Res-MII (No Cover)	0.4040	0.5119	-4.3%

Table 3: Synergy Ablation (500 samples). Masking the Global bank destroys comprehension. Event/Cover banks act as latent noise without text, but ground local details when sparse text is present.

Reader Model	Blind	2.5K	5.0K	7.5K	10.0K	Best
Qwen-2.5-7B	12.4	1.0	7.6	3.8	21.4	22.2
Mistral-v0.3-7B	17.2	12.4	20.6	23.2	-	23.2
Meta-Llama-3-8B	10.8	2.0	13.8	13.8	17.6	28.4

Table 4: Cross-Model Readability (EM %). Translators bridge the vocabulary chasm, enabling heterogeneous LLMs to read the same .mii state.

Figure 4: Spectral Specialization. Learned $\Delta$ biases exhibit near-disjoint support. Initial penalties vanish early, yielding spontaneous separation.

Probe Operation	Parasite Entity	Host Document Context	Reader’s Zero-Shot Generation from Composite MII State
Successful Transplantation: Isolating Transplanted Knowledge
Transplant tokens [43, 19, 46]	‘The Fly (song)‘	Chandan Prabhakar (Comedian)	"The memory mentions ’The Fly’ song. The memory says ’The Fly’ is a song by
			British rock band The Rolling Stones, released in..."
Transplant tokens [54, 39, 43]	‘New Frontier Hotel‘	Crichton F.C. (Football Club)	"Memory loaded. The memory mentions New Frontier Hotel and Casino. It says
			"The New Frontier Hotel and Casino, a 1960s-era..."
Emergent Stitching: Spontaneous Logical Integration

Table 5: Case studies in zero-shot latent space editing and transplantation. Following causal tracing [ 9 ] , we identify causal tokens for a "parasite" entity via

\Delta

NLL and surgically insert them into an unrelated "host" MII state. Probing this composite state reveals emergent modularity and compositionality.

Paradigm	Representative Works	Searchable	Generative	Cross-Model	Amortized	Gradient-Free	Fixed	Surgical
		Routing?	Payload?	A2A?	$\mathcal{O}(1)$ Read?	Compilation?	Bandwidth?	Editability?
Long-Context LLMs	Llama-3, Gemini-1.5	✗	✓ (Raw)	✓	✗ ( $\mathcal{O}(N^{2})$ )	✓	✗ ( $\mathcal{O}(N)$ )	✗
Hierarchical RAG	RAPTOR [ 39 ]	✓	✓ (Raw)	✓	✗ ( $\mathcal{O}(k\log N)$ )	✓	✗ (Variable)	✗
Extreme Compression	xRAG [ 40 ] (1-Token)	✓	✗	✗	✓	✓	✓ ( $\mathcal{O}(1)$ )	✗
Latent Compressors	Gist [ 5 ] , ICAE [ 6 ]	✗	✓ (Latent)	✗	✓	✓	✗ (Variable)	✗
Test-Time Compilation	LCC [ 42 ]	✗	✓ (Buffer)	✗	✓	✗ (Test-Time Opt.)	✓ ( $\mathcal{O}(1)$ )	✗

Table 6: Qualitative comparison of document representation paradigms. MII uniquely unifies searchable routing and generative reading into a strictly fixed-bandwidth, cross-model artifact. It compiles via a single gradient-free forward pass and exhibits white-box editability, redefining the Pareto frontier for long-document comprehension.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Writer Scaling Dynamics. As the Writer capacity scales from 1x to 4x, the reconstruction loss over the fixed 56-token protocol state decays exponentially faster, confirming that semantic compression follows predictable scaling laws.

LLM Context Compression

See all 42 papers

DocAtlas: Long-Document Understanding as Mutable-State Interaction
Jul 21, 2026Hongchen Wei, Yuanzhe Wang, Bei Liu +8Document Understanding Long-Context QA
Wuhan University · Microsoft
Doc-to-Atom: Learning to Compile and Compose Memory Atoms
Jun 10, 2026Xingjian Diao, Wenbo Li, Yashas Malur Saidutta +3Memory-Augmented Language Models Long-Context Language Model Inference
AI Center-Mountain View, Samsung Electronics · Dartmouth College
One Model Is Enough: Native Retrieval Embeddings from LLM Agent Hidden States
Mar 9, 2026Bo JiangAgentic Retrieval Text Embedding Models
Temple University

Jul 21, 2026J/K move · Enter open · S save

DocAtlas: Long-Document Understanding as Mutable-State Interaction

Hongchen Wei, Yuanzhe Wang, Bei Liu, Yifan Yang+7

Wuhan University · Microsoft

Document Understanding Long-Context QA Long-Context Modeling Agentic RAG Document QA

Open paper AskPDF arXiv

Long-document understanding requires models to find and combine evidence across many pages, layouts, tables, figures, and charts. Existing retrieval-augmented systems usually select evidence from a static index before generation, while recent agentic systems add multi-turn tool use but often rely on frozen proprietary backbones whose behavior is set by prompts. We present DocAtlas, a system that treats long-document understanding as a mutable-state information-seeking process. We instantiate DocAtlas as a mutable document harness: an external environment that determines what document information is searched, read, stored, reviewed, and shown to the model at each step. Given a document and question, the harness exposes search, reading, note-taking, and review tools, maintains a hierarchical tree and note store, and updates both as the agent records evidence. DocAtlas combines self-improving retrieval, selective evidence access, and active working memory under a fixed context budget. The same harness supports inference-time use with large VLMs and end-to-end reinforcement learning for compact VLM agents. With GPT-5.4, DocAtlas reaches 71.4% on MMLongBench-Doc, exceeding the human-expert reference of 65.8%. A Qwen3.5-4B VLM trained with end-to-end RL in the DocAtlas environment reaches 63.7%, compared with a 54.4% direct-input baseline, showing that mutable document-harness design can improve compact document agents by a large margin.

Abstract

LLM Context Compression

Explore similar work

DocAtlas: Long-Document Understanding as Mutable-State Interaction

Doc-to-Atom: Learning to Compile and Compose Memory Atoms

One Model Is Enough: Native Retrieval Embeddings from LLM Agent Hidden States