Realize What Matters: Principled Context Representation for Large-Scale Reasoning

Michael Theologitis; Dean Light; Shuyue Stella Li; Benjamin Newman; Yulia Tsvetkov; Dan Suciu

cs.CLSep 23, 2026

Realize What Matters: Principled Context Representation for Large-Scale Reasoning

Authors: Michael Theologitis, Dean Light, Shuyue Stella Li, Benjamin Newman, Yulia Tsvetkov, Dan Suciu

Organizations: University of Washington

Abstract

Solving complex tasks in domains such as science, medicine, law, and finance often requires assembling interdependent information scattered across vast, heterogeneous sources far beyond model context limits. Existing approaches tackle this challenge by organizing information into more manageable representations over which models can reason, such as graphs, textual memories, and retrieval collections. These representations dictate what downstream reasoning is possible and, ultimately, whether it succeeds; yet their design and construction remain largely ad hoc. In this work, drawing on the cognitive theory of relevance realization, we propose concrete principles for designing AI systems that construct effective representations of very large contexts. We analyze existing approaches and show how their successes and failures map onto their alignment with these principles, and introduce R3Con, a harness designed to operationalize the principles more systematically. We evaluate R3Con against nine state-of-the-art baselines on two recent benchmarks of reasoning over large document corpora. On these benchmarks, R3Con substantially outperforms the strongest baseline, by $20$ and $8.4$ percentage points. It also enables smaller models to outperform much larger ones: R3Con with 4B and 9B models outperforms all evaluated 35B baselines, while R3Con with a 35B-A3B model outperforms Claude Code with Claude-Sonnet-5 at $3.7\times$ lower cost. Our results show that context representations following our principled approach can reduce reliance on model scale, pointing toward a future of AI systems with frontier-level performance powered by smaller models. Our code is available at https://github.com/michaeltheologitis/r3con

Figures & tables

Figure 1: R3Con with a 3B -active model outperforms Claude Code equipped with Anthropic’s frontier Sonnet models on CorpusQA ( Lu et al., 2026 ) , while costing roughly

3.7\times

3.9\times

less.

Figure 2: Running example based on Bram Stoker’s Dracula ( 1897 ) . Full details in § B .

Figure 3: An investigation of the events aboard the ship Demeter. Figuring out that the entire crew of nine died, and that Dracula was behind it, requires connecting evidence across partial and incomplete accounts.

Figure 4: R3Con answers a task over a large context by first constructing an effective representation using our design principles (§ 3.2 ). It extracts the relevant context $\mathcal{R}$ (Phase 1) and then uses $\mathcal{R}$ to further structure the context (Phase 2). Together, the extracted relevant context and structured context form the representation, which is then used by a coding agent to produce the answer (Phase 3).

Figure 5: Evolution of the relevance snippets for the ship’s log ( $D_{37}$ ) and The Dailygraph newspaper ( $D_{12}$ ) across two rounds of extracting relevance. For example, the top-left and top-right cells show $R_{37}^{(1)}$ and $R_{37}^{(2)}$ , respectively. ‡ Document ordering is arbitrary. The full snippets are shown in Fig. 12 .

Panel of Table 1 — Table 1: Accuracy (% $\uparrow$ ) across real-world domains requiring reasoning over large document corpora. R3Con achieves the highest accuracy across all domains, outperforming the strongest baseline by +19.88 pp on CorpusQA and +8.42 pp on Loong overall (SEs in Tab. 10 ). Accuracy is micro-averaged.

Variant	Loong ( $\uparrow$ )	CorpusQA ( $\uparrow$ )
R3Con (default)	37.43%	70.64%
[0.6pt/2pt] w/o extracting relevance	23.96% (-13.47)	52.58% (-18.1)
w/o structuring	32.97% (-4.46)	63.22% (-7.42)

Table 3: Accuracy in the ablation of the two phases of R3Con (see Fig. 4 ).

Panel of Table 4 — Table 4: Taxonomy of baselines according to the principles (§ 3.2 ) satisfied by their representation-construction phase. See § C.2 for detailed analysis. = limited support; N/A = not applicable.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Document type	# Docs.	Description
Journal / diary	5	Personal journals and diaries (e.g., Jonathan Harker’s journal)
Letter	20	Letters between characters (e.g., Mina to Lucy)
Telegram	8	Telegrams between characters (e.g., Van Helsing to Seward)
Memorandum	4	Written accounts of events (e.g., Lucy’s final memorandum)
Newspapers	4	Newspaper cuttings (e.g., The Pall Mall Gazette)
Note	2	Short written notes (e.g., Van Helsing’s contingency note)

Appendix

Table 5: The reconstructed document collection in Bram Stoker’s Dracula ( 1897 ) .

Model	Parametric knowledge		With corpus in context
Model	Answer	Correct	Answer	Correct
Qwen3.5-35B-A3B ( 2026 )	31	✗	7	✗

Appendix

Table 6: Answers to the Dracula question using parametric knowledge alone and with the full novel provided in context. The ground-truth answer is 13, but also requires the correct people named. Full model responses are shown in Fig. 8 .

Model	Temp.	Top- $p$	Top- $k$	Min- $p$	Pres. pen.	Rep. pen.
Qwen3.5-4B	1.0	0.95	20	0.0	1.5	1.0
Qwen3.5-9B	1.0	0.95	20	0.0	1.5	1.0
Qwen3.5-35B-A3B	1.0	0.95	20	0.0	1.5	1.0

Appendix

Table 7: Sampling parameters used for each model, following the recommendations from the official HuggingFace model cards.

Method	CorpusQA	Loong
Qwen3.5-4B
StructRAG ( 2025b )	29.79%	16.84%
CodeAgent ( 2025 )	10.92%	19.44%
R3Con (ours)	49.24%	30.23%
Qwen3.5-9B
StructRAG ( 2025b )	40.43%	20.30%

Appendix

Table 8: Accuracy (

\uparrow

) across benchmarks and model sizes of the strongest baselines.

Figure 9: Distribution of the representation components (i.e., final relevant context and structuring) for the two benchmarks with the default Qwen3.5-35B-A3B model.

Figure 12: Evolution of the relevance snippets for the log of the Demeter ( $D_{37}$ ) and The Dailygraph newspaper ( $D_{12}$ ) across two rounds of extracting relevance. For example, the top-left and top-right cells show $R_{37}^{(1)}$ and $R_{37}^{(2)}$ , respectively. ‡ Document ordering is arbitrary.

Downstream Reasoning	CorpusQA	Loong
LLM	65.43%	40.17%
Coding Agent (default)	70.64%	37.43%

Appendix

Table 9: Accuracy (

\uparrow

) with different downstream reasoning methods over the representation constructed by R3Con .

Figure 13: Effect of the number of relevance-extraction rounds

N

on accuracy and cost.

Method	CorpusQA ( Lu et al., 2026 )				Loong ( Wang et al., 2024a )
Method	Edu.	Finance	Real-estate	Overall	Paper	Finance	Legal	Overall
RAPTOR ( 2024 )	8.00 $\pm$ 3.13	11.11 $\pm$ 2.47	17.95 $\pm$ 4.35	12.06 $\pm$ 1.84	0.00 $\pm$ 0.00	22.17 $\pm$ 1.71	5.71 $\pm$ 1.24	12.12 $\pm$ 0.92
ReadAgent ( 2024 )	42.68 $\pm$ 5.46	37.42 $\pm$ 3.79	31.25 $\pm$ 5.18	37.23 $\pm$ 2.68	15.15 $\pm$ 2.08	35.33 $\pm$ 2.03	14.93 $\pm$ 1.95	24.49 $\pm$ 1.25
MemAgent ( 2026 )	7.32 $\pm$ 2.88	7.83 $\pm$ 2.09	8.64 $\pm$ 3.12	7.90 $\pm$ 1.49	0.00 $\pm$ 0.00	26.80 $\pm$ 1.81	4.00 $\pm$ 1.05	13.63 $\pm$ 0.96
HippoRAG2 ( 2025 )	7.32 $\pm$ 2.88	7.83 $\pm$ 2.09	20.99 $\pm$ 4.52	10.94 $\pm$ 1.72	0.32 $\pm$ 0.32	17.42 $\pm$ 1.55	6.29 $\pm$ 1.30	10.06 $\pm$ 0.85
LinearRAG ( 2026 )	8.54 $\pm$ 3.09	6.63 $\pm$ 1.93	16.05 $\pm$ 4.08	9.42 $\pm$ 1.61	0.61 $\pm$ 0.43	9.38 $\pm$ 1.19	5.43 $\pm$ 1.21	6.04 $\pm$ 0.67

Appendix

Table 10: Accuracy (%

\uparrow

) across real-world domains requiring reasoning over large document corpora. R3Con achieves the highest accuracy across all domains, outperforming the strongest baseline by +19.88 pp on CorpusQA and +8.42 pp on Loong overall. Overall accuracy is micro-averaged. We report standard errors (SE) as

\pm

values.

Method	Spot.	Comp.	Chain	Clust.	Overall
RAPTOR ( 2024 )	51.31 $\pm$ 3.62	16.25 $\pm$ 2.38	2.04 $\pm$ 0.82	1.54 $\pm$ 0.54	12.12 $\pm$ 0.92
ReadAgent ( 2024 )	60.11 $\pm$ 3.67	33.77 $\pm$ 3.13	18.49 $\pm$ 2.27	10.70 $\pm$ 1.40	24.49 $\pm$ 1.25
MemAgent ( 2026 )	42.13 $\pm$ 3.52	35.00 $\pm$ 3.08	0.64 $\pm$ 0.45	0.95 $\pm$ 0.42	13.63 $\pm$ 0.96
HippoRAG2 ( 2025 )	46.70 $\pm$ 3.55	13.33 $\pm$ 2.19	0.33 $\pm$ 0.33	0.38 $\pm$ 0.27	10.06 $\pm$ 0.85
LinearRAG ( 2026 )	32.49 $\pm$ 3.34	3.75 $\pm$ 1.23	0.64 $\pm$ 0.45	0.38 $\pm$ 0.27	6.04 $\pm$ 0.67
StructRAG ( 2025b )	44.16 $\pm$ 3.54	30.83 $\pm$ 2.98	24.76 $\pm$ 2.45	12.19 $\pm$ 1.43	23.72 $\pm$ 1.19

Appendix

Table 11: Accuracy (%

\uparrow

) across Loong’s reasoning facets. We show standard errors as

\pm

values.

Explore similar work

Where Larger Models Excel: The Primacy of Constraint-Guided Reasoning
May 9, 2026Guan-Yi Lin, Hen-Hsen HuangReasoning Benchmark Model Size
National Chengchi University Taipei, Taiwan · Academia Sinica Taipei, Taiwan
Structured Thoughts For Improved Reasoning And Context Pruning
Jul 11, 2026Zain Sarwar, Supriyo Chakraborty, Berkcan Kapusuzoglu +5Reasoning Traces Tree-Of-Thoughts
University of Chicago · Capital One
RECON: Benchmarking Agent Memory for Compositional Reasoning over Long Contexts
Jul 18, 2026Mihir Shriniwas Arya
Department of Computer Science and Engineering RV College of Engineering

May 9, 2026, cs.CLJ/K move · Enter open · S save

Where Larger Models Excel: The Primacy of Constraint-Guided Reasoning

Guan-Yi Lin, Hen-Hsen Huang

National Chengchi University Taipei, Taiwan · Academia Sinica Taipei, Taiwan

Reasoning Benchmark Model Size Strong Large Language Model Reasoning Traces Constraint Satisfaction Priming

Open paper AskPDF arXiv

Larger language models consistently outperform smaller ones on reasoning benchmarks, yet the reasoning differences underlying this gap remain underexplored. Across benchmarks in mathematics, physics, chemistry, and programming, we observe stable performance gaps: averaged over datasets, Qwen3-32B outperforms Qwen3-8B by 6.43%, while GPT-OSS-120B exceeds GPT-OSS-20B by 7.38%. To study the reasoning differences behind these gains, we develop AdvCluster, an automated framework that identifies questions where the larger model shows a stable advantage, extracts fine-grained advantage descriptions from paired reasoning traces produced by larger and smaller models, and organizes them through semantic clustering with quantitative evaluation and selection guided by a reviewer model. Our analysis yields a systematic taxonomy of larger model reasoning advantages, spanning both common advantages that recur across domains and specialized advantages associated with particular domains. Across these patterns, a recurring theme is Constraint-Guided Reasoning: larger models are better at identifying explicit and implicit constraints, organizing them into structured reasoning, and using them to rule out infeasible paths and verify intermediate steps.

Abstract

Explore similar work

Where Larger Models Excel: The Primacy of Constraint-Guided Reasoning

Structured Thoughts For Improved Reasoning And Context Pruning

RECON: Benchmarking Agent Memory for Compositional Reasoning over Long Contexts