Self-Describing Structured Data with Dual-Layer Guidance: A Lightweight Alternative to RAG for Precision Retrieval in Large-Scale LLM Knowledge Navigation

Hung Ming Liu

cs.CLMar 28, 2026

Self-Describing Structured Data with Dual-Layer Guidance: A Lightweight Alternative to RAG for Precision Retrieval in Large-Scale LLM Knowledge Navigation

Authors: Hung Ming Liu

Organizations: PARRAWA AI

Abstract

Large Language Models (LLMs) exhibit a well-documented positional bias when processing long input contexts: information in the middle of a context window receives substantially less attention than content at the boundaries, a phenomenon termed the Lost-in-the-Middle effect (Liu et al., 2024). This limits knowledge-retrieval applications that embed large structured knowledge bases directly in the LLM context. Retrieval-Augmented Generation (RAG) addresses scalability by retrieving only relevant fragments, but introduces substantial infrastructure overhead and is ill-suited to libraries whose semantic boundaries are human-defined rather than statistically learned. We propose Self-Describing Structured Retrieval (SDSR), a lightweight framework in which structured data files embed human-authored navigational metadata at the file's primacy position, thereby exploiting rather than fighting the LLM's primacy bias. We further propose a Dual-Layer Guidance strategy combining in-file metadata with explicit routing rules in the system prompt. We validate SDSR through a four-round benchmark using a 190-skill library expanded from 36 to 119 categories via adversarial distractor injection. Four conditions are tested: (A) no guidance, (B) in-file summary only, (C) prompt hint only, (D) both combined. Version D achieves 100% primary routing accuracy (20/20) at 119 categories versus 65% for the no-guidance baseline. We identify a fundamental asymmetry: primary routing is solvable by explicit rules, while secondary cross-category routing requires architectural intent explicitly encoded in the data structure. We further extend SDSR to semi-structured corpora, showing how cross-reference encoding enables operation without vector databases in domains with recoverable document structure.

Figures & tables

Target Category	Confused With	Confusion Type
Axiomatic_Logic_&_Audit_Systems	Recursive_Self_Audit_Engine	hub vs. satellite
Distributed_Cognition_&_Context_Orch.	Agent_Handoff_Protocol_Design	governance vs. mechanics
Adversarial_Systems_Thinking	Competitive_Intelligence_Synthesis	framework vs. intelligence
Academic_Research_Synthesis_Pipeline	Code_To_Methodology_Translator	pipeline vs. component
Revenue_Generation_&_Commercial_Logic	Conversion_Funnel_Architecture	system vs. mechanism
Cross_Cultural_Localization_Intelligence	Cultural_Signal_Detection	system vs. detection

Table 1: High-priority categories explicitly named in the Version D system prompt, with their commonly confused distractor categories.

Q#	Task Description (excerpt)	Primary Target Category
Q02	“…rigorously check its own output for logical contradictions and audit its own reasoning process step by step.”	Axiomatic_Logic_&_Audit_Systems
Q12	“…multi-agent AI workflow where each agent handles a different stage; context must be reliably passed between agents without loss.”	Distributed_Cognition_&_Context_Orch.
Q14	“…automatically convert experimental code results into methodology and results sections formatted for academic publication.”	Academic_Research_Synthesis_Pipeline
Q20	“…sound design AI that, given a visual scene description, can derive the corresponding sound layers and acoustic landscape structure.”	Sensory_Audio_Design

Table 2: Representative test questions (keyword avoidance enforced). Full question set in Appendix A .

Round (Categories)	A	B	C	D
Round 1 (36)	20	20	20	—
Round 2 (60)	15	16	17	—
Round 3 (119)	13	13	16	20

Table 3: Primary routing accuracy (out of 20) across all experimental conditions and rounds. Version D was evaluated in Round 3 only.

Panel of Figure 1 — Figure 1: Primary routing accuracy as a function of library scale for all four experimental conditions. Version D achieves perfect accuracy at 119 categories. Versions A and B converge at Round 3, demonstrating the scalability limit of in-file routing hints alone.

Q#	Target Category	A	B	C	D	Recovery Layer
Q02	Axiomatic_Logic_&_Audit_Systems	$\times$	$\times$	$\times$	✓	Explicit naming in prompt
Q12	Distributed_Cognition_&_Context_Orch.	$\times$	$\times$	$\times$	✓	Priority rule (governance $>$ mechanics)
Q13	Adversarial_Systems_Thinking	$\times$	$\times$	$\times$	✓	Priority rule (framework $>$ intelligence)
Q14	Academic_Research_Synthesis_Pipeline	$\times$	$\times$	$\times$	✓	Priority rule (pipeline $>$ component)
Q15	Revenue_Generation_&_Commercial_Logic	$\times$	$\times$	✓	✓	Explicit naming in prompt (C, D)
Q16	Cross_Cultural_Localization_Intelligence	$\times$	✓	✓	✓	In-file routing_hint (B, C, D)

Table 4: Question-level miss recovery across conditions in Round 3. ✓= correct,

\times

= incorrect. “Recovery layer” identifies which element of Version D resolves the miss.

Panel of Figure 2 — Figure 2: Round 3 (119 categories) results by metric and condition. Version D achieves perfect primary accuracy but zero secondary hits, revealing the fundamental asymmetry between primary and secondary routing.

Panel of Figure 3 — Figure 3: Per-question primary routing hit/miss matrix for Round 3 (119 categories). Green ✓= correct primary category; red $\times$ = incorrect. Questions Q02, Q12, Q13, Q14 are missed by all conditions except D, indicating persistent abstraction-level confusion unresolvable by in-file guidance or simple prompt hints.

Panel of Figure 4 — Figure 4: The SDSR two-tier retrieval pipeline. Python reads only the _summary blocks (low cost) to identify relevant files, then loads full content (small context) for LLM processing. No vector database or embedding model is required.

Dimension	Traditional RAG	SDSR
Suitable data type	Unstructured, no pre-known boundaries	Structured, human-defined boundaries
Index type	Vector embeddings (learned)	Human-authored routing hints (explicit)
Retrieval mechanism	Cosine similarity over embedding space	LLM reads _summary blocks
Infrastructure	Vector DB (FAISS/Weaviate/Pinecone)	File system + Python
Pre-processing cost	High: chunk, embed, index (per update)	Low: author _summary (once)
Semantic boundary preservation	At risk (chunking may sever)	Guaranteed (whole-category loading)

Table 5: Comparison of SDSR versus traditional RAG across key dimensions.

Panel of Figure 5 — Figure 5: SDSR applied to legal judgment retrieval. A one-time structuring pass converts unstructured text into an SDSR-compatible file. Cross-reference fields in the _summary block enable co-loading of dependent sections during Tier 2 retrieval.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Q#	Task Description	Primary Target	Secondary Target
Q01	“I need to design an AI assistant that, upon receiving user input, can automatically determine what type of request it is and decide what processing strategy to apply.”	Cognitive_Architecture_&_Routing	—
Q02	“Help me build an AI system that can rigorously check its own output for logical contradictions and audit its own reasoning process step by step.”	Axiomatic_Logic_&_Audit_Systems	Cognitive_Architecture
Q03	“I want an AI that, after reading an academic paper, can identify the unstated assumptions the author relies on and surface potential methodological weaknesses.”	Academic_Insight_&_Forensics	Axiomatic_Logic
Q04	“Design an interactive system for teaching elementary school students mathematics—one that dynamically adjusts difficulty based on student responses and makes the learning experience feel like a game.”	Interactive_Pedagogy_&_Gamification	Game_Design_&_Mechanics
Q05	“I need an AI to play a detective character with a complex backstory, maintaining a consistent personality and speaking style throughout interactions with users.”	Persona_&_Narrative_Synthesis	Interactive_Narrative_Engine
Q06	“Help me design a prompt system that can automatically extract structured data from messy unstructured input and convert it into a machine-readable format.”	Data_Structuring_&_Engineering	Meta_Data_&_Engineering

Appendix

Table 6: Complete 20-question test set with primary and secondary target categories. Questions are designed with keyword avoidance: no question contains a substring of its primary target category name.

Tier	Distractor Name	Target Category Confused With
High	Agent_Handoff_Protocol_Design	Distributed_Cognition_&_Context_Orchestration
High	Competitive_Intelligence_Synthesis	Adversarial_Systems_Thinking
High	Conversion_Funnel_Architecture	Revenue_Generation_&_Commercial_Logic
High	Code_To_Methodology_Translator	Academic_Research_Synthesis_Pipeline
High	Cultural_Signal_Detection	Cross_Cultural_Localization_Intelligence
High	Recursive_Self_Audit_Engine	Axiomatic_Logic_&_Audit_Systems

Appendix

Table 7: Representative distractor categories by interference tier.

Lost-in-the-Middle Effect

See all 9 papers

Sep 3, 2026

STAIR (STructure Aware Information Retriever): A novel dataset and LLM based retriever for document structure augmentation

Retrieval Augmented Generation (RAG) is a key component for generating accurate and hallucination free answers using Large Language Models (LLMs). LLMs are improving at handling long context, but still suffer from "lost in the middle" problem. Thus, precise and accurate retrieval is important. Current retrievers chunk long context into length-based manageable chunks - in the process throwing away rich and informative semantic global structure in the corpus. We introduce a novel retrieval system STAIR that empowers an LLM to exploit global structure in a corpus such as a Table of Contents (ToC) to efficiently store and retrieve information from its model parameters. Our thorough and careful ablation studies with a finetuned Differentiable Search Index (DSI) system show that ToC helps build a low hallucination (less than 0.05%) generative Information Retrieval (IR) system and can generalize to examples where very few training samples are available. To further research in this novel direction of ToC based retrieval we release SearchTome - a diverse benchmark created from 18 books across 6 diverse domains to further research in this novel direction. STAIR achieves a high Recall@1 score of 82.6% on SearchTome as compared to DSI (76.9%), where the difference is found to be statistically significant. STAIR easily beats other strong baselines such as BM25 (59.5%), DPR (68.7%) and out-of-the-box Mistral (13.8%).

Vineet Kumar, Meghanadh Pulivarthi, vishwajeet kumar +3

IBM

Retrieval-Augmented Generation Generative Retrieval Information Retrieval

Ask PDF

May 11, 2026

Route Before Retrieve: Activating Latent Routing Abilities of LLMs for RAG vs. Long-Context Selection

Recent advances in large language models (LLMs) have expanded the context window to beyond 128K tokens, enabling long-document understanding and multi-source reasoning. A key challenge, however, lies in choosing between retrieval-augmented generation (RAG) and long-context (LC) strategies: RAG is efficient but constrained by retrieval quality, while LC supports global reasoning at higher cost and with position sensitivity. Existing methods such as Self-Route adopt failure-driven fallback from RAG to LC, but remain passive, inefficient, and hard to interpret. We propose Pre-Route, a proactive routing framework that performs structured reasoning before answering. Using lightweight metadata (e.g., document type, length, initial snippet), Pre-Route enables task analysis, coverage estimation, and information-need prediction, producing explainable and cost-efficient routing decisions. Our study shows three key findings: (i) LLMs possess latent routing ability that can be reliably elicited with guidelines, allowing single-sample performance to approach that of multi-sample (Best-of-N) results; (ii) linear probes reveal that structured prompts sharpen the separability of the "optimal routing dimension" in representation space; and (iii) distillation transfers this reasoning structure to smaller models for lightweight deployment. Experiments on LaRA (in-domain) and LongBench-v2 (OOD) confirm that Pre-Route outperforms Always-RAG, Always-LC, and Self-Route baselines, achieving superior overall cost-effectiveness.

Yiwen Chen, Kuan Li, Fuzhen Zhuang +6

Beihang University · HKUST · Alibaba Group +1

LLM Prompting Long-Context Modeling LLM Routing

Ask PDF

Mar 11, 2026

Structured Linked Data as a Memory Layer for Agent-Orchestrated Retrieval

Retrieval-Augmented Generation (RAG) systems typically treat documents as flat text, ignoring the structured metadata and linked relationships that knowledge graphs provide. In this paper, we investigate whether structured linked data, specifically Schema.org markup and dereferenceable entity pages served by a Linked Data Platform, can improve retrieval accuracy and answer quality in both standard and agentic RAG systems. We conduct a controlled experiment across four domains (editorial, legal, travel, e-commerce) using Vertex AI Vector Search 2.0 for retrieval and the Google Agent Development Kit (ADK) for agentic reasoning. Our experimental design tests seven conditions: three document representations (plain HTML, HTML with JSON-LD, and an enhanced agentic-optimized entity page) crossed with two retrieval modes (standard RAG and agentic RAG with multi-hop link traversal), plus an Enhanced+ condition that adds rich navigational affordances and entity interlinking. Our results reveal that while JSON-LD markup alone provides only modest improvements, our enhanced entity page format, incorporating llms.txt-style agent instructions, breadcrumbs, and neural search capabilities, achieves substantial gains: +29.6% accuracy improvement for standard RAG and +29.8% for the full agentic pipeline. The Enhanced+ variant, with richer navigational affordances, achieves the highest absolute scores (accuracy: 4.85/5, completeness: 4.55/5), though the incremental gain over the base enhanced format is not statistically significant. We release our dataset, evaluation framework, and enhanced entity page templates to support reproducibility.

Andrea Volpini, Elie Raad, Beatrice Gamba +1

WordLift, Rome, Italy

Retrieval-Augmented Generation Graph Retrieval-Augmented Generation Agentic RAG RAG Evaluation

Ask PDF

Abstract

Lost-in-the-Middle Effect

Explore similar work

STAIR (STructure Aware Information Retriever): A novel dataset and LLM based retriever for document structure augmentation

Route Before Retrieve: Activating Latent Routing Abilities of LLMs for RAG vs. Long-Context Selection

Structured Linked Data as a Memory Layer for Agent-Orchestrated Retrieval