StrucChunk: Structure-Aware Retrieval for Legal Document RAG

Surya Deep Singh; Ashutosh Tripathi; Gaurav Deep Singh

cs.AIOct 19, 2026

StrucChunk: Structure-Aware Retrieval for Legal Document RAG

Authors: Surya Deep Singh, Ashutosh Tripathi, Gaurav Deep Singh

Organizations: Independent Researcher.

Abstract

Retrieval-Augmented Generation (RAG) can ground LLM outputs in domain-specific knowledge, but legal documents are hard: standard chunking fragments hierarchical argument structure, and generic embeddings struggle with both precise citation matching and semantic understanding. We propose StrucChunk (Structure Aware Chunking) with hybrid dense-sparse retrieval for hierarchically structured legal documents. It treats documents as directed acyclic graphs of provisions, preserves clause boundaries, resolves cross-references before embedding, and combines BM25 with BGE-M3 via Reciprocal Rank Fusion. We evaluate on five corpora, three legal traditions and two languages. StrucChunk reaches 100% boundary precision (vs. 7.7% for recursive splitting on CrPC), preserves 97.6% of documented cross-references (vs. 0%), and matches section coverage with 40.3% fewer chunks on GDPR. On LegalBench-RAG (6,858 query-passage pairs), it achieves Char-Recall@8 of 48.18% vs. 23.59% for the best baseline (+104%), surpassing the published state of the art by +83%. On 501 queries each for CrPC and CPC (Indian law), Recall@5 improves from 0.557 to 0.804 (+44.4%) and from 0.158 to 0.259 (+64.6%); chunking accounts for 88% and 73% of the gain. On 501 GDPR queries, Recall@5 is 0.868 with R@1 improving +59%. On NitiBench-CCL (3,730 Thai financial law queries), HR@5 is 0.856 vs. 0.594 for naive chunking (+44.1%), showing that section-boundary alignment is the dominant factor across languages, and cross-reference augmentation adds +2.3% on cross-reference queries. We release the implementation, 5,256 evaluation queries across five legal corpora, annotation methodology and chunking quality metrics. Code: github.com/SinghSuryaDeep/legal-rag-strucchunk-1

Figures & tables

Document	Domain	Pages	Sections	Cross-Refs
CrPC (1973)	Criminal	263	517	1,131
CPC (1908)	Civil	316	158+	814

Table 3: Evaluation corpus statistics. CPC additionally contains 51 Orders with multiple Rules each.

Property	CrPC/CPC	GDPR
Legal tradition	Common law (India)	Civil law (EU)
Document type	Procedural code	Regulatory instrument
Primary hierarchy	Chapter $\to$ Section	Chapter $\to$ Article $\to$ Para.
Secondary struct.	Subsections	Recitals (1–173)
Cross-ref style	“Section 167”	“Article 6(1)(f)”, “Recital 47”

Table 4: Structural comparison of CrPC/CPC and GDPR.

Property	CrPC/CPC/GDPR	NitiBench-CCL
Language	English	Thai
Legal tradition	Common / Civil law	Civil law (Thailand)
Document type	Statutory codes / Regulation	Financial law statutes
Primary hierarchy	Chapter $\to$ Section	Book $\to$ Chapter $\to$ Section
Section notation	“Section N”	“mātrā N”
Corpus size	517–517 sections	5,127 sections, 36 laws

Table 5: Structural comparison of English corpora and NitiBench-CCL.

Component	Specification
Embedding Model	BAAI/bge-m3 (567M params, 1024 dims; multilingual)
Vector Database	FAISS IndexFlatIP
Sparse Retrieval	bm25s v0.2.0
PDF Parsing	pdfplumber (1–3 only)
NitiBench corpus	HuggingFace Parquet (no PDF parsing needed)
Chunk Parameters	max_tokens=512, overlap=0.15

Table 6: Implementation specifications.

Metric	Recursive	StrucChunk
Boundary Precision (CrPC)	7.7%	100.0%
Boundary Precision (GDPR)	2.8%	100.0%
Cross-Ref Preservation (CrPC)	0.0% (0/42)	97.6% (41/42)
Cross-Ref Preservation (GDPR)	0.0% (0/9)	100.0% (9/9)
Chunk Count (CrPC)	491	503
Chunk Count (GDPR)	211	126

Table 7: Chunking quality analysis (intrinsic metrics, no retrieval pipeline required). The slight CrPC chunk-count increase for StrucChunk is intentional: oversized sections are split at sentence boundaries, preserving logical atomicity. Section Coverage at 0% for Recursive reflects that no chunk boundary aligns with a section delimiter, making it impossible to guarantee that any section is wholly contained in a single chunk; this is the intra-chunk semantic coherence failure mode that StrucChunk eliminates by design.

Chunking $\backslash$ Retrieval	Dense	BM25	Hybrid (RRF)
Recursive (baseline)	0.521	0.503	0.557
StrucChunk	0.800	0.737	0.804
Chunking effect (Hybrid fixed)	—	—	$+0.248$
Retrieval effect (StrucChunk fixed)	ref	—	$+0.004$

Table 13: CrPC 2

\times

2 ablation on Recall@5. Chunking accounts for

0.248/0.283=88\%

of total improvement over Recursive+Dense. Total gain =

0.804-0.521=+0.283

Chunking $\backslash$ Retrieval	Dense	BM25	Hybrid (RRF)
Recursive (baseline)	0.120	0.138	0.158
StrucChunk	0.198	0.269	0.259
Chunking effect (Hybrid fixed)	—	—	$+0.102$
Retrieval effect (StrucChunk fixed)	ref	—	$+0.061$

Table 15: CPC 2

\times

2 ablation on Recall@5. Chunking accounts for

0.102/0.140=73\%

of total improvement over Recursive+Dense. Total gain =

0.259-0.120=+0.140

; retrieval effect =

0.259-0.198=+0.061

Chunking $\backslash$ Retrieval	Dense	BM25	Hybrid (RRF)
Recursive (baseline)	0.798	0.679	0.818
StrucChunk	0.908	0.816	0.868
Chunking effect (Hybrid fixed)	—	—	$+0.050$
Retrieval effect (StrucChunk fixed)	ref	—	$-0.040$

Table 17: GDPR 2

\times

2 ablation on Recall@5. Retrieval effect is negative: StrucChunk+Dense (

0.908

) outperforms StrucChunk+Hybrid (

0.868

), confirming dense semantics alone are optimal for EU regulatory text with discriminative article titles.

Legal IR

Legal information retrieval finds and ranks relevant statutes, cases, patents, and other legal materials for queries and evidence-grounded systems.

See all 53 papers

Mar 2, 2026

Legal RAG Bench: an end-to-end benchmark for legal RAG

We introduce Legal RAG Bench, a benchmark and evaluation methodology for assessing the end-to-end performance of legal RAG systems. As a benchmark, Legal RAG Bench consists of 4,876 passages from the Victorian Criminal Charge Book alongside 100 complex, hand-crafted questions demanding expert knowledge of criminal law and procedure. Both long-form answers and supporting passages are provided. As an evaluation methodology, Legal RAG Bench leverages a full factorial design and novel hierarchical error decomposition framework, enabling apples-to-apples comparisons of the contributions of retrieval and reasoning models in RAG. We evaluate three state-of-the-art embedding models (Isaacus' Kanon 2 Embedder, Google's Gemini Embedding 001, and OpenAI's Text Embedding 3 Large) and two frontier LLMs (Gemini 3.1 Pro and GPT-5.2), finding that information retrieval is the primary driver of legal RAG performance, with LLMs exerting a more moderate effect on correctness and groundedness. Kanon 2 Embedder, in particular, had the largest positive impact on performance, improving average correctness by 17.5 points, groundedness by 4.5 points, and retrieval accuracy by 34 points. We observe that many errors attributed to hallucinations in legal RAG systems are in fact triggered by retrieval failures, concluding that retrieval sets the ceiling for the performance of many modern legal RAG systems. We document why and how we built Legal RAG Bench alongside the results of our evaluations. We also openly release our code and data to assist with reproduction of our findings.

Abdur-Rahman Butler, Umar Butler

Isaacus

Retrieval-Augmented Generation RAG Evaluation Legal QA IR Evaluation

Ask PDF

May 19, 2026

Chunking German Legal Code

This paper investigates chunking strategies for retrieval-augmented generation on German statutory law, using the German Civil Code as a structured benchmark corpus. We implement and compare a range of segmentation approaches, including structural units (sections, subsections, sentences, propositions), fixed-size windows, contextual chunking, semantic clustering, Lumber-style chunking, and RAPTOR-based hierarchical retrieval. All methods are evaluated on a legal question-answering dataset with section-level gold labels, measuring recall, query latency, index build time, and storage requirements. Results show that chunking strategies aligned with the inherent legal structure - particularly section and subsection - based retrieval-achieve the highest recall, while more complex approaches that override this structure perform worse. These simpler methods also offer favorable computational efficiency compared to LLM-intensive techniques such as contextual chunking, RAPTOR, and Lumber. The findings highlight a key trade-off between semantic enrichment and operational cost, and demonstrate that preserving domain-specific structure is critical for effective legal information retrieval.

Max Prior, Natalia Milanova, Andreas Schultz

Technical University of Munich, Boltzmannstraße 3, 85748 Garching near Munich

Document Chunking Retrieval-Augmented Generation Legal IR Legal QA IR Evaluation

Ask PDF

Oct 9, 2026

Verifiable Retrieval for Legal Reasoning through Structure-driven Knowledge Graph

Any automation within reliability-critical contexts such as medicine or law has to be transparent and verifiable. Despite their advantages, current LLM-based systems do not meet these requirements due to their inherent black-box nature. Retrieval Augmented Generation (RAG) attempts to ground the generative process in the relevant interpretable documents, thereby reducing the risk of AI 'hallucinations' and potentially enabling verification of the initial generation context. However, as the number and size of potential sources increases, naive retrieval approaches tend to perform worse, ignoring critical context elements. In this paper we aim to tackle this problem in the context of German Legal Code. Using structure-driven parsing to create a Knowledge Graph (KG) for the source document navigation and retrieval, we compare several KG-informed chunking and retrieval strategies and evaluate their effect on the accuracy of retrieved citations.

Ibrahim Siddig, Sviatoslav Tugeev, Munir Georges

Technische Hochschule Ingolstadt, Esplanade 10, 85049 Ingolstadt, Germany

Legal IR Graph Retrieval-Augmented Generation Legal Citation Verification

Ask PDF

Abstract

Legal IR

Explore similar work

Legal RAG Bench: an end-to-end benchmark for legal RAG

Chunking German Legal Code

Verifiable Retrieval for Legal Reasoning through Structure-driven Knowledge Graph