StrucChunk: Structure-Aware Retrieval for Legal Document RAG

Surya Deep Singh; Ashutosh Tripathi; Gaurav Deep Singh

cs.AIOct 19, 2026

StrucChunk: Structure-Aware Retrieval for Legal Document RAG

Authors: Surya Deep Singh, Ashutosh Tripathi, Gaurav Deep Singh

Organizations: Independent Researcher.

Abstract

Retrieval-Augmented Generation (RAG) can ground LLM outputs in domain-specific knowledge, but legal documents are hard: standard chunking fragments hierarchical argument structure, and generic embeddings struggle with both precise citation matching and semantic understanding. We propose StrucChunk (Structure Aware Chunking) with hybrid dense-sparse retrieval for hierarchically structured legal documents. It treats documents as directed acyclic graphs of provisions, preserves clause boundaries, resolves cross-references before embedding, and combines BM25 with BGE-M3 via Reciprocal Rank Fusion. We evaluate on five corpora, three legal traditions and two languages. StrucChunk reaches 100% boundary precision (vs. 7.7% for recursive splitting on CrPC), preserves 97.6% of documented cross-references (vs. 0%), and matches section coverage with 40.3% fewer chunks on GDPR. On LegalBench-RAG (6,858 query-passage pairs), it achieves Char-Recall@8 of 48.18% vs. 23.59% for the best baseline (+104%), surpassing the published state of the art by +83%. On 501 queries each for CrPC and CPC (Indian law), Recall@5 improves from 0.557 to 0.804 (+44.4%) and from 0.158 to 0.259 (+64.6%); chunking accounts for 88% and 73% of the gain. On 501 GDPR queries, Recall@5 is 0.868 with R@1 improving +59%. On NitiBench-CCL (3,730 Thai financial law queries), HR@5 is 0.856 vs. 0.594 for naive chunking (+44.1%), showing that section-boundary alignment is the dominant factor across languages, and cross-reference augmentation adds +2.3% on cross-reference queries. We release the implementation, 5,256 evaluation queries across five legal corpora, annotation methodology and chunking quality metrics. Code: github.com/SinghSuryaDeep/legal-rag-strucchunk-1

Figures & tables

Document	Domain	Pages	Sections	Cross-Refs
CrPC (1973)	Criminal	263	517	1,131
CPC (1908)	Civil	316	158+	814

Table 3: Evaluation corpus statistics. CPC additionally contains 51 Orders with multiple Rules each.

Property	CrPC/CPC	GDPR
Legal tradition	Common law (India)	Civil law (EU)
Document type	Procedural code	Regulatory instrument
Primary hierarchy	Chapter $\to$ Section	Chapter $\to$ Article $\to$ Para.
Secondary struct.	Subsections	Recitals (1–173)
Cross-ref style	“Section 167”	“Article 6(1)(f)”, “Recital 47”

Table 4: Structural comparison of CrPC/CPC and GDPR.

Property	CrPC/CPC/GDPR	NitiBench-CCL
Language	English	Thai
Legal tradition	Common / Civil law	Civil law (Thailand)
Document type	Statutory codes / Regulation	Financial law statutes
Primary hierarchy	Chapter $\to$ Section	Book $\to$ Chapter $\to$ Section
Section notation	“Section N”	“mātrā N”
Corpus size	517–517 sections	5,127 sections, 36 laws

Table 5: Structural comparison of English corpora and NitiBench-CCL.

Component	Specification
Embedding Model	BAAI/bge-m3 (567M params, 1024 dims; multilingual)
Vector Database	FAISS IndexFlatIP
Sparse Retrieval	bm25s v0.2.0
PDF Parsing	pdfplumber (1–3 only)
NitiBench corpus	HuggingFace Parquet (no PDF parsing needed)
Chunk Parameters	max_tokens=512, overlap=0.15

Table 6: Implementation specifications.

Metric	Recursive	StrucChunk
Boundary Precision (CrPC)	7.7%	100.0%
Boundary Precision (GDPR)	2.8%	100.0%
Cross-Ref Preservation (CrPC)	0.0% (0/42)	97.6% (41/42)
Cross-Ref Preservation (GDPR)	0.0% (0/9)	100.0% (9/9)
Chunk Count (CrPC)	491	503
Chunk Count (GDPR)	211	126

Table 7: Chunking quality analysis (intrinsic metrics, no retrieval pipeline required). The slight CrPC chunk-count increase for StrucChunk is intentional: oversized sections are split at sentence boundaries, preserving logical atomicity. Section Coverage at 0% for Recursive reflects that no chunk boundary aligns with a section delimiter, making it impossible to guarantee that any section is wholly contained in a single chunk; this is the intra-chunk semantic coherence failure mode that StrucChunk eliminates by design.

Chunking $\backslash$ Retrieval	Dense	BM25	Hybrid (RRF)
Recursive (baseline)	0.521	0.503	0.557
StrucChunk	0.800	0.737	0.804
Chunking effect (Hybrid fixed)	—	—	$+0.248$
Retrieval effect (StrucChunk fixed)	ref	—	$+0.004$

Table 13: CrPC 2

\times

2 ablation on Recall@5. Chunking accounts for

0.248/0.283=88\%

of total improvement over Recursive+Dense. Total gain =

0.804-0.521=+0.283

Chunking $\backslash$ Retrieval	Dense	BM25	Hybrid (RRF)
Recursive (baseline)	0.120	0.138	0.158
StrucChunk	0.198	0.269	0.259
Chunking effect (Hybrid fixed)	—	—	$+0.102$
Retrieval effect (StrucChunk fixed)	ref	—	$+0.061$

Table 15: CPC 2

\times

2 ablation on Recall@5. Chunking accounts for

0.102/0.140=73\%

of total improvement over Recursive+Dense. Total gain =

0.259-0.120=+0.140

; retrieval effect =

0.259-0.198=+0.061

Chunking $\backslash$ Retrieval	Dense	BM25	Hybrid (RRF)
Recursive (baseline)	0.798	0.679	0.818
StrucChunk	0.908	0.816	0.868
Chunking effect (Hybrid fixed)	—	—	$+0.050$
Retrieval effect (StrucChunk fixed)	ref	—	$-0.040$

Table 17: GDPR 2

\times

2 ablation on Recall@5. Retrieval effect is negative: StrucChunk+Dense (

0.908

) outperforms StrucChunk+Hybrid (

0.868

), confirming dense semantics alone are optimal for EU regulatory text with discriminative article titles.

Legal IR

Legal information retrieval finds and ranks relevant statutes, cases, patents, and other legal materials for queries and evidence-grounded systems.

See all 53 papers

Legal RAG Bench: an end-to-end benchmark for legal RAG
Mar 2, 2026Abdur-Rahman Butler, Umar ButlerRetrieval-Augmented Generation RAG Evaluation
Isaacus
Chunking German Legal Code
May 19, 2026Max Prior, Natalia Milanova, Andreas SchultzDocument Chunking Retrieval-Augmented Generation
Technical University of Munich, Boltzmannstraße 3, 85748 Garching near Munich
Verifiable Retrieval for Legal Reasoning through Structure-driven Knowledge Graph
Oct 9, 2026Ibrahim Siddig, Sviatoslav Tugeev, Munir GeorgesLegal IR Graph Retrieval-Augmented Generation
Technische Hochschule Ingolstadt, Esplanade 10, 85049 Ingolstadt, Germany

Mar 2, 2026J/K move · Enter open · S save

Legal RAG Bench: an end-to-end benchmark for legal RAG

Abdur-Rahman Butler, Umar Butler

Isaacus

Retrieval-Augmented Generation RAG Evaluation Legal QA IR Evaluation

Open paper AskPDF arXiv

We introduce Legal RAG Bench, a benchmark and evaluation methodology for assessing the end-to-end performance of legal RAG systems. As a benchmark, Legal RAG Bench consists of 4,876 passages from the Victorian Criminal Charge Book alongside 100 complex, hand-crafted questions demanding expert knowledge of criminal law and procedure. Both long-form answers and supporting passages are provided. As an evaluation methodology, Legal RAG Bench leverages a full factorial design and novel hierarchical error decomposition framework, enabling apples-to-apples comparisons of the contributions of retrieval and reasoning models in RAG. We evaluate three state-of-the-art embedding models (Isaacus' Kanon 2 Embedder, Google's Gemini Embedding 001, and OpenAI's Text Embedding 3 Large) and two frontier LLMs (Gemini 3.1 Pro and GPT-5.2), finding that information retrieval is the primary driver of legal RAG performance, with LLMs exerting a more moderate effect on correctness and groundedness. Kanon 2 Embedder, in particular, had the largest positive impact on performance, improving average correctness by 17.5 points, groundedness by 4.5 points, and retrieval accuracy by 34 points. We observe that many errors attributed to hallucinations in legal RAG systems are in fact triggered by retrieval failures, concluding that retrieval sets the ceiling for the performance of many modern legal RAG systems. We document why and how we built Legal RAG Bench alongside the results of our evaluations. We also openly release our code and data to assist with reproduction of our findings.

Abstract

Legal IR

Explore similar work

Legal RAG Bench: an end-to-end benchmark for legal RAG

Chunking German Legal Code

Verifiable Retrieval for Legal Reasoning through Structure-driven Knowledge Graph