A Systematic Investigation of Document Chunking Strategies and Embedding Sensitivity

Muhammad Arslan Shaukat; Muntasir Adnan; Carlos C. N. Kuhn

cs.CLMar 7, 2026

A Systematic Investigation of Document Chunking Strategies and Embedding Sensitivity

Authors: Muhammad Arslan Shaukat, Muntasir Adnan, Carlos C. N. Kuhn

Organizations: Open Source Institute, Faculty of Science and Technology, University of Canberra, Canberra, Australia

Abstract

We present the first large-scale, cross-domain evaluation of document chunking strategies for dense retrieval, addressing a critical but underexplored aspect of retrieval-augmented systems. In our study, 36 segmentation methods spanning fixed-size, semantic, structure-aware, hierarchical, adaptive, and LLM-assisted approaches are benchmarked across six diverse knowledge domains using five different embedding models. Retrieval performance is assessed using graded relevance scores from a state-of-the-art LLM evaluator, with Normalised DCG@5 as the primary metric (complemented by Hit@5 and MRR). Our experiments show that content-aware chunking significantly improves retrieval effectiveness over naive fixed-length splitting. The top-performing strategy, Paragraph Group Chunking, achieved the highest overall accuracy (mean nDCG@5~~0.459) and substantially better top-rank hit rates (Precision@1~~24%, Hit@5~~59%). In contrast, simple fixed-size character chunking as baselines performed poorly (nDCG@5 < 0.244, Precision@1~~2-3%). We observe pronounced domain-specific differences: dynamic token sizing is strongest in biology, physics and health, while paragraph grouping is strongest in legal and maths. Larger embedding models yield higher absolute scores but remain sensitive to suboptimal segmentation, indicating that better chunking and large embeddings provide complementary benefits. In addition to accuracy gains, we quantify the efficiency trade-offs of advanced chunking. Producing more, smaller chunks can increase index size and latency. Consequently, we identify methods (like dynamic chunking) that approach an optimal balance of effectiveness and efficiency. These findings establish chunking as a vital lever for improving retrieval performance and reliability.

Figures & tables

Figure 1: End-to-end experimental pipeline for evaluating document chunking strategies in dense retrieval. Documents from six knowledge domains (Biology, Physics, Health, Legal, Maths, Agriculture) in the UltraDomain dataset are segmented using 36 chunking strategies spanning six design categories. The resulting chunks are embedded using five dense embedding models and indexed into separate Qdrant vector stores, yielding 1,080 unique configurations. For each query, the top-5 retrieved chunks are evaluated by a Mixtral-8x22B LLM judge against the golden reference answer using a three-point graded relevance scale. The query is intentionally withheld from the evaluator to prevent lexical bias. Efficiency metrics - including index size, query latency, and memory usage are recorded in parallel across all configurations. No generation component is included; evaluation is retrieval only.

Category	Strategy Name	Abbreviation	Configuration Set
Deterministic / Rule-Based	Fixed Character Chunking	FCC	char_size=100, overlap=10
	Fixed Token Chunking	FC	token_size=50, overlap=0
	Overlapping Token Chunking	OFC	token_size=50, overlap=10
	Sliding Window Token Chunking	SWC	window=50, step=25
	Length Aware Chunking	LAC	target=500, tolerance=100
	Sentence Based Chunking	SBC	sentence boundaries

Table 1: Strategy configuration matrix showing chunking category, strategy name, shorthand alias used throughout the paper, and the corresponding operational configuration derived from the experimental setup.

Strategy	Operational Definition	Formal Specification and Parameters
Fixed Token Chunking (FC)	The document is tokenised and split into contiguous, non-overlapping chunks of fixed length. Each chunk is treated as an independent retrieval unit.	Let $T=[t_{1},\dots,t_{N}]$ be the token sequence of document $D$ . The $i$ -th chunk is defined as $C_{i}=T[(i-1)K:(i-1)K+K]$ , where $K$ is the fixed number of tokens per chunk.
Fixed Character Chunking (FCC)	The raw document text is divided into contiguous character spans of equal length, independent of linguistic structure.	Let $D$ be a character sequence. $C_{i}=D[(i-1)L:(i-1)L+L]$ , where $L$ is the fixed number of characters per chunk.
Sliding Window Token Chunking (SWC / OFC)	Overlapping token windows are generated to preserve contextual continuity across chunk boundaries.	$C_{i}=T[(i-1)S:(i-1)S+W]$ , where $W$ is the window size (tokens) and $S$ is the stride. Overlap occurs when $S<W$ .
Sentence-Based and Sentence-Group Chunking (SBC / SGC)	Chunks are formed by grouping one or more consecutive sentences, preserving sentence boundaries.	Let $\{s_{1},\dots,s_{n}\}$ denote sentences in $D$ . $C_{i}=\{s_{(i-1)(G-O)+1},\dots,s_{(i-1)(G-O)+G}\}$ , where $G$ is the number of sentences per chunk and $O$ is sentence overlap.
Paragraph-Based and Paragraph-Group Chunking (PBC / PGC)	Chunks align with paragraph boundaries or groups of adjacent paragraphs.	Let $\{p_{1},\dots,p_{m}\}$ denote paragraphs. $C_{i}=\{p_{(i-1)(G-O)+1},\dots,p_{(i-1)(G-O)+G}\}$ , where $G$ is the number of paragraphs and $O$ is overlap.
Recursive Chunking (RC / RTF)	Documents are recursively segmented using increasingly fine rules until a size constraint is met, with fallback token splitting if required.	Given segment $x$ , recursively apply $\text{split}(x)$ until $\|C_{i}\|\leq L$ , where $L$ is the maximum allowable chunk size (tokens or characters).

Table 2: Detailed specification of chunking strategies evaluated in this study. Each row provides a complete operational and mathematical definition; all implemented strategies correspond to parameterised instances or hybrids of these formulations.

Figure 2: Distribution of nDCG@5 for all chunking strategies across embedding models and domains, ordered by descending mean nDCG@5. Red diamonds denote mean values and black lines denote medians. Higher-ranked methods such as PGC, HPGC, and DFC show stronger overall retrieval effectiveness, while FC, FCC, and HFCF exhibit weaker performance. Differences in box and whisker widths highlight stability variations acroos queries. The chunking method abbreviations on the x-axis correspond to those defined in Table 1

Figure 3: Mean nDCG@5 scores for Domain-specific retrieval performance across Agriculture, Biology, Health, Legal, Maths and Physics. Dynamic Token Size Chunking (DFC) achieves the highest single score in Health and Physics while paragraph-aware and late-stage strategies, particularly Paragraph Group Chunking (PGC), Hybrid Paragraph Group Fixed Token Chunking (HPGC), and Late Chunking Token Spans (LCTS), consistently rank among the top three methods across all domains. The chunking method abbreviations on the x-axis correspond to those defined in Table 1 .

Figure 4: Heatmap of mean nDCG@5 across five embedding models and thirty-six chunking strategies. The results show that bge-m3 (0.456) delivers the strongest overall performance, with all-MiniLM-L6-v2 (0.416) as the clear second best. The potion-base variants trail behind, with scores generally tapering off across later chunking strategies. The chunking method abbreviations on the x-axis correspond to those defined in Table 1 .

Figure 5: Effectiveness-efficiency trade-off plots. Left: mean nDCG@5 against index size, highlighting how each strategy scales in terms of index growth. Right: mean nDCG@5 against query latency, illustrating the trade-off between retrieval accuracy and response time for each chunking approach. The chunking method abbreviations on the x-axis correspond to those defined in Table 1 .

Strategy	RAM Cost	Chunking	Query Time	Strategy	RAM Cost	Chunking	Query Time
	(MB)	Time (s)	(ms)		(MB)	Time (s)	(ms)
LSTC	5,672.16	10.02	3.84	LCTS	929.34	7.77	5.28
LBDC	5,417.98	9.27	3.62	LCSI	926.96	9.77	3.92
TBC	2,849.53	133.79	4.37	HPGC	925.70	8.52	4.27
HSSC	1,744.49	77.40	4.02	SBC	922.47	9.52	3.88
HSmFC	1,700.33	75.15	4.40	FC	915.20	7.77	4.02

Table 3: Performance trade-offs for all 36 evaluated chunking strategies, sorted in descending order by chunking phase RAM. While all methods maintain an average query time of under 7 milliseconds, the preprocessing costs vary severely. LLM-based strategies (e.g., LSTC, LBDC) demand extreme memory overhead, and complex semantic clustering (e.g., TBC, HSSC) requires significantly higher processing time. Conversely, high-performing structural methods like Paragraph Group Chunking (PGC) remain highly resource efficient across all three metrics.

Figure 6: Pearson correlation matrix across all evaluation metrics. The nDCG@5 family shows strong internal consistency and high correlation with MRR@5 (r=0.92) and Precision@5 (r=0.88), validating it as the primary ranking metric. Embedding dimensionality shows minimal correlation with performance metrics (|r| < 0.2), confirming that chunking strategy selection dominates model architecture as a performance driver. Efficiency metrics (index size, latency) occupy the lower-right quadrant and show expected negative trade-off relationships.

Document Chunking

Document chunking splits long or structured documents into coherent units for retrieval, question answering, and other language-model tasks.

See all 52 papers

May 21, 2026

Evaluation of Chunking Strategies for Effective Text Embedding in Low-Resource Language on Agricultural Documents

In this study, we compare the performance of four text chunking approaches: Recursive, Khmer-Aware, Sentence-Based, and LLM-Based within a Retrieval-Augmented Generation (RAG) framework applied to Khmer agricultural documents. The document chunks are encoded using the BGE-M3 multilingual embedding model and retrieved using the FAISS library. Performance is evaluated using four metrics: Average Retrieval Score (L2 distance), Answer Relevance, Khmer Coverage, and Khmer Intersection over Union, all measured against ground-truth question-answer pairs. For evaluation, we perform 5-fold cross-validation over 18 question-answer pairs. We observe the best performance for the character-based Recursive chunking method with a chunk size of 300 characters, achieving the lowest L2 distance (0.4295 +- 0.0461), highest Answer Relevance (0.8663 +- 0.0199), and highest Khmer IoU (0.6441 +- 0.0347). A paired t-test shows a statistically significant improvement over the Sentence-Based chunking method in L2 distance (p = 0.0121). These results highlight the importance of segmentation granularity and structural preservation for optimizing dense retrieval in morphologically complex, low-resource languages such as Khmer.

Sovandara Chhoun, Pichdara Po, Sereiwathna Ros +2

Department of Big Data, Chungbuk National University, Cheongju-si, South Korea · Department of Computer Science, Chungbuk National University, Cheongju-si, South Korea · BigDataLabs Co., Ltd. Department of Management Information Systems, Chungbuk National University, South Korea

Document Chunking Multilingual IR Information Retrieval Low-Resource Language Processing Text Embedding Models

Ask PDF

Aug 24, 2026

Comparing Chunking and Embedding Strategies for Turkish RAG Systems

Retrieval-Augmented Generation conditions a language model on chunks retrieved from a document collection. Its accuracy is therefore limited by the chunking and embedding stages that determine what can be retrieved. We compare Turkish document question answering across three chunking strategies (fixed-length, semantic, and layout-aware Docling), five embedding models, and two LLMs, over three documents with contrasting layouts. Every configuration answers the same question set, which allows component effects to be separated by paired testing rather than inferred from separate benchmarks. The fully crossed design yields 9{,}000 graded question-answer evaluations, each scored by an independent judge model, and component comparisons are tested by paired McNemar tests under Holm correction. The three leading embedding models are statistically indistinguishable, so language specialization yields no measurable retrieval advantage. The faster LLM is not the more accurate one. The preferred configuration depends on content type, since layout-aware chunking helps table-heavy documents far more than text-heavy ones.

Mustafa Sertaç Türkel, Fatma Nur Korkmaz, Ahmet Tuğrul Bayrak

Data Science and Innovation Ata Technology Platforms Istanbul, Turkey

Retrieval-Augmented Generation RAG Evaluation Document Chunking Text Embedding Evaluation

Ask PDF

Apr 29, 2026

Query-Adaptive Semantic Chunking for Retrieval-Augmented Generation: A Dynamic Strategy with Contextual Window Expansion

Retrieval-Augmented Generation (RAG) systems depend critically on document chunking quality for retrieving relevant context. Fixed chunking segments documents into uniform units irrespective of semantics or user intent, producing a precision-recall trade-off unresolvable by tuning chunk size alone. Semantic and agentic methods partially address these limitations but do not integrate user queries at the chunking stage. We present Query-Adaptive Semantic Chunking (QASC), which dynamically constructs chunks by integrating queries into segmentation through three mechanisms: cosine similarity scoring between sentence and query embeddings to identify seed sentences, contextual window expansion around seeds to preserve coherence, and chunk-level score aggregation to ensure holistic relevance. We evaluate QASC on 100 technical documents across 200 queries spanning four types, comparing against fixed chunking at five granularities, recursive splitting, semantic chunking, and agentic chunking. QASC achieves an F1-score of 0.85, a relative improvement of 18-27% over fixed chunking and 8-12% over semantic and agentic alternatives. Ablation studies confirm each component contributes meaningfully. Human evaluation by three annotators (Cohen kappa = 0.82) corroborates that QASC produces more relevant and coherent chunks than existing methods.

Mudit Rastogi

Independent Researcher

Document Chunking Retrieval-Augmented Generation

Ask PDF

Abstract

Document Chunking

Explore similar work

Evaluation of Chunking Strategies for Effective Text Embedding in Low-Resource Language on Agricultural Documents

Comparing Chunking and Embedding Strategies for Turkish RAG Systems

Query-Adaptive Semantic Chunking for Retrieval-Augmented Generation: A Dynamic Strategy with Contextual Window Expansion