MultiDocFusion: Hierarchical and Multimodal Chunking Pipeline for Enhanced RAG on Long Industrial Documents

Joongmin Shin; Chanjun Park; Jeongbae Park; Jaehyung Seo; Heuiseok Lim

cs.AIApr 14, 2026

MultiDocFusion: Hierarchical and Multimodal Chunking Pipeline for Enhanced RAG on Long Industrial Documents

Authors: Joongmin Shin, Chanjun Park, Jeongbae Park, Jaehyung Seo, Heuiseok Lim

Organizations: Human-inspired AI Research, Korea University · School of Software, Soongsil University · Department of Computer Science and Engineering, Korea University

Abstract

RAG-based QA has emerged as a powerful method for processing long industrial documents. However, conventional text chunking approaches often neglect complex and long industrial document structures, causing information loss and reduced answer quality. To address this, we introduce MultiDocFusion, a multimodal chunking pipeline that integrates: (i) detection of document regions using vision-based document parsing, (ii) text extraction from these regions via OCR, (iii) reconstruction of document structure into a hierarchical tree using large language model (LLM)-based document section hierarchical parsing (DSHP-LLM), and (iv) construction of hierarchical chunks through DFS-based grouping. Extensive experiments across industrial benchmarks demonstrate that MultiDocFusion improves retrieval precision by 8-15% and ANLS QA scores by 2-3% compared to baselines, emphasizing the critical role of explicitly leveraging document hierarchy for multimodal document-based QA. These significant performance gains underscore the necessity of structure-aware chunking in enhancing the fidelity of RAG-based QA systems.

Figures & tables

Figure 1: The pipeline for MultiDocFusion . The figure illustrates the step-by-step process for handling a long industrial document. (a) DP extracts layout structures; (b) OCR recognizes and annotates text; (c) DSHP-LLM constructs a hierarchical tree from identified section headers and general nodes; (d) DFS-based Grouping constructs coherent hierarchical chunks for retrieval tasks. The color-coded blocks represent document elements: yellow for Root and Title, red for Section Headers, and green for general nodes (tables, figures, and text blocks).

Model	DocHieNet		HRDH
Model	F1	TEDS	F1	TEDS
GPT-4	0.5139	0.6961	0.2594	0.3342
Llama-3.2-3B	0.2558	0.5464	0.4389	0.4904
$\hookrightarrow$ DSHP-LLM	0.4894	0.7549	0.8664	0.8459
Qwen-2.5-3B	0.4122	0.6995	0.3299	0.3734
$\hookrightarrow$ DSHP-LLM	0.4808	0.6957	0.8856	0.8658

Table 1: Performance on DHP datasets (DocHieNet + HRDH) for DSHP-LLM (section headers).

\hookrightarrow

: DSHP-LLM applied. Bold : improvement over baseline.

Chunking Method	DUDE			MPVQA			CUAD			MOAMOB
Chunking Method	Recall	Precision	nDCG	Recall	Precision	nDCG	Recall	Precision	nDCG	Recall	Precision	nDCG
Length chunking	0.2628	0.1686	0.2166	0.2523	0.1587	0.1933	0.9011	0.8537	0.8776	0.6462	0.5676	0.6209
Semantic chunking	0.0956	0.0549	0.0775	0.0939	0.0524	0.0680	0.7684	0.6719	0.7181	0.2737	0.1950	0.2453
LumberChunker	0.2395	0.1533	0.1986	0.2152	0.1298	0.1609	0.9031	0.8576	0.8800	0.6130	0.5205	0.5692
Perplexity chunking	0.2428	0.1559	0.2020	0.2159	0.1318	0.1629	0.8869	0.8395	0.8603	0.6173	0.5241	0.5785
Structure-based chunking	0.2219	0.1450	0.1862	0.2036	0.1230	0.1524	0.8844	0.8311	0.8581	0.5544	0.4662	0.5149

Table 2: Retrieval performance by Chunking Method (Average Recall, Precision, nDCG for top-

k=1\sim 4

), Best scores are in bold .

	DUDE			MPVQA			CUAD			MOAMOB
Chunking Method	ANLS	ROUGE-L	METEOR	ANLS	ROUGE-L	METEOR	ANLS	ROUGE-L	METEOR	ANLS	ROUGE-L	METEOR
Length chunking	0.1611	0.1444	0.1988	0.1398	0.0966	0.1408	0.2585	0.1677	0.1662	0.2497	0.0823	0.1115
Semantic chunking	0.1548	0.1261	0.1657	0.1332	0.0805	0.0978	0.2593	0.1491	0.1468	0.2455	0.0846	0.1043
LumberChunker	0.1531	0.1284	0.1752	0.1307	0.0769	0.0993	0.2657	0.1630	0.1650	0.2536	0.0848	0.1167
Perplexity chunking	0.1653	0.1390	0.1855	0.1344	0.0751	0.0950	0.2641	0.1646	0.1524	0.2532	0.0894	0.1190
Structure-based chunking	0.1751	0.1489	0.1921	0.1537	0.0980	0.1278	0.2498	0.1556	0.1591	0.2501	0.0979	0.1114

Table 3: Average QA performance (ANLS, ROUGE-L, METEOR) of six chunking strategies on DUDE, MPVQA, CUAD, and MOAMOB datasets, for top-

k\in\{1,4\}

. Results are averaged over Llama-3.2-3B, Mistral-8B, and Qwen-2.5-7B models. Best scores are in bold .

Chunking Method	DETR	DiT	VGT	Avg
Length chunking	0.4952	0.4863	0.4497	0.4771
Semantic chunking	0.3247	0.3263	0.2297	0.2936
LumberChunker	0.4620	0.4533	0.4412	0.4522
Perplexity chunking	0.4636	0.4460	0.4436	0.4511
Structure-based chunking	0.4396	0.4171	0.4269	0.4279
MultiDocFusion	0.5014	0.4976	0.5061	0.5017

Table 4: Average performance of Chunking Methods by DP model (top-

k=1\sim 4

nDCG).

Chunking Method	EasyOCR	Tesseract	TrOCR	Avg
Length chunking	0.5369	0.4799	0.4144	0.4771
Semantic chunking	0.3213	0.2757	0.2423	0.2798
LumberChunker	0.5057	0.4546	0.3963	0.4522
Perplexity chunking	0.5115	0.4674	0.3739	0.4509
Structure-based chunking	0.5194	0.4650	0.2993	0.4279
MultiDocFusion	0.5681	0.5068	0.4097	0.4949

Table 5: Average performance of Chunking Methods by OCR model (top-

k=1\sim 4

nDCG).

Chunking Method	BGE	E5	BM25	Avg
Length chunking	0.4834	0.4715	0.4764	0.4771
Semantic chunking	0.3114	0.3378	0.1825	0.2772
LumberChunker	0.4708	0.4319	0.4539	0.4522
Perplexity chunking	0.4715	0.4318	0.4495	0.4509
Structure-based chunking	0.4679	0.4040	0.4118	0.4279
MultiDocFusion	0.5213	0.4884	0.5085	0.5061

Table 6: Average performance of Chunking Methods by embedding model (top-

k=1\sim 4

nDCG).

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Type/Domain	#Documents	Avg. Pages	#QA pairs
DocHieNet	Mixed (Reports/Papers/Industrial)	1,673	5.3	-
HRDH	Academic Papers (arXiv)	1,500	7.1	-
MPVQA	General Documents (Multi-page)	17,000	3.4	48,000+
CUAD	Legal Documents (Contracts)	510	6.2	13,000+
DUDE	Mixed (Financial Reports/Manuals)	3,000+	4.9	7,000+
MOAMOB	Industrial Technical Documents	2	35.5	71

Appendix

Table 7: Summary of key datasets used in MultiDocFusion.

Method	Metric	Avg. Length (Characters / Tokens)	Number of Chunks
Length chunking	Characters	789.25	6,807
Length chunking	Tokens	548.30	6,807
Semantic chunking	Characters	289.10	18,498
Semantic chunking	Tokens	201.54	18,498
LumberChunker	Characters	702.45	10,650
LumberChunker	Tokens	483.82	10,650

Appendix

Table 8: Chunk statistics (average length and total number) for Length chunking, Semantic chunking, LumberChunker, Perplexity, Structure-based, and MultiDocFusion chunking methods ( max_len=550 tokens)

Chunking Method	DUDE	MPVQA
Length chunking	0.1592	0.1348
Semantic chunking	0.1537	0.1294
LumberChunker	0.1573	0.1351
Perplexity chunking	0.1668	0.1299
Structure based Chunking	0.1683	0.1488
MultiDocFusion	0.1793	0.1544

Appendix

Table 9: Official test-server ANLS on DUDE and MPVQA. Ground-truth is hidden on the server, so retrieval metrics cannot be computed; main paper reports corpus-level retrieval+QA on validation splits.

System Prompt (Common)

You are an expert in analyzing section headers of documents and creating a hierarchical structure. The following is a list of ’section header’ texts extracted from a document. For each item, determine its relationship with the parent section (parent-child relationship). If possible, follow standard document numbering rules, such as treating ’3.1’ as a child of ’3’ and ’3.1.1’ as a child of ’3.1’. Even if there is no numeric pattern, infer hierarchy based on textual context. If an item is a top-level heading (i.e., the root node is its parent), set ‘parent‘ to null. Output format: json only. DO NOT include any other explanations or text. [ {"id": "<id from the original header_list>", "parent": "<id of the parent node or null if root>"} ]

Appendix

Table 10: Prompt examples for DSHP - LLM model training. The common prompt (top) is used for both Example 1 (easier, with numbered sections) and Example 2 (harder, no section numbers). Lines expanded so that both examples reach a similar height. The parent value null denotes the root node. The symbol ... indicates omitted content for brevity.

Figure 2: An example of a long industrial document (MPVQA) illustrating the content structure and formatting used for guidelines and requirements in nuclear power plant operations. The document contains various sections, such as general information, application scope, and specific criteria, serving as a representative case for evaluating document chunking methods.

Panel of Figure 3 — Figure 3: Step-by-step illustration aligned with the MultiDocFusion pipeline: (a) Document Parsing (DP), (b) OCR, (c) DSHP-LLM for section hierarchy reconstruction and node attachment, and (d) DFS-based Grouping for hierarchy-aware chunking.

Method	Chunk	Example Content
Length chunking	Chunk 1	A Proposal for — Jeffrey A. Solomon _ Product Director, HRT _ Women’s Health Care “Wyeth-Ayerst Pharmaceuticals REMARIN” FAMILY OF PRODUCTS Medical and Scientific Communications Plan 2000 tember 1999 PROPRIETARY AND CONFIDENTIAL …
	Chunk 2	INTRODUCTION Design Write has prepared the following proposal for a comprehensive educational and communications program …
	Chunk 3	we have been guided by the brand operating strategies for 2000. We have had the fortunate experience of working with the HRT Management Team since 1997 …
Semantic Chunking	Chunk 1	A Proposal for — Jeffrey A. Solomon _ Product Director, HRT _ Women’s Health Care “Wyeth-Ayerst Pharmaceuticals PREMARIN” FAMILY OF PRODUCTS Medical and Scientific Communications Plan 2000 tember 1999 PROPRIETARY AND CONFIDENTIAL © 1999 by Design Write, Ine. EXHIBIT W_§ Source: hops vAwaaw indusirvvdagumens cst eduideesipbwila \|?
	Chunk 2	Medical and Scientific Communications Plan. 2000: Premarin® Family of Products
	Chunk 3	INTRODUCTION Design Write has prepared the following proposal for a comprehensive educational and communications program to support the PREMARIN® Family of Products. Design Write thanks the Women’s Health Care, HRT Management Team, for this opportunity to present its ideas to further the goal of expanding the PREMARIN Family of Products’ position in the marketplace, In developing this proposal, we have been guided by the brand operating strategies for 2000.

Appendix

Table 11: Qualitative comparison of chunking methods applied to the document in Figure 2 . Each method shows three chunks (1 to 3) for six approaches: Length chunking, Semantic chunking, LumberChunker, Perplexity chunking, Structure-based chunking, and MultiDocFusion .

Chunking Method	$\mathbf{k}$	DUDE			MPVQA			CUAD			MOAMOB
		R	P	nDCG	R	P	nDCG	R	P	nDCG	R	P	nDCG
Length chunking	1	0.1642	0.1642	0.1642	0.1556	0.1556	0.1556	0.8607	0.8607	0.8607	0.5847	0.5847	0.5847
	2	0.2454	0.1684	0.2105	0.2367	0.1591	0.1815	0.9001	0.8513	0.8770	0.6435	0.5711	0.6218
	3	0.3001	0.1700	0.2368	0.2889	0.1599	0.2084	0.9157	0.8514	0.8842	0.6696	0.5605	0.6349
	4	0.3416	0.1717	0.2546	0.3278	0.1601	0.2278	0.9278	0.8513	0.8883	0.6869	0.5542	0.6423
Semantic chunking	1	0.0586	0.0586	0.0586	0.0562	0.0562	0.0562	0.6810	0.6810	0.6810	0.2079	0.2079	0.2079

Appendix

Table 12: Retrieval summary by Chunking Method — Comparison of VQA Datasets (top-

k=1\sim 4

)

Chunking Method	OCR	DUDE			MPVQA			CUAD			MOAMOB
		R	P	nDCG	R	P	nDCG	R	P	nDCG	R	P	nDCG
Length chunking	easyocr	0.2836	0.1805	0.2289	0.2701	0.1658	0.2042	0.9076	0.8756	0.8923	0.8511	0.7689	0.8223
	tesseract	0.2511	0.1617	0.2082	0.2804	0.1805	0.2168	0.8919	0.8469	0.8644	0.6489	0.5919	0.6301
	trocr	0.2538	0.1636	0.2125	0.2064	0.1298	0.1589	0.9037	0.8386	0.8760	0.4385	0.3421	0.4103
Semantic chunking	easyocr	0.1050	0.0593	0.0845	0.1037	0.0572	0.0757	0.7392	0.6517	0.6869	0.4437	0.3241	0.4078
	tesseract	0.0909	0.0516	0.0739	0.1014	0.0556	0.0731	0.7573	0.6584	0.7057	0.2785	0.2038	0.2502

Appendix

Table 13: OCR performance by Chunking Method (top-

k=1\sim 4

average)

Chunking Method	DP Model	DUDE			MPVQA			CUAD			MOAMOB
		R	P	nDCG	R	P	nDCG	R	P	nDCG	R	P	nDCG
Length chunking	detr	0.2918	0.2007	0.2507	0.2588	0.1680	0.2023	0.9097	0.8800	0.8934	0.6611	0.6078	0.6345
	dit	0.2852	0.1836	0.2370	0.2835	0.1852	0.2250	0.9047	0.8450	0.8804	0.6278	0.5365	0.6030
	vgt	0.2115	0.1214	0.1620	0.2145	0.1227	0.1526	0.8888	0.8361	0.8590	0.6496	0.5585	0.6252
Semantic chunking	detr	0.1298	0.0784	0.1069	0.1109	0.0605	0.0790	0.8621	0.7873	0.8226	0.3274	0.2374	0.2903
	dit	0.1037	0.0591	0.0845	0.1123	0.0647	0.0833	0.7373	0.6244	0.6746	0.2933	0.2241	0.2667

Appendix

Table 14: DP performance by Chunking Method (top-

k=1\sim 4

average)

Chunking Method	Embedding	DUDE			MPVQA			CUAD			MOAMOB
Chunking Method	Embedding	R	P	nDCG	R	P	nDCG	R	P	nDCG	R	P	nDCG
Length chunking	bge	0.2655	0.1646	0.2097	0.2810	0.1742	0.2118	0.9152	0.8778	0.8960	0.6504	0.5677	0.6162
	e5	0.2644	0.1602	0.2178	0.2469	0.1485	0.1855	0.8915	0.8491	0.8651	0.6467	0.5672	0.6175
	BM25	0.2586	0.1810	0.2222	0.2289	0.1534	0.1827	0.8965	0.8342	0.8717	0.6415	0.5679	0.6290
Semantic chunking	bge	0.0753	0.0433	0.0619	0.0876	0.0511	0.0653	0.8663	0.8046	0.8361	0.2956	0.2309	0.2824
	e5	0.1184	0.0665	0.0955	0.1152	0.0650	0.0839	0.8380	0.7628	0.8035	0.4126	0.2966	0.3681

Appendix

Table 15: Performance comparison by Embedding and Chunking Method (top-

k=1\sim 4

average)

		MPVQA			DUDE			CUAD			MOAMOB
Top-k	Chunking Method	ANLS	ROUGE-L	METEOR	ANLS	ROUGE-L	METEOR	ANLS	ROUGE-L	METEOR	ANLS	ROUGE-L	METEOR
1	Length chunking	0.1299	0.0679	0.0859	0.1573	0.1281	0.1722	0.2675	0.1761	0.1616	0.2498	0.0739	0.1054
	Semantic chunking	0.1256	0.0635	0.0794	0.1527	0.1126	0.1420	0.2611	0.1544	0.1409	0.2444	0.0737	0.1009
	LumberChunker	0.1394	0.0737	0.0873	0.1614	0.1264	0.1642	0.2690	0.1698	0.1657	0.2588	0.0748	0.1189
	Perplexity chunking	0.1254	0.0544	0.0676	0.1683	0.1357	0.1689	0.2592	0.1629	0.1457	0.2530	0.0868	0.1182
	Structure based Chunking	0.1440	0.0858	0.1103	0.1615	0.1217	0.1527	0.2489	0.1569	0.1592	0.2477	0.0892	0.1108

Appendix

Table 16: Average generation performance ( ANLS , ROUGE-L , METEOR ) of six chunking strategies on MPVQA, DUDE, CUAD, and MOAMOB datasets, separated by top-

k

settings (

k=1

and

k=4

). Best scores for each metric and dataset are highlighted in bold .

Document Chunking

Document chunking splits long or structured documents into coherent units for retrieval, question answering, and other language-model tasks.

See all 53 papers

M3DocDep: Multi-modal, Multi-page, Multi-document Dependency Chunking with Large Vision-Language Models
Apr 17, 2026Joongmin Shin, Jeongbae Park, Jaehyung Seo +1Multimodal RAG Document Chunking
Human-inspired AI Research, Korea University · Computer Science and Engineering, Konkuk University · Department of Computer Science and Engineering, Korea University
HiKEY: Hierarchical Multimodal Retrieval for Open-Domain Document Question Answering
May 28, 2026Joongmin Shin, Gyuho Shim, Jeongbae Park +2Multimodal RAG Multimodal IR
Human-inspired AI Research, Korea University · Department of Computer Science and Engineering, Korea University · Computer Science and Engineering, Konkuk University
MM-BizRAG: Rethinking Multimodal Retrieval-Augmented Generation for General Purpose Enterprise Q&A
Jun 2, 2026Hanoz Bhathena, Parin Rajesh Jhaveri, Rohan Mittal +7Multimodal RAG Multimodal QA
JPMorgan Chase & Co. · Georgia Institute of Technology

Apr 17, 2026J/K move · Enter open · S save

M3DocDep: Multi-modal, Multi-page, Multi-document Dependency Chunking with Large Vision-Language Models

Joongmin Shin, Jeongbae Park, Jaehyung Seo, Heuiseok Lim

Human-inspired AI Research, Korea University · Computer Science and Engineering, Konkuk University · Department of Computer Science and Engineering, Korea University

Multimodal RAG Document Chunking Multimodal Large Language Models Document Parsing Dependency Parsing

Open paper AskPDF arXiv

In long, multi-page industrial documents, retrieval-augmented generation (RAG) depends heavily on whether chunk boundaries follow the document's true structure. Existing text-centric chunkers and generative hierarchy parsers often miss cross-page parent-child relations, figure/table-caption bindings, and boundary cues, which leads to fragmented or redundant chunks and degrades both retrieval and answer quality. We propose M3DocDep, an LVLM-based pipeline that first recovers block-level dependencies and then constructs chunks along the recovered document tree. The pipeline uses SharedDet as a common DP+OCR preprocessing layer, extracts multimodal block embeddings with boundary-aware SoftROI pooling, scores candidate parent-child edges with a biaffine head, decodes a globally valid dependency tree with MST constraints, and builds tree-guided chunks annotated with section paths and page ranges. Under a shared-block evaluation protocol, M3DocDep improves STEDS by +28.5 to +39.6 percent on DHP benchmarks, retrieval nDCG by +1.1 to +15.3 percent, and QA ANLS by +4.5 to +15.3 percent on corpus-level RAG benchmarks. These results show that recovering document dependencies before chunking yields more coherent retrieval units for long, multi-page multimodal documents.

Abstract

Document Chunking

Explore similar work

M3DocDep: Multi-modal, Multi-page, Multi-document Dependency Chunking with Large Vision-Language Models

HiKEY: Hierarchical Multimodal Retrieval for Open-Domain Document Question Answering

MM-BizRAG: Rethinking Multimodal Retrieval-Augmented Generation for General Purpose Enterprise Q&A