Balancing Reasoning and Hardware Constraints in RAG Pipelines for Ukrainian Multi-Domain Document Understanding
Authors: Illya Havrylov
Organizations: National Technical University of Ukraine “Igor Sikorsky Kyiv Polytechnic Institute” Educational and Scientific Institute for Applied System Analysis (IASA) Department of Artificial Intelligence
This paper describes the system submitted to the UNLP 2026 Shared Task on Multi-Domain Document Understanding. The challenge required extracting precise answers, document IDs, and page numbers from a diverse corpus of Ukrainian PDF documents within a strict 9-hour offline Kaggle execution limit. During evaluation on the hidden private test set, optical character recognition (OCR) of scanned documents emerged as a severe bottleneck, consuming 5-7 hours of the total time budget due to sequential single-threaded execution. This overhead strictly limited the remaining time for Large Language Model (LLM) inference to approximately two hours for 500 questions. To guarantee pipeline completion without timeouts, we developed a resource-efficient Hybrid Retrieval-Augmented Generation (RAG) pipeline utilizing BM25, BGE-M3, and Cross-Encoder reranking. Rather than deploying parameter-heavy reasoning models (e.g., DeepSeek R1) which consistently timed out, we utilized a 4-bit quantized LapaLLM 12B model via llama.cpp on dual NVIDIA T4 GPUs. Prioritizing pipeline stability over multi-step reasoning, our system achieved a Private Score of 0.8095, placing 10th out of 15 active teams.
Figures & tables
Configuration
Public
Private
Status
MamayLM 7B (Top-5)
0.6918
0.7374
Completed
LapaLLM 12B Base (Top-5)
0.7741
0.7990
Completed
LapaLLM 12B Exp. (Top-7)
0.7831
0.8095
Completed
LapaLLM 12B Self-Consistency
Timeout
Timeout
Failed
DeepSeek R1 Distill 14B
Timeout
Timeout
Failed
Table 1: System performance on the UNLP 2026 leaderboard. Approaches exceeding the 9-hour limit are marked as Timeout.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Component / Parameter
Value
Chunking & Parsing
Chunk Word Count
250 words
Chunk Overlap
50 words
PDF Parsing Engine
pymupdf4llm (Markdown)
OCR Fallback
Tesseract (Single-threaded)
Hybrid Retrieval
Appendix
Table 2: Hyperparameter configurations extracted from the final submission pipeline.
We participated in the Fifth UNLP shared task on multi-domain document understanding, where systems must answer Ukrainian multiple-choice questions from PDF collections and localize the supporting document and page. We propose a retrieval-augmented pipeline built around three ideas: contextual chunking of PDFs, question-aware dense retrieval and reranking conditioned on both the question and answer options, and constrained answer generation from a small set of reranked passages. Our final system uses Qwen3-Embedding-8B for retrieval, a fine-tuned Qwen3-Reranker-8B for passage ranking, and Qwen3-32B for answer selection. On a held-out split, reranking improves Recall@1 from 0.6957 to 0.7935, while using the top-2 reranked passages raises answer accuracy from 0.9348 to 0.9674. Our best leaderboard run reached 0.9452 on the public leaderboard and 0.9598 on the private leaderboard. Our results suggest that, under strict code-competition constraints, preserving document structure and making relevance estimation aware of the answer space are more effective than adding complex downstream heuristics.
Anton Bazdyrev, Ivan Bashtovyi, Ivan Havlytskyi +2
National Technical University of Ukraine "Igor Sikorsky Kyiv Polytechnic Institute"
This paper presents a highly efficient Retrieval-Augmented Generation (RAG) system built specifically for Ukrainian document question answering, which achieved 2nd place in the UNLP 2026 Shared Task. Our solution features a custom two-stage search pipeline that retrieves relevant document pages, paired with a specialized Ukrainian language model fine-tuned on synthetic data to generate accurate, grounded answers. Finally, we compress the model for lightweight deployment. Evaluated under strict computational limits, our architecture demonstrates that high-quality, verifiable AI question answering can be achieved locally on resource-constrained hardware without sacrificing accuracy.
Mykola Trokhymovych, Yana Oliinyk, Nazarii Nyzhnyk
We present an initial investigation into Agentic Retrieval-Augmented Generation (RAG) for Ukrainian, conducted within the UNLP 2026 Shared Task on Multi-Domain Document Understanding. Our system combines two-stage retrieval (BGE-M3 with BGE reranking) with a lightweight agentic layer performing query rephrasing and answer-retry loops on top of Qwen2.5-3B-Instruct. Our analysis reveals that retrieval quality is the primary bottleneck: agentic retry mechanisms improve answer accuracy but the overall score remains constrained by document and page identification. We discuss practical limitations of offline agentic pipelines and outline directions for combining stronger retrieval with more advanced agentic reasoning for Ukrainian.