Meet, Compare, or Abstain: LatWeave for Deterministic Multi-Hop Question Answering on Knowledge Lattices

Yuze Ren; Shaoheng Fan; Tao Wang; Yabo Yan; Han Han

cs.CLSep 23, 2026

Meet, Compare, or Abstain: LatWeave for Deterministic Multi-Hop Question Answering on Knowledge Lattices

Authors: Yuze Ren, Shaoheng Fan, Tao Wang, Yabo Yan, Han Han

Organizations: ZenSmart Technology (Beijing) Co., Ltd., China

Abstract

Probabilistic question-answering systems -- whether large language models (LLMs) themselves, retrieval-augmented generation (RAG), or trained multi-hop retrievers -- conflate "what is known" and "how to reason" into a single probabilistic computation: hallucination cannot be eradicated, evidence chains cannot be audited, and the system answers even when it does not know. We present LatWeave, which organizes knowledge into a multidimensional knowledge lattice and compiles multi-hop QA into three deterministic operators -- meet (constraint intersection), compare (lattice-order comparison), and abstain (structural abstention); LLMs appear only on the construction side (one-shot extraction) and the query-planning side, while the answer-generation path is zero-LLM, zero-task-training, and auditable end to end -- so that question answering over Web-published knowledge becomes reproducible item by item. Rather than claiming across-the-board SOTA, we characterize the operating envelope of this paradigm on six public benchmarks: when knowledge is complete (MetaQA, 39,093 questions) meet chains are near-lossless over three hops (any-hit 0.9975, on par with fully supervised KBQA); on templated multi-hop home ground (2WikiMultihopQA held-out n=1,258) EM 0.865, well above published structure-augmented RAG reproductions; on open-text deep composition (MuSiQue) and extraction-coverage gaps (HotpotQA) we report degradation honestly and attribute it to causes outside the lattice-algebra layer; and when information is incomplete (IIRC) we achieve structural abstention with abstain accuracy 0.971 and leak rate 0.029. Within the operating envelope, deterministic execution pays no performance penalty, and every step on the answer path can be recomputed -- precisely the source of end-to-end auditability.

Figures & tables

Figure 1 . LatWeave in two stages. Build time (offline, one-off): an LLM extracts typed predicates from question-blind documents (dedup, normalization and a dimension whitelist absorb extraction uncertainty before it can enter the lattice), yielding a product lattice whose instances each carry their source_text . Query time (online, per question): an LLM planner compiles the question into a typed plan, and a deterministic core executes the three operators — meet , compare , abstain . The LLM is confined to the two orange boxes (extraction and planning); the answer path carries zero LLM calls and zero task training, so every answer is an auditable meet chain.

Figure 2 . A product-lattice slice (species

\times

habitat mini-example): (a) value hierarchies of the two dimensions; (b) an instance as a point in the product lattice; (c) one meet in three deterministic steps—a hit requires the descendant check on every dimension, and a conflict on any dimension entails abstain.

Dataset	Size used	Structure	Role
MetaQA ( Zhang et al., 2018 )	test $n=39{,}093$	1/2/3-hop: 9,947 / 14,872 / 14,274	Control: complete knowledge $\Rightarrow$ lattice algebra lossless
2WikiMultihopQA ( Ho et al., 2020 )	held-out $n=1{,}258$	4 question types	Main experiment: templated multi-hop, end-to-end
HotpotQA ( Yang et al., 2018 )	held-out $n=741$	bridge 592 / comparison 149	Boundary 1: fact-coverage gap
MuSiQue ( Trivedi et al., 2022 )	held-out $n=242$	2/3/4-hop: 125/76/41	Boundary 2: longer-chain degradation
IIRC ( Ferguson et al., 2020 )	held-out $n=130$	span 59/ value 23/ binary 13/ none 35	Refusal experiment: abstain under incompleteness
FRAMES ( Krishna et al., 2025 )	dev 742 + held-out 82	factuality $\times$ multi-hop	Compounded stress test: coverage $\times$ grounding

Table 1 . Datasets, structure, and role in this paper’s argument.

Method	EM	F1*	Resolved
Keyword baseline	0.083	0.111	73.2%
Naive RAG (TF-IDF + LLM)	0.198	0.205	100%
GraphRAG ( Edge et al., 2024 )	0.514 ‡	0.586 ‡	—
HippoRAG 2 ( Gutiérrez et al., 2025 )	0.650 ‡	0.710 ‡	—
IRCoT ( Trivedi et al., 2023 )	0.53	0.65	—
HopRAG ( Liu et al., 2025 )	0.62	0.69	—

Table 2 . 2WikiMultihopQA main results (held-out

n=1{,}258

Question type	Share	EM	$\Delta$ vs. overall
bridge_comparison	21.9%	0.975	+11.0pp
comparison	24.2%	0.888	+2.3pp
compositional	41.7%	0.815	$-$ 5.0pp
inference	12.3%	0.794	$-$ 7.1pp
Overall	100%	0.865	—

Table 3 . 2WikiMultihopQA per-type breakdown (held-out

n=1{,}258

Hop	$n$	any-hit	recall (complete)	EM (strict)	set-F1
1-hop	9,947	0.9990	0.9987	0.9447	0.9798
2-hop	14,872	0.9983	0.9983	0.6239	0.9448
3-hop	14,274	0.9955	0.9954	0.5304	0.9369
All	39,093	0.9975	0.9973	0.6714	0.9508

Table 4 . MetaQA test main results (

n=39{,}093

, single final run, frozen 9-shot configuration).

Method	1-hop	2-hop	3-hop	Paradigm (trained)
KV-Mem ( Miller et al., 2016 )	0.962	0.827	0.489	Memory networks (yes)
GraftNet ( Sun et al., 2018 )	0.970	0.948	0.777	Subgraph retrieval (yes)
SRN ( Qiu et al., 2020 )	0.970	0.951	0.752	Neural reasoning (yes)
PullNet ( Sun et al., 2019 )	0.970	0.999	0.914	Graph retrieval (yes)
EmbedKGQA ( Saxena et al., 2020 )	0.975	0.988	0.948	KGE (yes)
NSM ( He et al., 2021 )	0.971	0.999	0.989	State machine (yes)

Table 5 . MetaQA test vs. published KBQA methods (supervised rows: Hits@1 verified from source papers, converted to decimals). The Shrestha–Kim row reports Hit Rate—verbatim the any-hit measure, hence directly comparable to our any-hit row; their pooled micro-F1 is not cross-compared with our per-question set-F1 0.9508.

Figure 3 . Hop decay in two contrasts. MetaQA (complete KB, any-hit): lattice meet chains are nearly lossless over three hops (0.9990 / 0.9983 / 0.9955), while the same-corpus RAG baseline (hits@1) decays to 0.364 at 3-hop. MuSiQue (open-text extraction, EM): lattice retrieval collapses to zero on multi-hop text composition (0.048 / 0 / 0), while RAG stays flat at a low 0.12–0.14 (insensitive to hop count).

Level	Mechanism	Grounding object	Status
L0	Free generation	none	Rejected
L1	Global vocabulary	global relation names	Internalized (HotpotQA)
L2	Entity menu	seed entity’s lattice neighborhood	Internalized (FRAMES)
L3	Per-hop replanning	intermediate entities	Not adopted (argued below)

Table 6 . Four levels of query grounding.

Dataset	Knowledge-source form	Main bottleneck	Result
2WikiMultihopQA	Open text extraction, templated questions	—(home ground)	EM 0.865
MetaQA	KB triples mapped directly	Question parsing only	any-hit 0.9975
HotpotQA	Open text extraction, free paragraphs	Coverage gap (90.6%)	EM 0.1687
MuSiQue	Open text extraction, 4-hop chains	Chain deepening (48.3% hop-1 break)	EM 0.0248
FRAMES (dev $n=742$ )	Open text extraction, multi-hop $+$ factuality	Coverage gap $+$ chain-design error	EM 0.0256
IIRC (held-out $n=130$ )	Open text extraction, incomplete information	Answerability boundary	Prec. measure 0.9437

Table 7 . Operating envelope (all numbers are frozen final values).

Metric	Held-out 130	dev_tune 1,171
EM full (Phase-1-hit denominator)	0.0417 (1/24)	0.0264 (6/227)
Recall measure (answerable EM)	0.0105	0.0070
Resolved (Phase-1 hit rate)	25.3% (24/95)	26.4%
Leak rate (none answered)	0.029 (1/35)	0.016 (5/312)
Abstain accuracy	0.971 (34/35)	0.984
Precision measure (abstain $\times$ non-leak)	0.9437	0.9682

Table 8 . IIRC main results (held-out

n=130

; dev_tune as drift check).

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Approach family	Zero-train	Determ.	Audit.	Abstain
Neural multi-hop retrievers	partial 1	$\times$	$\times$	$\times$
RAG / GraphRAG	partial 2	$\times$	eng.-level	$\times$
KBQA / semantic parsing	$\times$ 3	✓	✓	$\times$
Selective prediction	$\times$ 4	—	$\times$	prob. thresh.
LatWeave (this work)	✓	✓	✓	✓

Appendix

Table 9 . Positioning: only LatWeave ticks all four columns. Zero-training is graded because several families mix trained and training-free members (see footnotes). Detection (auditability) is graded because chunk-ID provenance yields engineering-level auditability only: the step from retrieved passage to answer remains unverifiable.

Figure 4 . Query-time flow. The LLM planner compiles the question into a typed plan; deterministic execution on the lattice returns a meet hit, a compare verdict, or abstain ; the answer ships with its evidence text.

Figure 5 . Per-hop conditional meet accuracy

P(\text{correct}\mid\text{chain reaches hop }k)

on the MuSiQue tune pool (

n=2{,}175

), grouped by the question’s true depth.

Configuration	EM	Resolved
Full system	0.873	92.0%
$-$ train gold transfer (dev-only)	0.731	85.3%
$-$ alias normalization	0.853	90.8%
$-$ rule-based re-extraction (E15)	0.843	90.4%

Appendix

Table 10 . 2WikiMultihopQA ablation (dev tune pool, historical protocol

n=11{,}318

$\|A^{*}\|$	$n$	any-hit	recall (complete)	EM (strict)
1	14,834	0.9974	0.9974	0.8332
2	5,729	0.9977	0.9977	0.7413
3–5	7,277	0.9974	0.9968	0.6442
6–10	4,041	0.9968	0.9968	0.5494
$\geq$ 11	7,212	0.9978	0.9976	0.3788
All	39,093	0.9975	0.9973	0.6714

Appendix

Table 11 . MetaQA test bucketed by gold answer-set size (completeness structural analysis).

Dataset	Attribution	Count (share)
HotpotQA	Extraction coverage gap	667/736 (90.6%)
(residual broken chains)	H1 missing / H2 missing / seed no-out-edge	527 / 81 / 59
	Query-layer planner	69/736 (9.4%)
MuSiQue	Break at hop 1	117/242 (48.3%)
(chain-break locus)	Break at hop 2	81/242 (33.5%)
	Break at hop 3 / hop 4	16 / 4 (6.6% / 1.7%)

Appendix

Table 12 . Failure attribution matrix.

Explore similar work

Retrieval is Cheap, Show Me the Code: Executable Multi-Hop Reasoning for Retrieval-Augmented Generation
May 13, 2026Jiashuo Sun, Jimeng Shi, Yixuan Xie +10Multi-Hop Reasoning Reasoning Trajectory
University of Illinois Urbana-Champaign · Hong Kong University of Science and Technology · Texas A&M University +1
ConRAG: Consensus-Driven Multi-View Retrieval for Multi-Hop Question Answering
May 27, 2026Yikai Zhu, Kunfeng Chen, Qihuang Zhong +2Multi-Hop Reasoning Question
School of Computer Science, Wuhan University Wuhan, China
Reducing Hallucinations in Complex Question Answering using Simple Graph-based Retrieval-Augmented Generation (long version)
Jun 4, 2026Christopher J. Wedge, Joshua Stutter, Danny Dixon +1Question-Answering Benchmarks Hallucination Mitigation
National Innovation Centre for Data, Newcastle University

May 13, 2026, cs.AIJ/K move · Enter open · S save

Retrieval is Cheap, Show Me the Code: Executable Multi-Hop Reasoning for Retrieval-Augmented Generation

Jiashuo Sun, Jimeng Shi, Yixuan Xie, Saizhuo Wang+9

University of Illinois Urbana-Champaign · Hong Kong University of Science and Technology · Texas A&M University+1

Multi-Hop Reasoning Reasoning Trajectory

Open paper AskPDF arXiv

Retrieval-Augmented Generation (RAG) has become a standard approach for knowledge-intensive question answering, but existing systems remain brittle on multi-hop questions, where solving the task requires chaining multiple retrieval and reasoning steps. Key challenges are that current methods represent reasoning through free-form natural language, where intermediate states are implicit, retrieval queries can drift from intended entities, and errors are detected by the same model that produces them making self-reflection an unreliable, ungrounded signal. We observe that multi-hop question answering is a typical form of step-by-step computation, and that this structured process aligns closely with how code-specialized language models are trained to operate. Motivated by this, we introduce \pyrag, a framework that reformulates multi-hop RAG as program synthesis and execution. Instead of free-form reasoning trajectories, \pyrag represents the reasoning process as an executable Python program over retrieval and QA tools, exposing intermediate states as variables, producing deterministic feedback through execution, and yielding an inspectable trace of the entire reasoning process. This formulation further enables compiler-grounded self-repair and execution-driven adaptive retrieval without any additional training. Experiments on five QA benchmarks (PopQA, HotpotQA, 2WikiMultihopQA, MuSiQue, and Bamboogle) show that \pyrag consistently outperforms strong baselines under both training-free and RL-trained settings, with especially large gains on compositional multi-hop datasets. Our code, data and models are publicly available at https://github.com/GasolSun36/PyRAG.

Abstract

Explore similar work

Retrieval is Cheap, Show Me the Code: Executable Multi-Hop Reasoning for Retrieval-Augmented Generation

ConRAG: Consensus-Driven Multi-View Retrieval for Multi-Hop Question Answering

Reducing Hallucinations in Complex Question Answering using Simple Graph-based Retrieval-Augmented Generation (long version)