Locally Sound, Globally Insufficient: The Local-Global Gap in Multi-Hop Reasoning
Authors: Bohao Chu, Hendrik Damm, Qianli Wang, Hui Wang, Shuning Zhang, Christoph M. Friedrich, Norbert Fuhr
Organizations: University of Duisburg-Essen · University of Applied Sciences and Arts Dortmund · Institute for Medical Informatics, Biometry and Epidemiology (IMIBE) · Technische Universität Berlin · Tsinghua University
Reliable multi-hop reasoning requires more than locally supported steps: a trace can be sound at every reasoning step yet still fail to answer the question as a whole. We call this failure regime the local-global gap (LGG), in which the trace is locally sound yet globally insufficient. Local soundness requires each step to be supported by the available evidence and preceding steps, whereas global sufficiency requires the reasoning trace to align with the question and establish the submitted answer. In a human-adjudicated diagnostic of 2,598 responses across three multi-hop QA benchmarks and three models, we find that the LGG occurs in every benchmark-model combination and accounts for nearly half of globally insufficient responses overall. However, conventional faithfulness verifiers that check claims against the evidence largely miss these failures: at thresholds retaining at least 95% of reliable traces, recall for LGG cases is substantially lower than that for locally unsound traces. To address these failures, we formalize three dependencies for reliable reasoning: evidence-to-step support, question-to-trace alignment, and trace-to-answer closure. Instead of post-hoc diagnosis, we introduce E-Closure to supervise these dependencies during training, combining generation supervision on supported original and counterfactual responses with bidirectional switching constraints. Averaged over three benchmarks and three backbones, existing fine-tuning baselines improve accuracy and local soundness over the base models, but at the cost of global sufficiency. E-Closure improves both: among all fine-tuned methods, it achieves the highest average accuracy (92.8%) and trace reliability (89.0%) while yielding the lowest LGG rate (6.2%).
Figures & tables
Figure 1: Local and global reasoning diagnostics. Bars show percentages for each dataset–model pair. The bottom row pools all three datasets, with 866 responses per model. Pooled LggShare and LggRisk are computed using their respective conditional denominators. The main text reports results pooled over all 2,598 responses, with the full joint distribution provided in Appendix A.4 .
Dq→(r,a)
Dq→r
q→r
r→a
RAGAS
VeriScore
AlignScore
Alignment
Closure
Trace group
AUC
R@95
AUC
R@95
AUC
R@95
AUC
R@95
AUC
R@95
LGG ( n=145 )
0.679
11.0
0.635
9.7
0.469
6.2
0.787
42.1
0.816
42.8
Path deviation ( n=51 )
0.715
9.8
0.644
11.8
0.458
3.9
0.921
66.7
0.694
13.7
Incomplete reasoning ( n=25 )
0.680
20.0
0.593
4.0
0.448
0.0
0.844
60.0
0.885
64.0
Answer decoupling ( n=69 )
0.651
8.7
0.643
10.1
0.486
10.1
0.667
17.4
0.881
56.5
Table 1: Detector sensitivity across failure types. Each failure group is compared with the same 1,914 reliable traces. AUC denotes AUROC ( 0.5 : random ranking; 1.0 : perfect separation). R@95 is failure recall (%) at the highest empirical threshold that flags at most 5% of reliable traces. Higher values are better for both metrics. For evidence-support detectors, wavy and straight underlines mark recall on LGG and locally unsound responses, respectively. Bold highlights alignment’s sensitivity to path deviation and closure’s sensitivity to answer decoupling. Details appear in Appendix A.5 .
Figure 2: Edge Closure Learning objective. (a) Three generation anchors train the original and counterfactual responses under their supported inputs. (b) Six directional margins require preferences over the affected step, trace, or answer to reverse as the conditioning input switches between paired views. The S , T , and C panels instantiate Dq→st(e) , q→r , and r→a , respectively.
Figure 3: Aggregate performance. Bars show equal-weight means across nine dataset–backbone settings, with adapted methods first averaged over three seeds within each setting. Backbone uses fixed diagnostic generations. Error bars show 95% hierarchical bootstrap confidence intervals with paired resampling across methods. All metrics are percentages.
Table 2: Performance by dataset and backbone. Adapted methods report means over three seeds. All metrics are percentages: Acc. ↑ , LSR ↑ , GSR ↑ , TRR ↑ , and LGG ↓ . Bold and underlining mark the best and second-best displayed values, respectively, for each metric within each dataset–backbone setting, including ties. RM denotes Response-margin. Per-seed results appear in Appendix D.5.4 .
Multi-hop QA benchmarks often reward Large Language Models (LLMs) for spurious correctness, where models reach correct answers through invalid intermediate reasoning. We propose SAFE, an LLM-as-verifier framework for evidence-grounded multi-hop QA. Rather than judging only the final answer after generation, SAFE verifies reasoning during generation by checking intermediate steps against the provided passages and previous reasoning trajectory. To make this process checkable, SAFE decomposes reasoning into atomic, evidence-grounded units represented with Knowledge Graph (KG) triples. At train-time, SAFE verifies benchmark supervision under KG-grounded constraints and constructs reliable verifier training data. At inference-time, an external verifier checks each generated step, identifies invalid reasoning, and provides correction feedback before errors propagate. Across three multi-hop QA benchmarks, SAFE improves accuracy by 8.8 pp on average. These results show that evidence-grounded multi-hop QA benefits from shifting LLM-based evaluation from post-hoc answer judgment to stepwise reasoning verification.
Multi-hop question answering requires combining information from multiple documents to answer complex questions. These systems have grown increasingly capable, yet when they fail, the error is typically attributed to not finding the right documents. Whether this holds at the level of individual reasoning steps remains largely unexamined. We investigate this across three standard multi-hop QA benchmarks and find that failures decompose into two distinct modes: retrieval failures, where the needed passage was not retrieved, and extraction failures, where the passage was retrieved but the needed fact could not be extracted - a phenomenon we term the fact-grounding gap. Extraction failures account for nearly half of all per-hop deficiencies and are invisible to standard retrieval metrics. They remain unresolved by every retrieval intervention we test, establishing a ceiling for retrieval-only improvements. The gap's severity varies across benchmarks and question types, but extraction failures appear on every dataset we measure. Our findings reveal that retrieval failures and extraction failures are fundamentally different bottlenecks requiring different solutions - a distinction absent from current evaluation practice.
When copies of the same language model are prompted to debate, they produce diverse phrasings of one perspective rather than diverse perspectives. Multi-agent debate (MAD), and more broadly closed-system reasoning where agents iteratively transform each other's outputs, tends to preserve answer accuracy while degrading the reasoning behind those answers. We name the multi-agent case the Debate Trap and the broader phenomenon the Reasoning Trap, offering a programmatic theory of evidence-grounded reasoning failure.The framework has three parts: (i) SFS (Supported Faithfulness Score), a claim-level metric verifying decomposed atomic claims against provided evidence (decomposer-invariant rankings: Spearman rho=1.0); (ii) EGSR (Evidence-Grounded Socratic Reasoning), replacing adversarial argumentation with evidence-grounded inquiry; (iii) Theorem 1 (DPI Bound): under standard MAD, the chain E -> O^0 -> O^1 -> ... is Markov, and the Data Processing Inequality implies E[I(E;O^{t+1})] <= E[I(E;O^t)]. Three companion results -- open-system recovery (Theorem 2), EGSR accumulation (Lemma 2), and vote-aggregation floor (Proposition 1) -- partition multi-step LLM reasoning by its information-theoretic relationship to E. Across 16 conditions on SciFact (300 claims) and FEVER (1,000 claims), DebateCV (C13) preserves 88% of baseline accuracy while SFS drops 43%; majority-vote MAD (C15) reduces SFS to 1.7% of baseline (p < 10^{-6}, d = -0.96); EGSR recovers 98%. An R6 cohort study (Korean n=10x30 FEVER; English n=3x200 SciFact) finds inter-rater Fleiss kappa <= +0.018 with 0.8-1.4 Likert intra-rater shifts across language and domain -- the human agreement that faithfulness metrics have been calibrated against is not itself stable. We offer one falsifiable conjecture: any closed-system reasoning protocol preserving Theorem 1's Markov structure is, in expectation, subject to the same DPI bound.