While Large Language Models (LLMs) excel at generating fluent text, they often struggle with multi-step logical reasoning with inferential connections among statements. We introduce KUDEX (Knowledge-Upward-Downward Reasoning Extension), a novel zero-shot, decompositional framework for analyzing and verifying logical relationships in natural language. Unlike traditional approaches that rely on surface-level entailment checks, KUDEX leverages LLMs to decompose reasoning into atomic primitives and constructs transparent reasoning graphs that serve both as execution traces and audit logs. Our framework includes a general prompt adaptation pipeline that adapts to diverse reasoning tasks without additional fine-tuning or domain-specific pipelines. Through comprehensive experiments on benchmarks across four domains, we demonstrate that KUDEX provides superior performance compared to existing methods, offering a more granular and robust framework for assessing logical relationships between statements.
Figures & tables
Figure 1: An example of two-pass annotation process of KUDEX-Core.
Type
What It Means
How to Mark It
Implied U
A D-unit depends on a generalization that is not stated but clearly required.
Add a parenthetical marker: (implied: [U-...])
Delayed U
The U (or K) a D-unit depends on appears later in the text.
Add a marker: (delayed-U): See St , where St is the sentence appeas later
Table 1: Handling Special U-Type Cases
Figure 2: An overview of the prompt adaptation pipeline with HITL, which extends KUDEX-Core to various reasoning tasks, e.g., reasoning QA [ 20 ] , cohesion evaluation [ 23 ] , etc.
Method
Accuracy
LLM
0.7462
LRM
0.7615
KUDEX-CF
0.8268
Table 2: Citation Faithfulness Results
Dimension
LLM
LRM
KUDEX-CoH
Coherence
0.6615
0.6719
0.7167
Cohesion
0.4889
0.5474
0.5600
Consistency
0.4634
0.5195
0.5432
Relevance
0.4444
0.5000
0.4000
Table 3: F1 Score comparison of different reasoning methods on Cohesentia benchmark.
Metric
LLM
LRM
KUDEX-PF
Accuracy
0.6820
0.6770
0.7010
F1 Score
0.6750
0.6620
0.7010
Table 4: Accuracy and F1 Score comparison of different reasoning methods on LogiQA2.0 dataset.
Metric
SmartyPat-Bench
SmartyPatBench-Augmented
LLM
LRM
KUDEX-Fallacy
LLM
LRM
KUDEX-Fallacy
Macro avg F1
0.3265
0.2624
0.3683
0.6248
0.5529
0.6524
Weighted avg F1
0.5645
0.5204
0.5799
Table 5: F1 Score comparison across different reasoning methods on fallacy detection datasets.
Figure 3: Disagreement patterns between human annotations and different methods across four benchmarks. The bars show the distribution of labeling agreement and disagreements: agreement cases , instances where humans assign labels that all methods miss , situations where all methods converge on labels not assigned by humans and cases where both disagreement types occur simultaneously (stripped). For LogiQA 2.0’s binary classification, all disagreements fall into the overlapping category. While for multi-label tasks (Cohesentia and SmartyPat-Bench), cases where all three models converge on labels not assigned by humans outnumber those where humans assign labels missed by all models, suggesting that human annotators often overlook valid categories. This pattern is less pronounced in the Citation Faithfulness dataset, which involves single-label prediction.
Appendix figures & tables27 assets
Supplementary material from the paper’s appendix.
Appendix
Operation
Definition
Suffix
Example
K : Knowledge
States a fact, belief, perception, definition, or intention