While Large Language Models (LLMs) excel at generating fluent text, they often struggle with multi-step logical reasoning with inferential connections among statements. We introduce KUDEX (Knowledge-Upward-Downward Reasoning Extension), a novel zero-shot, decompositional framework for analyzing and verifying logical relationships in natural language. Unlike traditional approaches that rely on surface-level entailment checks, KUDEX leverages LLMs to decompose reasoning into atomic primitives and constructs transparent reasoning graphs that serve both as execution traces and audit logs. Our framework includes a general prompt adaptation pipeline that adapts to diverse reasoning tasks without additional fine-tuning or domain-specific pipelines. Through comprehensive experiments on benchmarks across four domains, we demonstrate that KUDEX provides superior performance compared to existing methods, offering a more granular and robust framework for assessing logical relationships between statements.
Figures & tables
Figure 1: An example of two-pass annotation process of KUDEX-Core.
Type
What It Means
How to Mark It
Implied U
A D-unit depends on a generalization that is not stated but clearly required.
Add a parenthetical marker: (implied: [U-...])
Delayed U
The U (or K) a D-unit depends on appears later in the text.
Add a marker: (delayed-U): See St , where St is the sentence appeas later
Table 1: Handling Special U-Type Cases
Figure 2: An overview of the prompt adaptation pipeline with HITL, which extends KUDEX-Core to various reasoning tasks, e.g., reasoning QA [ 20 ] , cohesion evaluation [ 23 ] , etc.
Method
Accuracy
LLM
0.7462
LRM
0.7615
KUDEX-CF
0.8268
Table 2: Citation Faithfulness Results
Dimension
LLM
LRM
KUDEX-CoH
Coherence
0.6615
0.6719
0.7167
Cohesion
0.4889
0.5474
0.5600
Consistency
0.4634
0.5195
0.5432
Relevance
0.4444
0.5000
0.4000
Table 3: F1 Score comparison of different reasoning methods on Cohesentia benchmark.
Metric
LLM
LRM
KUDEX-PF
Accuracy
0.6820
0.6770
0.7010
F1 Score
0.6750
0.6620
0.7010
Table 4: Accuracy and F1 Score comparison of different reasoning methods on LogiQA2.0 dataset.
Metric
SmartyPat-Bench
SmartyPatBench-Augmented
LLM
LRM
KUDEX-Fallacy
LLM
LRM
KUDEX-Fallacy
Macro avg F1
0.3265
0.2624
0.3683
0.6248
0.5529
0.6524
Weighted avg F1
0.5645
0.5204
0.5799
Table 5: F1 Score comparison across different reasoning methods on fallacy detection datasets.
Figure 3: Disagreement patterns between human annotations and different methods across four benchmarks. The bars show the distribution of labeling agreement and disagreements: agreement cases , instances where humans assign labels that all methods miss , situations where all methods converge on labels not assigned by humans and cases where both disagreement types occur simultaneously (stripped). For LogiQA 2.0’s binary classification, all disagreements fall into the overlapping category. While for multi-label tasks (Cohesentia and SmartyPat-Bench), cases where all three models converge on labels not assigned by humans outnumber those where humans assign labels missed by all models, suggesting that human annotators often overlook valid categories. This pattern is less pronounced in the Citation Faithfulness dataset, which involves single-label prediction.
Appendix figures & tables27 assets
Supplementary material from the paper’s appendix.
Appendix
Operation
Definition
Suffix
Example
K : Knowledge
States a fact, belief, perception, definition, or intention
Large Language Models (LLMs) still struggle with multi-step logical reasoning. Existing approaches either purely refine the reasoning chain in natural language form or attach a symbolic solver as an external module. In this work, we instead ask whether LLMs contain a shared internal logical subspace that simultaneously aligns natural-language and symbolic-language views of the reasoning process. Our hypothesis is that this logical subspace captures logical reasoning capabilities in LLMs that are shared across views while remaining independent of surface forms. To verify this, we employ Canonical Correlation Analysis on the paired residual activations from natural-language and symbolic-language reasoning chains, learning a low-dimensional subspace with maximum cross-view correlation. Furthermore, we design a training-free approach that steers LLMs reasoning chain along this logical subspace, thereby leveraging the complementary reasoning signals from both views. Experiments on four logical reasoning benchmarks demonstrate the effectiveness of our approach, improving accuracy by up to 11 percentage points and generalizing well on out-of-domain problems.
Feihao Fang, My T. Thai, Yuanyuan Lei
University of Illinois Urbana-Champaign, Champaign, IL · Computer & Information Science and Engineering, University of Florida, Gainesville, FL
Large language models (LLMs) have demonstrated impressive performance in various natural language processing tasks, yet their ability to perform multi-step logical reasoning remains an open challenge. Although Chain-of-Thought prompting has improved logical reasoning by enabling models to generate intermediate steps, it lacks mechanisms to assess the coherence of these logical transitions. In this paper, we propose a novel, lightweight evaluation strategy for logical reasoning that uses query-key alignments inside transformer attention heads. By computing a single forward pass and extracting a "QK-score" from carefully chosen heads, our method reveals latent representations that reliably separate valid from invalid inferences, offering a scalable alternative to traditional ablation-based techniques. We also provide an empirical validation on multiple logical reasoning benchmarks, demonstrating improved robustness of our evaluation method against distractors and increased reasoning depth. The experiments were conducted on a diverse set of models, ranging from 1.5B to 70B parameters.
Eduard Tulchinskii, Anastasia Voznyuk, Laida Kushnareva +4
Skolkovo Institute of Science and Technology · AI Foundation and Algorithm Lab · Moscow Institute of Physics and Technology +2
Large Language Models (LLMs) have demonstrated remarkable proficiency in natural language understanding, yet they struggle with strict multi-step reasoning, frequently suffering from hallucinations and inconsistency. Existing solutions like Chain-of-Thought (CoT) lack rigorous verification mechanisms, while standard Retrieval-Augmented Generation (RAG) often misses the complex, structural dependencies inherent in logical tasks. To bridge this gap, we propose a Neuro-Symbolic architecture that integrates a Logical Knowledge Graph (LKG) with dynamic solver routing. Specifically, we introduce an ontology-based LKG that treats logical rules and constraints as first-class topological nodes, enabling explicit modeling of dependencies extracted from text. We further design a Logic Router to dynamically dispatch tasks to the optimal symbolic engine, which is supported by a topology-aware hybrid retrieval mechanism. Experimental results on logical reasoning benchmarks demonstrate that our framework significantly outperforms state-of-the-art prompting and RAG baselines, delivering higher accuracy and verifiable reasoning paths.