cs.AIOct 30, 2026

ReX-GG: a LLM Ensemble Pipeline for Relation-extraction and Graph Generation

Authors: Giacomo Magnifico, Eduard Barbu

Organizations: Institute of Computer Science, University of Tartu, Tartu, Estonia

Abstract

Current LLM ensemble frameworks focus on multi-step setups with additional modules for answer ranking, often opting for token and span analysis rather than structured outputs, leading to heavyweight architectures with potential fail states along the pipeline. Faster, lighter solutions are more vulnerable to hallucination propagation and can lack output control in more complex pipelines. This paper proposes a customisable, lightweight ensemble workflow of coordinated Large Language Models that leverages JSON-structured outputs and anonymous peer-review ranking to mitigate hallucinatory outputs and single-model failure points. The pipeline is demonstrated on a relation extraction task applied to popular science articles in English, targeting four ontologically-grounded relation types (strong causation, weak causation, contrastive, and compositional), with semantic node canonicalisation and interactive, colour-coded HTML causal graphs as the final output. Performance is evaluated through an anonymous user study, achieving an average perceived accuracy of 0.778 against a human-annotated gold standard. The modular architecture supports flexible deployment across both API-based and in-house LLM setups, and the full framework is released under an open license to foster reproducibility and collaborative research.

Figures & tables

Explore similar work

Apr 9, 2026

LLMs Underperform Graph-Based Parsers on Supervised Relation Extraction for Complex Graphs

Relation extraction represents a fundamental component in the process of creating knowledge graphs, among other applications. Large language models (LLMs) have been adopted as a promising tool for relation extraction, both in supervised and in-context learning settings. However, in this work we show that their performance still lags behind much smaller architectures when the linguistic graph underlying a text has great complexity. To demonstrate this, we evaluate four LLMs against a graph-based parser on six relation extraction datasets with sentence graphs of varying sizes and complexities. Our results show that the graph-based parser increasingly outperforms the LLMs, as the number of relations in the input documents increases. This makes the much lighter graph-based parser a superior choice in the presence of complex linguistic graphs.
Jul 20, 2026

PEARL: Auditable Repair for Scientific Reasoning Graph Extraction

Scientific Reasoning Graph Extraction (SRGE) aims to recover explicit links among observations, evidence, intermediate claims, and paper-level conclusions. LLMs can produce graph-like scientific explanations, but their outputs often mix malformed syntax, drifting edge labels, incorrectly oriented roots, and weak source anchors. We propose PEARL (Peircean Extraction via Abstraction and Repair Layer), a training-free framework that turns noisy LLM graph responses into auditable reasoning graphs and repairs them toward strict semantic validity. PEARL first materializes explicit graph content under a closed Peircean schema, then uses matched evidence-grounded judge feedback to repair rejected edge types, local inference steps, and terminal roots while preserving an audit trail. On five 70-paper model archives from ARCHE, a benchmark for latent reasoning-chain extraction, PEARL raises strict gate passes from 0/350 for the LLM baseline to 300/350, with average REA improving from 0.339 to 0.906. The graphs provide a reliability layer for research-agent and AI scientist workflows that need inspectable reasoning traces rather than unconstrained graph regeneration. Code and audit artifacts are available at https://github.com/BohanSu/auditable-repair-reasoning-graphs/tree/300-350_workshop .
Jul 30, 2026

Reasoning Consensus: Structural Ensembling of LLM Reasoning via Weighted DAG Aggregation

Large Language Models (LLMs) reason in free-form chain-of-thought (CoT) without revealing which steps were well supported and which alternatives were rejected. In high-stakes domains, users then cannot judge if an answer rests on sound reasoning. Majority voting and multi-agent debate improve answers using model agreement, but do not expose agreement on individual steps. Reasoning Consensus (RC) extracts a typed reasoning graph from each CoT, merges equivalent steps across graphs from diverse models, and weights every step by how many chains attest to it. RC reveals this step-wise support and the varied justifications behind each answer. On 7 benchmarks across law, science, medicine, narrative reasoning and logic, RC matches majority voting and debate in accuracy (69.4% vs. 69.0-70.3%). An LLM judge prefers RC's best-supported graph to other graphs for the same answer. In a user study, participants read RC graphs faster than debate transcripts or vote tallies, and 14 of 18 preferred them for future use.