Overcoming Challenges of Interpretive Structural Modeling with Large Language Models

Everett Rush; David J. Icove; Ari Kim; Byung H. Park; Michael A. Langston

cs.LGSep 30, 2026

Overcoming Challenges of Interpretive Structural Modeling with Large Language Models

Authors: Everett Rush, David J. Icove, Ari Kim, Byung H. Park, Michael A. Langston

Organizations: Department of Electrical Engineering and Computer Science, University of Tennessee, Knoxville, TN · Oak Ridge National Laboratory, Oak Ridge, TN

Abstract

Interpretive Structural Modeling (ISM) is a well-known process for multi-criteria decision making. The success of ISM over other methodologies is its ability to model causal relationships, the binary scale of factors, and resulting hierarchical representation. Traditionally, the modeling process is performed by repeated interactions with subject matter experts until consensus is reached. This process is tedious, labor-intense, and most importantly limits the ability of ISM to scale to studies with hundreds of variables. Drawing on existing work of causal graph discovery with large language models (LLM) as imperfect experts, this work explores an integrated LLM-ISM approach for ISM. Pairwise, k-wise, rowwise, and full graph discovery methodologies are compared and evaluated. It is shown that causal graph discovery methods for ISM perform best using rowwise (SHD=160, F1-score=0.77) and full graph methods (SHD=135, F1-score=0.73).

Figures & tables

Figure 1: Prompting methods for causal graph discovery in a single prompt. Pairwise (k=1) prompting considers only two variables at once (left). As

k

increases, the number of variables compared in a single prompt increases up to the entire row (k=n-1). Full graph prompting tries to discover the entire graph in a single prompt (right).

Study	Nodes	Edges	Density
i4t	17	115	.40
smartOrganizations	10	57	.57
automationRobots	8	30	.47
constructionWorkers	13	55	.33
socialMedia	15	69	.31
genAIInHigherEducation	9	37	.46

Table 1: ISM Graph Statistics

Batch Size	GPT-5.2		Nemotron 3		GPT-OSS-120B
	SHD $\downarrow$	F1-Score $\uparrow$	SHD $\downarrow$	F1-Score $\uparrow$	SHD $\downarrow$	F1-Score $\uparrow$
Pairwise (1)	170	0.75	190	0.73	180	0.74
2	185	0.74	180	0.73	195	0.72
4	174	0.74	175	0.74	195	0.72
Rowwise (n)	160	0.77	180	0.73	165	0.75
Full Graph	164	0.73	175	0.75	135	0.73

Table 2: Comparison of Causal Graph Methods

Multi-Criteria Decision Analysis

Multi-Criteria Decision Analysis (MCDA) ranks or selects alternatives by balancing competing criteria, preferences, uncertainty, and constraints.

See all 35 papers

Why LLMs Fail at Causal Discovery and How Interventional Agents Escape
May 26, 2026Amartya Roy, Sonali ParbhooCausal Discovery Causal Reasoning in Language Models
SIRE, IIT Delhi and Robert Bosch GmbH, India · Imperial College London
Leveraging Large Language Models for Causal Discovery: a Constraint-based, Argumentation-driven Approach
Feb 18, 2026Zihao Li, Fabrizio RussoCausal Discovery Causal Structure Learning
Department of Computing, Imperial College London, UK
LLM Explainability with Counterfactual Chains and Causal Graphs
Jun 4, 2026Nirit Nussbaum-Hoffer, Nitay Calderon, Liat Ein-Dor +1LLM Reasoning with Graphs LLM Interpretability
Faculty of Data and Decision Sciences, Technion · IBM Research

May 26, 2026J/K move · Enter open · S save

Why LLMs Fail at Causal Discovery and How Interventional Agents Escape

Amartya Roy, Sonali Parbhoo

SIRE, IIT Delhi and Robert Bosch GmbH, India · Imperial College London

Causal Discovery Causal Reasoning in Language Models Causal Identifiability Causal Interventions in Language Models

Open paper AskPDF arXiv

Causal discovery is a cornerstone of scientific reasoning, yet whether large language models can perform it reliably remains an open question. Recent benchmarks show that even fine-tuned models plateau on simple causal graphs and degrade as complexity grows, but why they fail has not been established. We prove the failure is fundamental: supervised fine-tuning, direct preference optimization, and in-context learning all produce predictors that cannot distinguish between causal graphs generating similar observational data, and any attempt to do so requires the model's internal representations to grow unboundedly, violating the very conditions under which these methods work. We formalize this as a kernel obstruction theorem, establishing that the limitation is intrinsic to the learning paradigm, not any particular model or dataset. We propose Agentic Causal Bayesian Optimization (A-CBO), wherein a frozen language model serves as an interventional oracle answering targeted queries about intervention effects, while an external Bayesian loop concentrates beliefs over candidate graphs in logarithmically many rounds. Because the decision operates outside the space where the obstruction applies, A-CBO provably converges while the underlying model remains unchanged. On Corr2Cause, A-CBO matches fine-tuned baselines without any training. On Extended Corr2Cause, a new benchmark scaling to 24 variables with 18K test samples, A-CBO significantly outperforms both fine-tuning and preference optimization, with the advantage growing

Abstract

Multi-Criteria Decision Analysis

Explore similar work

Why LLMs Fail at Causal Discovery and How Interventional Agents Escape

Leveraging Large Language Models for Causal Discovery: a Constraint-based, Argumentation-driven Approach

LLM Explainability with Counterfactual Chains and Causal Graphs