Decision-Centric Design for LLM Systems

Wei Sun

cs.AIApr 1, 2026

Decision-Centric Design for LLM Systems

Authors: Wei Sun

Organizations: IBM Research

Abstract

LLM systems must make control decisions in addition to generating outputs: whether to answer, clarify, retrieve, call tools, repair, or escalate. In many current architectures, these decisions remain implicit within generation, entangling assessment and action in a single model call and making failures hard to inspect, constrain, or repair. We propose a decision-centric framework that separates decision-relevant signals from the policy that maps them to actions, turning control into an explicit and inspectable layer of the system. This separation supports attribution of failures to signal estimation, decision policy, or execution, and enables modular improvement of each component. It unifies familiar single-step settings such as routing and adaptive inference, and extends naturally to sequential settings in which actions alter the information available before acting. Across three controlled experiments, the framework reduces futile actions, improves task success, and reveals interpretable failure modes. More broadly, it offers a general architectural principle for building more reliable, controllable, and diagnosable LLM systems.

Figures & tables

Component	Model Routing	Adaptive Inference Scaling
Actions $\mathcal{A}$	Candidate models	Inference strategies and compute configurations
Decision context $c$	Query features, predicted quality, API cost, latency, SLA	Query features, difficulty signals, token budget, latency constraints
Decision problem $\delta(c)$	$\arg\max_{a\in\mathcal{F}(c)}U(a,c)$ , where $U$ trades quality against cost (Eq. 1 )

Table 1: Two canonical settings mapped to the decision-centric abstraction (

\mathcal{A}

c

\delta

). Both instantiate the same structure; they differ in what actions represent and what signals populate the context.

	Prompt-Clarify				DC (ours)
$k$	Succ.	1st	Wasted	Clarif.	Succ.	1st	Wasted	Clarif.
0	100%	90%	0.00	0.10	100%	100%	0.00	0.00
1	100%	10%	1.35	1.25	100%	100%	0.65	1.65
2	75%	10%	2.20	2.25	100%	100%	0.00	1.00
3	60%	45%	2.20	2.80	100%	100%	0.55	1.55
4	10%	10%	2.90	3.00	100%	100%	0.10	1.40

Table 2: Calendar results by missing-field count

k

, averaged over ambiguity types. 1st : first-action optimality; Wasted : discarded executions; Clarif. : clarification turns per run.

$k$	Retry	Prompt-Clarify	DC (original)	DC (constrained) †
0	100%	70%	100%	100%
1	5%	50%	75%	100%
2	0%	75%	100%	100%
3	0%	45%	90%	95%
4	0%	30%	100%	100%

Table 3: LLaMA success rates by missing-field count

k

, averaged over ambiguity types. DC (original) uses the default question-generation module; † DC (constrained) restricts question generation to the listed missing fields.

Scenario	Retry	Prompt	Prompt (w/ policy)	DC
S1: Clean (baseline)	100%	100%	100%	100%
S2: Ambiguous ( $\hat{p}_{\text{suff}}$ only)	45%	85%	95%	100%
S3: Unreliable ( $\hat{p}_{\text{corr}}$ only)	100%	100%	100%	100%
S4: Orthogonal (joint, $T{=}6$ )	65%	95%	100%	100%
S5: Correlated (coupled update)	60%	35%	35%	100%

Table 4: Graph disambiguation success rates.

	Prompt		DC-LLM		DC-Dense		DC-Composite
Bucket	Succ.	RR	Succ.	RR	Succ.	RR	Succ.	RR
Easy	100%	0.10	100%	0.62	100%	1.12	100%	0.84
Medium	14%	0.08	88%	1.34	90%	1.66	94%	1.62
Hard	14%	0.14	18%	1.66	18%	1.84	18%	1.84

Table 5: Retrieval control results. Succ. = success rate; RR = avg retrieval rounds.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

$k$	Ambiguity	Initial query
0	—	Schedule a meeting with Jack on 2026-02-17 at 11:30 for 30 minutes.
1	Absent	Schedule a meeting with Jack on 2026-02-17 at 11:30.
1	Unresolvable	Schedule a meeting with Jack on Jack’s usual slot at 11:30.
2	Absent	Schedule a meeting with Jack at 11:30.
2	Unresolvable	Schedule a meeting with Jack at Jack’s usual time.
3	Absent	Schedule a meeting with Jack.

Appendix

Table 6: All 8 generated scenarios. Fields withheld at each

k

k{=}1

: duration_min ;

k{=}2

: duration_min , date ;

k{=}3

: duration_min , date , start_time ;

k{=}4

: all fields. Private facts (ground truth) are identical across all scenarios.

$k$	Ambiguity	Approach	Success	Wasted	Avg turns
0	—	Retry baseline	100%	0.00	1.00
		Prompt-based clarify	100%	0.00	1.10
		Decision-centric	100%	0.00	1.00
1	Absent	Retry baseline	0%	6.00	6.00
		Prompt-based clarify	100%	0.90	2.90
		Decision-centric	100%	0.10	2.20

Appendix

Table 7: Full results by missing-field count

k

and ambiguity type. Granite 4 (micro),

T{=}6

N{=}10

per cell. “Absent”: missing fields simply omitted. “Unresolvable”: missing fields referenced but not inferable.

		Retry		Prompt-clarify		DC (orig.)	DC (constrained)
$k$	Ambiguity	Success	Turns	Success	Turns	Success	Success
0	—	100%	1.00	70%	5.50	100%	100%
1	Absent	10%	5.90	60%	5.50	50%	100%
1	Unresolvable	0%	6.00	40%	5.90	100%	100%
2	Absent	0%	6.00	90%	5.60	100%	100%
2	Unresolvable	0%	6.00	60%	5.80	100%	100%

Appendix

Table 8: LLaMA 3 (8B) full per-scenario breakdown,

N{=}10

T{=}6

. DC (orig.): default question-generator prompt. DC (constrained): stricter prompt constraining question generation to missing fields only. The single change at

k{=}1

absent (50%

\to

100%) isolates the broken component: only the question generator was changed; all other components are identical. Prompt-clarify uses 5.5–6.0 turns on every scenario, exhausting the budget in clarification rather than executing.

Panel of Figure 1 — Figure 1: Diagnostic trace for LLaMA 3 at $k{=}1$ (before and after fix). Three logged values pinpoint the broken component: $\hat{p}_{\text{suff}}=0.75$ is correct; missing=[duration_min] is correct; yet the question asks about date and time—only the question generator is wrong. The fix targets only the question-generation prompt; the estimator, policy, and execution function are unchanged.

Node	Location	Level	Role in S5
Target	Tokyo	L3	Correct answer
D	Tokyo	L1	Forced first visit; triggers elimination
E	Tokyo	L1	Eliminated when D’s level=L1 is observed
F	Tokyo	L2	Survives elimination
F2	Tokyo	L4	Survives elimination
A	London	L2	Removed by location=Tokyo clarification

Appendix

Table 9: Injected S5 subgraph. All 10 nodes share department=Engineering , project=Alpha , role=Manager . After clarifying location=Tokyo : 5 candidates remain (Target, D, E, F, F2). After visiting D: level=L1 is minority (1 of 4 non-target peers

=

25%), so E is eliminated, leaving 3 candidates.

Scenario	Approach	Success	Wasted	Clarify
S1: Clean	Retry	100%	0.00	0.00
	Prompt	100%	0.00	0.00
	Prompt (w/ policy)	100%	0.00	0.00
	DC	100%	0.00	0.00
S2: Ambiguous ( $\hat{p}_{\text{suff}}$ only)	Retry	45%	3.70	0.00
	Prompt	85%	1.35	1.55

Appendix

Table 10: Full graph disambiguation results (

N{=}20

, structural

\hat{p}_{\text{corr}}

estimator). Wasted : traversals to wrong candidates per run. Clarify : clarification turns per run. Backtrack : backtrack actions per run.

$\alpha$	$\tau$	Easy Succ.	Easy RR	Medium Succ.
0.4	0.5	100%	0.66	78%
0.4	0.6	100%	0.66	78%
0.4	0.7	100%	0.70	82%
0.4	0.8	100%	0.84	88%
0.4	0.9	100%	1.16	92%
0.2	0.8	100%	0.66	78%

Appendix

Table 11: Offline threshold sweep for DC-Composite on the held-out test set (

N{=}50

per bucket). Parameters were selected on a separate validation set (

N{=}30

) before the test set was run. Top block: varying

\tau

\alpha{=}0.4

. Bottom block: varying

\alpha

\tau{=}0.8

. Bold row is the paper’s setting. Easy success is 100% throughout; the chosen operating point sits below the recoverable peak, trading a small gain in medium-bucket success for fewer wasted retrieval rounds on easy questions.

Bucket	Failure type	$N$	%
Medium	Early stop, both signals high	3	100%
Medium	(3 total failures)
Hard	Corpus gap (over-expand)	40	98%
Hard	Early stop, both signals high	1	2%
	(41 total failures)

Appendix

Table 12: Failure attribution for DC-Composite on the test set. Medium failures are dominated by cases in which both signals support an incorrect early stop . Hard failures are dominated by corpus gaps, where the gold passage is never retrieved within budget.

Type	Question / Gold answer	Prompt reason $\to$ action
Type 1 Correct assessment, wrong action (28/44)	“where did Iowa State football coach come from” / Massillon, Ohio	“Passages do not explicitly contain the answer…” $\to$ stop
	“what does it mean to fuse the spine” / neurosurgical technique…	“Passages do not explicitly contain the meaning…” $\to$ stop
Type 2 Latches onto wrong passage (13/44)	“who are the two teams in Super Bowl 2018” / Philadelphia Eagles	“Passage 1 lists the teams as the Steelers and Patriots.” $\to$ stop
	“where was Walking Tall filmed” / Squamish, BC	“Passage 1 mentions filming in Astoria, Oregon.” $\to$ stop
Type 3 Other (3/44)	“how many Super Bowl games has the Patriots played” / ten	“Passages mention the Patriots have played in 9 Super Bowls.” $\to$ stop

Appendix

Table 13: Prompt failure modes on medium questions (test set,

N{=}44

failures). Type 1 is the dominant pattern (64%): the model’s own reason string acknowledges the passages are insufficient, yet the fused call still outputs stop . This illustrates the core limitation of implicit control: a correct internal assessment cannot be acted upon when it is not externalized.

LLM Decision-Making

LLM decision-making studies how language models choose, act, seek information, and allocate resources under goals, uncertainty, constraints, and social pressures.

See all 242 papers

CogniConsole: Externalizing Inference-Time Control as a Formal Abstraction for Reliable LLM Interactions
Apr 21, 2026Vanessa Figueiredo, Wilter FranceschiAI Agent Reliability Language Model-Based Control
Orbital Sea Regina, SK, Canada
Credo: Declarative Control of LLM Pipelines via Beliefs and Policies
Apr 15, 2026Duo Lu, Andrew Crotty, Uğur ÇetintemelLLM Agent Orchestration LLM Agents
Brown University · Northwestern University
Multi-Head Latent Control: A Unified Interface for LLM Agent Decision Making
Jul 15, 2026Amirhosein Ghasemabadi, Ruichen Chen, Bahador Rashidi +1Multi-Agent LLM Systems Language Model-Based Control
1ECE Department, University of Alberta · 2Huawei Technologies Canada Co., Ltd.

Apr 21, 2026J/K move · Enter open · S save

CogniConsole: Externalizing Inference-Time Control as a Formal Abstraction for Reliable LLM Interactions

Vanessa Figueiredo, Wilter Franceschi

Orbital Sea Regina, SK, Canada

AI Agent Reliability Language Model-Based Control LLM Reliability LLM Context Management LLM Agent Reliability

Open paper AskPDF arXiv

Reliability in large language model (LLM) systems is typically framed as a function of model capability. We challenge this by demonstrating that reliability is significantly influenced by inference-time control -- the computational layer governing task framing and context selection. We introduce CogniConsole, an architectural instantiation that externalizes this control into a structured interface combining programmatic coordination with bounded prompt-based reasoning. Through controllability-oriented probes (

N=489

) in a multi-step interactive environment, we show that increasing structural scaffolding -- from unstructured to fully scaffolded -- systematically reduces output variance and failure rates under a fixed model architecture. Our results indicate that many observed failure modes, such as context drift and inconsistent constraint adherence, arise from under-specified control rather than insufficient capability. This work provides an empirical basis for treating inference-time control as a first-class abstraction, opening new directions for designing and evaluating LLM systems beyond scaling alone.

Abstract

LLM Decision-Making

Explore similar work

CogniConsole: Externalizing Inference-Time Control as a Formal Abstraction for Reliable LLM Interactions

Credo: Declarative Control of LLM Pipelines via Beliefs and Policies

Multi-Head Latent Control: A Unified Interface for LLM Agent Decision Making