Narrow Decision Models: A System-1 Architecture for Calibrated, Typed Decisions

Carlos André Antunes; Aldry Albuquerque; Gabriel de Melo Schmith; Natã Batista; Pedro Oziel Escobar da Silva; Stefano Benko Rossi; Lingrid Dietrich Lorenz da Silva; Flávia Cristina da Costa

cs.AIOct 18, 2026

Narrow Decision Models: A System-1 Architecture for Calibrated, Typed Decisions

Authors: Carlos André Antunes, Aldry Albuquerque, Gabriel de Melo Schmith, Natã Batista, Pedro Oziel Escobar da Silva, Stefano Benko Rossi, Lingrid Dietrich Lorenz da Silva, Flávia Cristina da Costa,

Organizations: Softplan

Abstract

System One models, introduced with TypeSafe's Jev, answer typed questions about a state with calibrated probabilities instead of generated text: choosing among options described in natural language (Choice), placing the state on an ordered scale (Score), or judging a yes/no statement (Noul). Jev is general-purpose: one set of weights, served as an API and adapted to a domain only through the request. We study the other end of the design space. A Narrow Decision Model (NDM) offers the same interface but is trained for the decisions of one organization, is small enough to run on its own infrastructure, and has an open architecture whose properties can be stated and tested. The state can be any text or structured record about which decisions are made; our running case is the routing of court communications. The architecture serializes a request as a prefix tree (the state, then each question, then each option) and runs a bidirectional encoder under a block attention mask with logical positions, so that each answer is, by construction, invariant to the order of its options and independent of the other questions asked. Routing through an organization chart is a composition of Choice decisions with an explicit "none of these" option. We describe training with proper scoring rules through a curriculum, calibration and abstention, decisions composed by rubrics, and a protocol that accepts or rejects the mechanism before held-out data is opened. A minimal validation on public and fictitious data shows that the invariances hold to rounding in a trained model, that a small instance routes on organization charts it has never seen well above lexical retrieval, and that, at equal compute, the block architecture reaches a lower loss than a pairwise cross-encoder. A first instance trained on real legal documents agrees with the LLM classifier in production on 91% of held-out documents.

Figures & tables

Figure 1: Overview of NDM -Block. (a) A request is a prefix tree: the state, the typed questions under it, and the options or levels under each question; the yes/no criteria of a Noul question sit inside its instruction block. (b) The tree is packed into one sequence and encoded once under a block mask. Positions are logical: every question starts at the same position

a

, right after the state, and sibling options share position

b

. (c) One vector per marker feeds the readout heads, which return one distribution per question. All probabilities in the figures are illustrative.

Figure 2: Block attention mask for a request with two questions, two options under

Q_{1}

and one under

Q_{2}

. (a) The logical positions of a block continue from the end of its parent. (b) Token-level mask: each row attends to its own block and to its ancestors. The siblings

O_{11}

and

O_{12}

occupy the same logical positions and never see each other, which is what makes Propositions 1–3 hold by construction.

	NDM -Block	NDM -Pair	Decoder scoring
Forward passes per request	$1$	$\sum_{i}K_{i}$	$\sum_{i}K_{i}$ a
Times the state is encoded	$1$	$\sum_{i}K_{i}$	$1$ a
Options compared with each other	yes (set head)	no	no
Option-order invariance	by construction	by construction	no
Question isolation	by construction	by construction	prompt-dependent
Trained for calibration	yes	yes	no

Table 1: The same decision contract served by three architectures.

K_{i}

is the number of options of question

i

. a With prefix caching, one pass over the shared prefix and one short pass per option.

Figure 3: Routing as a descent over a fictitious organization chart. Every model node is a Choice over its children plus “none of these”; structural options encode procedural attributes, and rule nodes apply a deterministic rule over metadata. The bold path is the greedy hybrid descent.

Figure 4: A composed decision. The urgency distribution is the push-forward, through a frozen rubric, of the model’s own readings (type of act, injunction, sanction) and of the deadline in the state. The rubric shown is illustrative; the composed distribution is computed exactly from the readings displayed.

Figure 5: Training curriculum. Each stage starts from the weights of the previous one. Stage 2a never sees the target organization, so its artifact measures the skill of reading descriptions on options never seen (regime Z).

Figure 6: Synthetic data engine. The seed material is split four ways. Each communication starts from a sampled specification; its labels follow from frozen rules and its text is realized from the seed of the same partition. Validity guards send failing seed material back for rewriting.

	float32		float64
test	sdpa	eager	sdpa	eager	negative control
P1: permute the options	2.6e-6	4.7e-6	2.3e-9	1.6e-7	sequential positions: 3.7
P2: question alone vs. batched	3.8e-6	3.5e-6	6.2e-9	3.2e-7	no mask: 14.3
P2: state with and without questions	0	0	0	0	no mask: 2.0
P2: change a sibling question	0	0	0	0	no mask: 3.9
P2: order of the questions	2.4e-6	2.7e-6	1.9e-8	1.3e-7	sequential positions: 6.2

Table 2: Largest difference between two forms of the same request on ndm-base , and the smallest difference of the negative control. Tolerances:

10^{-4}

in float32; in float64,

10^{-12}

when the batch shape is kept and

10^{-6}

(sdpa) or

10^{-5}

(eager) when it changes. A control must differ by more than

10^{-3}

dataset (labels)	$K=4$	$8$	$16$	$32$	$60$	$77$
MASSIVE intents	0.956	0.924	0.878	0.811	0.740	—
Banking77	0.953	0.913	0.860	0.801	—	0.701

Table 3: Accuracy of ndm-base on public test splits (

n=2{,}974

and

3{,}080

sentences) by number of options read in one pass. 95% bootstrap intervals are within

\pm 0.02

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Component	Value
Backbone	mmBERT-base: 22 layers (8 global, 14 local, window $\pm 64$ logical positions), $d=768$ ; frozen embeddings (196.6 M), trained body (110.3 M)
Markers	9 used and 7 reserved vocabulary entries; offsets $\Delta\in\mathbb{R}^{17\times 768}$ , initialized to the mean embedding of an equivalent phrase
Question and option vectors	$W_{q}:\mathbb{R}^{1536}\to\mathbb{R}^{384}$ on $[h_{S};h_{Q}]$ plus a type embedding; $W_{o}:\mathbb{R}^{768}\to\mathbb{R}^{384}$
Set head	2 pre-norm layers, width 384, 6 heads, feed-forward 1,536; output projections initialized at zero
Relative scorer	$\mathrm{LN}[o;q;o\odot q]$ (1,152) $\to$ linear (384) $\to$ GELU $\to$ one linear reading per type
Adequacy	$\mathrm{LN}[h_{O};h_{Q};h_{O}\odot h_{Q}]$ (2,304) $\to$ linear (384) $\to$ GELU $\to$ linear

Appendix

Table 4: Sizes, weights and volumes of the reference configuration.

LLM Decision-Making

LLM decision-making studies how language models choose, act, seek information, and allocate resources under goals, uncertainty, constraints, and social pressures.

See all 229 papers

Oct 13, 2026

Jeva: System One Models from First Principles

System one models like Jev enable direct, low-latency decision-making. Yet recent open-source practice of predicting an answer from serialized text does not, by itself, respect the structure of a decision problem. We argue that current designs underemphasize four important structural requirements. (i) null-aware: Models should distinguish answer absence from uncertainty among candidates; (ii) permutation-invariant: Reordering alternatives should not change the decision; (iii) option-scalable: Growing candidate sets should not require redefining the output space; (iv) cacheable: Request-independent computation over recurring alternatives should be reusable across requests. We introduce Jeva, a decision-native architecture that addresses these requirements jointly. A shared encoder independently represents the request and each option, while a compact decision stack refines only the request state against immutable option memory. Symmetric aggregation yields permutation-equivariant probabilities; shared scoring accommodates variable-size candidate sets. A separate coverage estimate distinguishes answer absence from candidate competition, enabling explicit null outcomes. Request-independent option representations and layer projections support reuse across newly assembled sets. These mechanisms unify Choice, Noul, and Score outputs. Across evaluations against Jev and open source models, Jeva combines strong typed-decision performance with near-perfect null detection and no decision flips under any tested permutations. It scales to thousands of candidates and substantially reduces latency through cross-request reuse without changing decisions.

Qi Li, Xinchao Wang

National University of Singapore

LLM Decision-Making Efficient Language Model Inference Selective Prediction Permutation-Invariant Learning

Ask PDF

Oct 12, 2026

Releasing the Inner Jev: A Simple Recipe for Open Decision Models

LLM-based decision-making systems, also called System One models, have recently gained popularity, most notably through Jev (Almeida, 2026). Given a typed input state, a question, and a set of options, they return a probability for each option in a single forward pass, without text generation. We show that existing open instruction-tuned language models can achieve comparable performance without any additional training, by reading out answers from letters (A, B, C, etc.) in the model's output logits at the last token position. Furthermore, we show how a simple supervised fine-tuning recipe can improve their performance further. We evaluate four open instruction-tuned language models: Gemma 4 31B, Gemma 4 12B, Qwen3.6-27B, and Qwen3.5-4B. We fine-tune each using LoRA and a decision head initialized from a slice of the language model head. Furthermore, we introduce OpenDecisions, a novel 2,352-question validation dataset designed to reflect the task composition of JevBench (JevBench main- tainers, 2026) and which computes a Capability score based on accuracy and calibration metrics. Without fine-tuning, Gemma 4 31B scores 79.6 on OpenDecisions, compared to Jev's 80.1. After undergoing our fine-tuning recipe, it scores 84.4 on OpenDecisions and 81.5 on JevBench v1.7, outperforming Jev by 4.3 and 4.4 Capability points respectively and setting a new state-of-the-art on OpenDecisions. On a suite of public decision datasets, our models achieve accuracy comparable to Jev, with lower calibration error. Our results show that open language models already possess System One capabilities, and with simple fine-tuning they outperform Jev on a wide range of decision tasks. We release our models, serving code, and OpenDecisions.

Jonathan Geuter, Andrew Cheng, David Alvarez-Melis

Harvard University · Kempner Institute

LLM Decision-Making LLM Fine-Tuning Discrete Choice Modeling

Ask PDF

Sep 29, 2026

Benchmarking System One decision models against trained classifiers and language models for automated decision gates

Software that hands branching decisions to a model needs a declared option and a probability it can threshold. Typed decision models, also called System One models, return such probabilities without generating text, while supervised classifiers and generative language models are the established alternatives. One harness sends eight decision-model checkpoints from six families, including the hosted model Jev, and four open generative models from three developers the same semantic requests, and scores trained and zero-shot classifiers on the same workflow, intent, emotion and social-science items. With task labels, a fine-tuned DeBERTa-v3-large has the highest observed accuracy on every labeled benchmark but one. Without labels, no decision model is significantly more accurate than Jev on workflows or intents, but Gemma-4-31B matches it on workflows and exceeds it on CLINC-150 at higher cost and latency. Stated probabilities of generative models become unreadable when replies miss the key format, whereas key likelihoods avoid this but can saturate. A guaranteed 5 percent risk leaves Jev 0.528 of the intent decisions, and an in-scope threshold still accepts 0.310 of out-of-scope requests. On typed-decisions, swapping yes and no flips 50.5 answers per hundred for Jev and at least 16.8 for every generative model tested, against at most 6.5 for four fine-tuned decision checkpoints. Exposure to a benchmark's training data explains the largest lead of an open checkpoint, which vanishes on rater-labeled emotions. An intent-trained first stage escalating to Gemma-4-31B reaches that model's accuracy at about Jev's price. The results yield condition-dependent design rules for automated decision gates.

Amir Rafe, Subasish Das

Texas State University, 601 University Drive, San Marcos, 78666, Texas, USA

LLM Decision-Making LLM Evaluation Confidence Estimation in Language Models Selective Prediction

Ask PDF

Abstract

LLM Decision-Making

Explore similar work

Jeva: System One Models from First Principles

Releasing the Inner Jev: A Simple Recipe for Open Decision Models

Benchmarking System One decision models against trained classifiers and language models for automated decision gates