The Hitchhiker's Guide to System 1 Models: Fast Decision-Making Beyond LLMS

Haggai Roitman

cs.AIOct 24, 2026

The Hitchhiker's Guide to System 1 Models: Fast Decision-Making Beyond LLMS

Authors: Haggai Roitman

Abstract

Large Language Models excel at reasoning and generation but their autoregressive, token-by-token inference makes them unnecessarily slow, expensive, and poorly calibrated for the many routine decisions (routing, gating, scoring, classification) that modern AI systems require at sub-second latency. A growing class of models addresses this gap: System 1 decision models, architectures that map an input directly to a typed, structured decision in a single forward pass. This guide provides a comprehensive, practitioner-oriented treatment of the field. We propose a five-family taxonomy covering encoder-based classifiers, reward models, non-autoregressive decision engines, contrastive decision models, and extraction models, and we provide in-depth coverage of each family's architecture, training methodology, and deployment trade-offs. A central theme is calibration: we develop the theory of proper scoring rules, analyze Reinforcement Learning for Calibrated Decisions (RLCD), and examine recently discovered phenomena including IIA violations, ordinal-scale bias, and few-shot recalibration under distribution shift. On the integration side, we describe how System 1 models serve as decision infrastructure within agentic workflows and System 1 + System 2 cascades that route between fast and slow models based on calibrated confidence. The guide covers 18 application domains, maps the rapidly growing open-source ecosystem (over 70 model reproductions in weeks), provides runnable code throughout, and includes a practitioner's checklist, an open-source model zoo, and 22 self-assessment questions with detailed answers. Spanning 190 references, it is intended as a unified entry point for researchers and engineers working at the intersection of fast decision-making, calibration, and production AI systems.

Figures & tables

Property	System 1	System 2
Speed	Fast (milliseconds)	Slow (seconds–minutes)
Effort	Effortless, automatic	Effortful, deliberate
Awareness	Unconscious, opaque	Conscious, introspectable
Parallelism	Highly parallel	Sequential
Evolutionary origin	Old, shared with animals	Recent, uniquely human
Knowledge	Associative, experiential	Rule-based, symbolic

Table 1: Properties of System 1 and System 2 cognition, adapted from Kahneman [62] and Stanovich and West [118] .

Precision $\varepsilon$	Min retries $N$	Cost ratio ( $C_{2}/C_{1}=50$ )	Practical meaning
$\pm 0.10$	97	$4{,}802\times$	Coarse: “roughly 70%”
$\pm 0.05$	385	$19{,}208\times$	Moderate: “between 65–75%”
$\pm 0.01$	9,604	$480{,}200\times$	Fine: “70.0% $\pm$ 1%”

Table 2: Minimum System 2 retries

N

for probability estimation at precision

\pm\varepsilon

(95% confidence, worst case

p^{*}=0.5

), and total cost ratio assuming

C_{2}/C_{1}=50

(a conservative mid-range for LLM vs. encoder-based System 1 models).

Panel of Figure 1 — Figure 1: Penalty profiles of the Brier score and log loss. Solid lines: loss when $y=1$ ; dashed lines: loss when $y=0$ . Both are minimized at the correct prediction ( $\hat{p}=y$ ), but they differ sharply at the extremes: the Brier score is bounded by 1, while the log loss diverges to $+\infty$ as the model becomes confidently wrong (visible as the steep rise at the edges). This makes log loss the more punitive objective for avoiding catastrophic false confidence.

Panel of Figure 2 — Figure 2: Reliability diagram (illustrative). A perfectly calibrated model follows the dashed diagonal. The System 1 model (green) hugs the diagonal (ECE $=$ 0.015). The overconfident LLM (red) curves below – at the 0.85 bin it claims 85% confidence but is correct only 66% of the time (bracket). The underconfident model (blue) curves above – it is more accurate than it claims.

Dimension	Description	Status
Decision interface	Maps input to a bounded, structured output	Defining characteristic
Inference architecture	Non-autoregressive or few-pass computation	Defining characteristic
Latency	Sub-100ms inference	Evaluated property
Calibration	Predicted probabilities match empirical frequencies	Evaluated property
Cost efficiency	Lower per-decision cost than LLM alternatives	Evaluated property
Determinism	Same input yields same output	Evaluated property

Table 5

Panel of Figure 3 — Figure 3: Taxonomy of System 1 Decision Models. The field is organized into five families based on historical lineage and typical deployment role. Family V (Extraction Models) is emerging and shares architectural DNA with Family III. Boundaries between families are porous (see Section 3 for discussion).

∗ Emerging family; boundaries with Family III are fluid.
Family	Architecture	Latency	Output	Calibration	Training	Params
I. Encoder-Based	Bidir. encoder + head	5–50ms	Logits/probs	Post-hoc	CE / NLI	100M–600M
II. Reward Models	Enc./dec. + scalar head	10–100ms	Scalar score	Task-dependent	BT / PL loss	300M–70B
III. Non-Autoregressive	Encoder + typed heads	10–70ms	Choice/Score/Noul	Incentivized (RLCD)	RLCD / RLCR	150M–27B
IV. Contrastive	Dual encoder + projection	5–30ms	Similarity score	Requires calib. layer	InfoNCE	300M–8B
V. Extraction ∗	Encoder + typed extraction	1–10ms	Typed values	Edit-distance based	Supervised / RLCD	150M–1B

Table 3: Comparison of the five System 1 decision model families. Latency and calibration values are indicative ranges from representative systems; actual performance depends on model size, hardware, input length, and evaluation protocol.

Panel of Figure 4 — Figure 4: BERT encoder architecture with [CLS] classification. Input tokens pass through an embedding layer and $L$ bidirectional transformer layers (horizontal arrows indicate full self-attention). The final [CLS] hidden state is fed to a classification head that outputs class probabilities $p(y\mid x)$ in a single forward pass.

Panel of Figure 5 — Figure 5: Bi-encoder vs. cross-encoder architectures. (a) A bi-encoder encodes query and document independently, then compares via dot product; embeddings are cacheable but interactions are shallow. (b) A cross-encoder jointly encodes the concatenated pair, enabling deep token-level interactions at the cost of requiring a full forward pass per pair.

Panel of Figure 6 — Figure 6: Bradley-Terry reward model training. Both the preferred ( $y_{w}$ ) and rejected ( $y_{l}$ ) responses are scored by the same encoder. The BT loss pushes the preferred response’s score above the rejected one. At inference, a single forward pass maps any (prompt, response) pair to a scalar reward.

Panel of Figure 7 — Figure 7: Unified architecture of Family III non-autoregressive decision models. All models share the pattern: bidirectional encoder $\to$ head selector $\to$ typed prediction head. The entire pipeline executes in one forward pass.

Model Type	FLOPs	Passes
Autoregressive LLM	$O(n\cdot m\cdot d^{2})$	$m$ (sequential)
Family III (Choice)	$O(L(nd^{2}+n^{2}d)+Kd)$	1 (parallel)
Family III (Score)	$O(L(nd^{2}+n^{2}d)+(u{-}\ell)d)$	1 (parallel)
Family III (Noul)	$O(L(nd^{2}+n^{2}d)+d)$	1 (parallel)

Table 4: Inference complexity: Family III vs. autoregressive LLMs.

n

: input length,

m

: output tokens,

d

: model dimension,

K

: options,

L

: layers.

Panel of Figure 8 — Figure 8: Training pipeline comparison: RLHF optimizes for human preference through a learned reward model and generates free-form text; RLCD optimizes for calibrated decisions through proper scoring rules and outputs typed decisions.

	Jev 1.14	Laya	THX-01	Sys. One Gemma	OpenJev	Tachyone
Parameters	Undiscl.	421M	322M	270M	1.7B	$\sim$ 150M
Encoder layers	Undiscl.	28	22	18	28	22
Hidden dim $d$	Undiscl.	1024	768	640	2048	768
Score head	Full ordinal	Full ordinal	Simplified	Simplified	Full ordinal	Simplified
Head design	Parallel+Box	Independent	Weight-tied	Independent	Independent	LoRA-switched
Routing	–	Ckpt. bank	–	–	–	Lang. routing

Table 5: Architectural comparison of Family III non-autoregressive decision models.

Model	Provider	Params	Latency	Distinguishing Feature
Jev 1.14 †	TypeSafe AI	Undiscl.	70–500ms	Pioneer; RLCD; Box prim.; images
OpenAI Decisions †	OpenAI	Undiscl.	–	gpt-6-luna; zero-shot schema
MS-Decision-1 †	Microsoft	9B	–	Qwen3.5-9B + RLCD post-training
d1	Liquid AI	Undiscl.	–	Open-weight transformer; multilingual
Mercury Decide	Mercury	Undiscl.	–	Real-time agentic routing
Solar Decide	Solar	Undiscl.	–	Edge-cloud hybrid routing

Table 6: Commercial System 1 decision models (late October 2026). All are closed-source APIs. New entries since the initial release are marked with

\dagger

Model	Provider	Params	Latency	Distinguishing Feature
Laya / Lia	Independent	421M	$\sim$ 33ms	Most popular OSS; multilingual
OpenJev-RLCD †	Independent	1.7B	$\sim$ 80ms	First open RLCD pipeline
Tachyone †	Independent	$\sim$ 150M	54–59ms	Local-first; 20+ languages
DeepOpen †	Independent	–	–	Laya-based; lang. routing
Decision 2.0 family	vLLM Sem. Router	2–27B	7–71ms	Top JevArena per size; vLLM
Strands Decider 2B	Strands	2B	$\sim$ 115ms	Pointer head on Qwen3.5-2B

Table 7: Open-source System 1 decision models (late October 2026). All are released under Apache 2.0. New entries since the initial release are marked with

\dagger

Panel of Figure 9 — Figure 9: CLM-8B dual-encoder architecture (Family IV). State and action branches share the frozen Qwen3-8B encoder but use separate projection MLPs. Action embeddings can be precomputed and cached, reducing per-decision cost to a single state encoding plus $K$ dot products.

Dimension	Contrastive / Dual	Cross-Encoder
Encoding	Independent	Joint
Caching	$\checkmark$ (action side)	$\times$
Latency ( $K$ actions)	$O(1)+K\cdot O(d^{\prime})$	$K\cdot O(\text{forward})$
Interaction depth	Shallow (dot product)	Deep (full attention)
Accuracy (small $K$ )	Lower	Higher
Scalability (large $K$ )	Higher	Lower

Table 8: Contrastive (dual-encoder) vs. Cross-encoder trade-offs.

Panel of Figure 10 — Figure 10: Decision models vs. extraction models. A decision model selects from a finite set of options with a calibrated probability. An extraction model reads a typed value (date, amount, entity) from the input with extraction confidence. Both share the single-forward-pass architecture but differ in output semantics and calibration frameworks.

Panel of Figure 11 — Figure 11: The Box primitive bridges decision and extraction. Given a state and a typed schema, the Box head outputs both the extracted value and a calibrated confidence score.

Panel of Figure 12 — Figure 12: Illustrative reliability diagram. The dashed diagonal represents perfect calibration. An overconfident RLHF-tuned model (red squares) systematically predicts higher confidence than its actual accuracy; the shaded bars show the per-bin ECE contribution. A well-calibrated RLCD model (green circles) tracks the diagonal closely. Data points are schematic, representing typical patterns reported in the literature [ 42 , 40 ] .

Panel of Figure 13 — Figure 13: Probability contracts as runtime SLAs. The model team publishes a contract specifying calibration guarantees. A runtime monitor enforces the contract; violations trigger automatic fallback to System 2.

Aspect	RLHF	RLCD
Optimizes for	Human preference	Calibrated probabilities
Reward signal	Learned reward model	Proper scoring rules
Training data	Preference pairs	Decision tasks with outcomes
Output	Free-form text	Typed decisions (Choice/Score/Noul)
Calibration	Poor (side effect)	By construction (objective)
Cost	Expensive (4 models)	Moderate (2 models)

Table 9: RLHF vs. RLCD training paradigm comparison.

Panel of Figure 14 — Figure 14: Knowledge distillation from a large LLM teacher to a compact System 1 student. The student trains against both hard ground-truth labels (cross-entropy) and the teacher’s soft probability distribution (KL divergence at temperature $T$ ). The soft labels carry “dark knowledge” about inter-class similarities that hard labels discard.

Panel of Figure 15 — Figure 15: Curriculum training for System 1 decision models. Training progresses from easy, high-agreement decisions (Stage 1) through moderately ambiguous tasks (Stage 2) to hard edge cases (Stage 3). Within each stage, multi-task batches mix different decision types (Noul, Choice, Score).

Panel of Figure 16 — Figure 16: Agent routing with confidence-gated fallback. At each step, a System 1 router selects a tool in $\sim$ 30ms. If confidence exceeds threshold $\theta$ , the tool executes directly; otherwise, a System 2 LLM fallback deliberates over the choice.

Panel of Figure 17 — Figure 17: Traditional RAG (left) performs embedding, vector search, and reranking at query time. CROW (right) replaces retrieval with classifier-routed navigation over a pre-organized knowledge base, reducing per-query overhead.

Panel of Figure 18 — Figure 18: System 1 decision points within an agent loop iteration. Three fast decisions (green, milliseconds) orchestrate the control flow around one slow generative step (purple, seconds). A loop with 20 iterations makes ${\sim}60$ System 1 decisions.

Loop Primitive	System 1 Role	Decision Type	Model Family
Automations	Trigger classification	Should this event fire the loop?	Noul / Choice
Worktrees	Conflict detection	Are parallel branches compatible?	Score
Skills	Skill selection	Which workflow template applies?	Contrastive (CLM)
Connectors	Tool routing	Which external system to invoke?	Choice / Contrastive
Sub-agents	Task decomposition	Route subtask to which specialist?	Choice / Score

Table 10: System 1 decision points within five loop engineering primitives.

Panel of Figure 19 — Figure 19: Confidence-based cascade: System 1 handles the majority of decisions at millisecond latency. Only uncertain cases ( $\hat{p}<\theta$ ) escalate to the slower, more powerful System 2 model. Calibration quality directly determines cascade efficiency.

Benchmark	Task Type	Size	Metrics
GLUE / SuperGLUE [ 127 ]	NLI, classification	9/10 tasks	Accuracy, F1
MS MARCO [ 92 ]	Passage reranking	500K queries	MRR, NDCG
RewardBench [ 71 ]	Reward model eval	2,985 examples	Accuracy
DeepSWE [ 20 ]	Code verification	113 tasks	Accuracy
Terminal-Bench 2.1 [ 45 ]	Agent verification	89 tasks	Accuracy
LLM-WikiRace [ 46 ]	Navigation routing	Procedural	Success rate

Table 11: Key benchmarks for evaluating System 1 decision models.

Dimension	Primary Metrics	What It Answers	Current Coverage
1. Accuracy	F1, AUC-ROC, $\kappa$	Is the decision correct?	Well covered
2. Calibration	Brier, ECE, log loss	Can we trust the confidence?	Partial
3. Latency	P50/P95/P99, throughput	Is it fast enough?	Partial
4. Cost	$/million decisions	Is it economical?	Rarely reported
5. Robustness	OOD $\Delta$ acc, adversarial flip rate	Does it fail gracefully?	Rarely reported

Table 12: Five evaluation dimensions for System 1 models with recommended metrics.

Panel of Figure 20 — Figure 20: Schematic Pareto frontier of decision latency vs. decision complexity. System 1 models (green) cluster in the fast/structured regime; LLMs (red) occupy the slow/flexible regime. The intermediate region (yellow, 50–200ms, moderate complexity) is largely unexplored. Points are illustrative, not from a single benchmark.

Predicted $\hat{p}$	Observed positive rate		$\|\hat{p}-\text{acc}\|$
	Train dist.	Shifted dist.	Train	Shifted
0.2	0.21	0.35	0.01	0.15
0.4	0.39	0.59	0.01	0.19
0.6	0.61	0.76	0.01	0.16
0.8	0.79	0.90	0.01	0.10
ECE			0.01	0.15

Table 13: Calibration degradation under label shift. The model’s predictions are unchanged, but the true positive rate per bin shifts upward. Accuracy drops modestly, while ECE degrades

15\times

(0.01

\to

0.15).

Panel of Figure 21 — Figure 21: Reliability diagram under distribution shift. Green : before shift, the model is well-calibrated (points hug the diagonal). Red : after label shift, observed accuracy systematically exceeds predicted confidence (the model is underconfident ); the shaded area shows the ECE gap. Blue dashed : after recalibration via Eq. 70 , predictions are shifted rightward to realign with the diagonal.

Panel of Figure 22 — Figure 22: The calibration–plasticity dilemma. After learning Tasks 2 and 3, a continual learning model retains high accuracy on Task 1 (left), but its calibration (ECE) degrades substantially (right). Above ${\sim}5\%$ ECE (dashed line), cascade thresholds and safety gates become unreliable—a silent failure mode that accuracy metrics alone do not detect.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Model	Base	Params	License	RLCD?	Prims	Latency
Clef	Cloudflare	27B / 9B	Apache-2.0	No	C,N	–
Julia	Custom	$<$ 500M	Apache-2.0	No	C,N	–
Tachyone	ModernBERT	$\sim$ 150M	Apache-2.0	No	C,S,N	54ms
Sys. One Gemma	Gemma 3	270M	Apache-2.0	No	C,S,N	–
THX-01	Custom	322M	Apache-2.0	Yes	C,S,N	10ms
Laya / Lia	Custom	421M	Apache-2.0	Yes	C,S,N	33ms

Appendix

Table 14: Open-source System 1 model zoo (late October 2026). RLCD = uses RLCD training; Prims = supported decision primitives (C=Choice, S=Score, N=Noul, B=Box).

Language Model Calibration

Language model calibration studies whether models’ confidence estimates match their actual accuracy across tasks, prompts, and deployment settings.

See all 386 papers

Oct 15, 2026

System1-Arena: Which Decisions Can Be Left to System 1?

Large language models (LLMs) often make discrete decisions, such as flagging phishing or escalating to a human. System-1 models, such as Jev and the OpenAI Decisions API, return probabilities over options in a single forward pass, while System-2 approaches generate reasoning steps. Which decisions can System-1 models handle, and which require generative reasoning? We introduce System1-Arena, a benchmark that gives all models the same decision requests and scores their probabilities. It contains 92k questions across 105 tasks from 268 public sources, in four groups: text classification and knowledge, applied decisions, evaluation and control, and agent and action. It also includes 138 closed-loop environments, from web shopping and SWE-bench coding control to robotics, games, and driving; in real-time mode, the world evolves while a model decides. We evaluate 30 models: 7 closed LLMs, 13 open LLMs read at the answer token, and 10 System-1 models. The largest System-1 models match or exceed fast closed LLMs in non-agentic accuracy, with better-calibrated probabilities at one-quarter of the latency. A reasoning LLM is the most accurate, at about 70x the cost of a fast LLM. In closed loops, large open LLMs succeed most often (up to 32.9%); small System-1 models rarely succeed.

Tianyu Chen, Xinyang Liu, Ruyi An +2

The University of Texas at Austin

LLM Decision-Making LLM Evaluation Efficient Language Model Inference

Ask PDF

Oct 1, 2026

Fast Models, Slow Evidence: A Paired and Self-Audited Evaluation of System-1 Decision Models for LLM Agent Harnesses

Agent harnesses make many small, typed decisions per task: which model to call, which tool to use, whether retrieved text is relevant, whether an input carries an injection. System-1 decision models answer such questions in a single forward pass with class probabilities, promising large cost and latency savings over LLM calls. We present a paired evaluation of an open-weight (Laya) and a hosted (Jev) System-1 model on 11 agent decision points built from 18 public sources: 7,283 base cases plus 6,640 robustness variants, with byte-identical inputs, paired tests, and cross-hardware and cross-day reproducibility checks. Jev is significantly more accurate on 9 of 11 decision points (+10.8 to +46.0 pp). Neither model beats chance on zero-shot model routing, and they tie on RAG relevance gating. Laya changes 30% of its answers when the option order is reversed and degrades sharply with many or similar candidates (31% at 50 nearest-neighbour tools, vs. 98% for Jev on items with a unique correct tool). We also audit our own pipeline. Three analysis errors and one design confound distorted headline deployment claims: an omitted pre-screen cost (reported 23.9% saving, actual 4.3%), gate accuracy reported as end-to-end quality (58% vs. 98%), in-sample thresholds (5% target, up to 17% held-out misses), and a "channel effect" on injection false positives that vanishes with channel-native content. Two other suspected confounds did not change the conclusions. All cases, raw outputs and analysis code are available at https://github.com/David-DL-Space/sys1-eval.

Jiawei Li

LLM Agent Evaluation LLM Decision-Making LLM Agent Harnesses LLM Routing LLM Tool Use

Ask PDF

Oct 12, 2026

Releasing the Inner Jev: A Simple Recipe for Open Decision Models

LLM-based decision-making systems, also called System One models, have recently gained popularity, most notably through Jev (Almeida, 2026). Given a typed input state, a question, and a set of options, they return a probability for each option in a single forward pass, without text generation. We show that existing open instruction-tuned language models can achieve comparable performance without any additional training, by reading out answers from letters (A, B, C, etc.) in the model's output logits at the last token position. Furthermore, we show how a simple supervised fine-tuning recipe can improve their performance further. We evaluate four open instruction-tuned language models: Gemma 4 31B, Gemma 4 12B, Qwen3.6-27B, and Qwen3.5-4B. We fine-tune each using LoRA and a decision head initialized from a slice of the language model head. Furthermore, we introduce OpenDecisions, a novel 2,352-question validation dataset designed to reflect the task composition of JevBench (JevBench main- tainers, 2026) and which computes a Capability score based on accuracy and calibration metrics. Without fine-tuning, Gemma 4 31B scores 79.6 on OpenDecisions, compared to Jev's 80.1. After undergoing our fine-tuning recipe, it scores 84.4 on OpenDecisions and 81.5 on JevBench v1.7, outperforming Jev by 4.3 and 4.4 Capability points respectively and setting a new state-of-the-art on OpenDecisions. On a suite of public decision datasets, our models achieve accuracy comparable to Jev, with lower calibration error. Our results show that open language models already possess System One capabilities, and with simple fine-tuning they outperform Jev on a wide range of decision tasks. We release our models, serving code, and OpenDecisions.

Jonathan Geuter, Andrew Cheng, David Alvarez-Melis

Harvard University · Kempner Institute

LLM Decision-Making LLM Fine-Tuning Discrete Choice Modeling

Ask PDF

Abstract

Language Model Calibration

Explore similar work

System1-Arena: Which Decisions Can Be Left to System 1?

Fast Models, Slow Evidence: A Paired and Self-Audited Evaluation of System-1 Decision Models for LLM Agent Harnesses

Releasing the Inner Jev: A Simple Recipe for Open Decision Models