Intern-Decision: Efficient Multimodal Decision Making with Autoregressive Masked Language Modeling

Tong Zhu; Zhe Xu; Zefeng He; Yunkai Chai; Xuyang Hu; Dehong Zheng; Yun Luo; Yafu Li

cs.CLOct 15, 2026

Intern-Decision: Efficient Multimodal Decision Making with Autoregressive Masked Language Modeling

Authors: Tong Zhu, Zhe Xu, Zefeng He, Yunkai Chai, Xuyang Hu, Dehong Zheng, Yun Luo, Yafu Li,

Organizations: Shanghai AI Lab · Shanghai Jiao Tong University · Fudan University · Nanyang Technological University

Abstract

Interactive agents repeatedly choose among a finite set of actions, labels, or ordinal outcomes. Generating a free-form response for each choice adds latency and complicates the interface between a model and its environment. We present Intern-Decision, a family of 0.8B, 2B, and 4B multimodal decision models that return structured choices and candidate probabilities from text and optional images. Our autoregressive masked language modeling objective places one decision placeholder per field in a complete assistant JSON skeleton. Ground-truth symbols appear only in supervision, while all fields are evaluated in a single causal forward pass using the backbone's native language-model head. The 4B model achieves a 90.02% unweighted average accuracy across seven evaluation suites, compared with 88.74% for Jev v1.13.0. For a fixed 289-token request with three fields on one RTX 4090, mean latency is 44.16 ms for the 4B model and approximately 33--34 ms for the smaller models. On a separate 96-case known-distribution pilot, temperature scaling reduces expected multiclass Brier from 0.628 to 0.550 and expected calibration error from 0.213 to 0.089. A confidence-based handoff study on 111 public Jevbench-Hard items improves accuracy from 82/111 (73.87%) to 97/111 (87.39%) by routing 42 items to Atria-Dawn-Preview. We also describe qualitative game, visual interface, and browser demonstrations. These results support compact decision models as a practical component of larger agent systems.

Figures & tables

Panel of Figure 1 — Figure 1: A constructed three-field request. Blue panels show visible evidence and masked assistant context; gold shows target answer symbols: A for billing, C for urgency 2, and B for yes. These symbols supervise the decision positions and never enter the input. System instructions, control tokens, and some option descriptions are omitted or shortened.

Panel of Figure 2 — Figure 2: Intern-Decision inference. All fields share the same state and schema, and their placeholders are processed together. The readout uses next-token alignment, not same-position supervision.

Component	Representative sources / tasks	Capability focus
Synthetic text decisions	General and Hard decisions synthesized by deepseek-v4.1-flash	Evidence use, rule following, and difficult textual judgments
Public task distributions	AG News [ 31 ] , ToolACE [ 19 ] , Typed Decisions [ 21 ] , WildGuardMix [ 14 ] , WildJailbreak [ 18 ] , and Ruozhiba (LLM Ruozhiba QA and COIG-CQIA) [ 15 , 2 ]	Classification, tool routing, typed decisions, safety, and reasoning
Synthetic visual and interactive tasks	Game environments, VizDoom, and shapes/captcha-style tasks	Visual grounding, discrete control, and pointer-action selection

Table 1: Training dataset composition.

Figure 3: Training mixture shared by the released models. The inner ring shows components; numbered outer sectors show data categories, including each game. Counts are training-input samples.

Suite	Rows	Decisions	Evaluation focus
Jevbench-Easy	48	48	Explicit facts and direct decisions
Jevbench-Original	72	72	Policy, routing, and typed judgments
Jevbench-Hard	111	111	Longer contexts and interacting rules
Typed Decision	400	2,000	Multiple typed fields per state
ToolACE	310	310	Tool selection
AG News	7,600	7,600	News classification

Table 2: Accuracy evaluation scope. Counts refer to the released test bundle, not the full upstream datasets.

Model	Easy	Original	Hard	Typed	ToolACE	AG News	WildJB	Average
Jev	100.00	98.61	72.07	73.35	91.29	89.57	96.29	88.74
Laya	95.83	72.22	28.83	35.95	63.87	92.84	14.84	57.77
SemIf	100.00	98.61	61.26	62.80	85.16	89.22	92.53	84.23
Kev	100.00	93.06	45.05	65.60	87.42	89.82	75.97	79.56
JevK5	100.00	97.22	73.87	64.50	80.97	89.13	90.45	85.16
Intern-Decision-0.8B	97.92	80.56	52.25	77.35	94.52	88.61	64.48	79.38

Table 3: Decision accuracy in percent. Average is the unweighted mean of the seven suites. Denominators are given in Table 2 ; bold data values mark column maxima, including ties. The comparison is restricted to these evaluated systems and protocols.

Model	Brier $\downarrow$	ECE $\downarrow$	Temperature
Jev	0.358420	0.094685	Reported/default
Laya	0.804244	0.246455	Reported/default
SemIf	0.498012	0.112213	1.000000
Kev	0.738253	0.262200	2.962195
JevK5	0.366162	0.046672	1.532000
Intern-Decision-0.8B	0.529519	0.065710	2.747761

Table 4: Jevbench-Hard probability metrics on 111 items. ECE uses maximum candidate probability and 10 equal-width bins. Intern-Decision uses separately fitted temperatures; baseline probabilities use their reported/default settings. Temperatures are not comparably refitted for every baseline.

Figure 4: Jevbench-Hard reliability: 111 items and ten equal-width bins. Only populated bins are shown.

Model	Mean	P50	P95	Mean ratio
Jev	109.70	106.30	146.70	1.00 $\times$
Intern-Decision-0.8B	33.98	33.44	37.50	3.23 $\times$
Intern-Decision-2B	33.28	33.15	33.55	3.30 $\times$
Intern-Decision-4B	44.16	44.03	44.60	2.48 $\times$

Table 5: Reported request latency in milliseconds. Ratios are Jev mean divided by model mean. The supplied protocol establishes the local Intern-Decision configuration; Jev’s complete serving and transport conditions still require documentation.

Category	Jev	4B uncalibrated	4B calibrated
Direct randomness and support	0.490 / 0.216	0.483 / 0.181	0.421 / 0.129
Composed events and mixtures	0.682 / 0.274	0.677 / 0.254	0.577 / 0.150
History, conditioning, hidden state	0.657 / 0.113	0.711 / 0.219	0.613 / 0.108
Daily evidence and observation bias	0.603 / 0.114	0.701 / 0.328	0.575 / 0.210
Selective disclosure and puzzles	0.483 / 0.138	0.540 / 0.119	0.510 / 0.049
Sequential and combinatorial processes	0.657 / 0.116	0.656 / 0.180	0.605 / 0.058

Table 6: Known-distribution pilot results. Each cell is expected Brier / expected ECE; bold marks the lowest value across the three settings for each metric separately. All settings have 96/96 valid predictions. Category rows have 16 cases each. Overall ECE is recomputed on the pooled cases, not averaged over categories.

System	Correct / 111	Accuracy	Atria calls	Total time
Intern-Decision-4B	82	73.87%	0	4.90 s, derived
Confidence-based handoff	97	87.39%	42	140.16 s, estimated
Atria-Dawn-Preview only	99	89.19%	111	300.03 s, measured

Table 7: Public model handoff study on Jevbench-Hard. The hybrid time is an estimate, while the Atria-only total is reported as measured. The local total is derived from 111 times the fixed-request mean latency and is not a per-item Hard timing trace.

Figure 5: Qualitative examples of visual game control with Intern-Decision. From left to right: navigating toward food while avoiding the snake body; catching fruit while avoiding bombs; moving and jumping through a Mario level; searching for health packs; and firing at approaching enemies. Each column shows two consecutive video frames, with the earlier frame on top and the later frame below.

Figure 6: Visual selection trajectories rendered as static video strips. Each row contains five chronologically ordered frames: (a) selecting the largest red sphere; (b) selecting the smallest red cube; (c) identifying the odd shape; and (d) selecting the second blue object from the left. Arrows indicate temporal progression, with intermediate steps omitted. Panels preserve the recorded object layouts and cursor positions; interface text is translated into English for presentation, and video presentation overlays are omitted.

Figure 7: Browser-use trajectories rendered as static video strips. Each row contains five chronologically ordered frames: (a) Hugging Face checkpoint and configuration verification; (b) GitHub repository and API-reliability investigation; and (c) PyTorch documentation and pinned-source inspection. Arrows indicate temporal progression. All panels show the browser interface itself without the presentation overlays used in the demo videos.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Checkpoint family	Temperature
Intern-Decision-0.8B	2.747760550702957
Intern-Decision-2B	2.100509348277736
Intern-Decision-4B	1.9924182353655278

Appendix

Table 8: Full-precision released temperature presets. Each value belongs to its checkpoint and the reported XTuner execution setting; it is not a freely transferable model-size constant.

Setting	Recorded value
Initialization	Matching Qwen3.5-0.8B, Qwen3.5-2B, or Qwen3.5-4B backbone
Epochs / optimizer updates	2 / 156
Peak / minimum learning rate	$2\times 10^{-6}$ / $2\times 10^{-7}$
Schedule / warm-up ratio	Cosine / 0.03
Global batch / sequence length	16 packed sequences / 8,192 tokens
Precision / GPU count / seed	BF16 / 4 / 42

Appendix

Table 9: Recovered settings shared by the released 0.8B, 2B, and 4B runs. Global batch size counts packed sequences rather than raw examples.

Component	Samples	Decisions
Synthetic decision text	6,711	6,711
Public task distributions	5,669	10,789
Synthetic visual and interactive data	5,500	6,000
Total	17,880	23,500

Appendix

Table 10: Frozen training mixture shared by the released Intern-Decision models.

LLM Decision-Making

LLM decision-making studies how language models choose, act, seek information, and allocate resources under goals, uncertainty, constraints, and social pressures.

See all 246 papers

Releasing the Inner Jev: A Simple Recipe for Open Decision Models
Oct 12, 2026Jonathan Geuter, Andrew Cheng, David Alvarez-MelisLLM Decision-Making LLM Fine-Tuning
Harvard University · Kempner Institute
StreamDecisionBench: Evaluating Decisions in Force on Evolving Language Streams
Sep 29, 2026Jhen-Ke Lin, Chung Chun WangLLM Evaluation LLM Inference
National Yang Ming Chiao Tung University
Seeing the Options: Typed Decisions over Visual Candidates with Reusable Option Memory
Oct 24, 2026Fengxin LiEfficient VLM Inference Multimodal Model Evaluation
Harbin Institute of Technology, Shenzhen

Oct 12, 2026J/K move · Enter open · S save

Releasing the Inner Jev: A Simple Recipe for Open Decision Models

Jonathan Geuter, Andrew Cheng, David Alvarez-Melis

Harvard University · Kempner Institute

LLM Decision-Making LLM Fine-Tuning Discrete Choice Modeling

Open paper AskPDF arXiv

LLM-based decision-making systems, also called System One models, have recently gained popularity, most notably through Jev (Almeida, 2026). Given a typed input state, a question, and a set of options, they return a probability for each option in a single forward pass, without text generation. We show that existing open instruction-tuned language models can achieve comparable performance without any additional training, by reading out answers from letters (A, B, C, etc.) in the model's output logits at the last token position. Furthermore, we show how a simple supervised fine-tuning recipe can improve their performance further. We evaluate four open instruction-tuned language models: Gemma 4 31B, Gemma 4 12B, Qwen3.6-27B, and Qwen3.5-4B. We fine-tune each using LoRA and a decision head initialized from a slice of the language model head. Furthermore, we introduce OpenDecisions, a novel 2,352-question validation dataset designed to reflect the task composition of JevBench (JevBench main- tainers, 2026) and which computes a Capability score based on accuracy and calibration metrics. Without fine-tuning, Gemma 4 31B scores 79.6 on OpenDecisions, compared to Jev's 80.1. After undergoing our fine-tuning recipe, it scores 84.4 on OpenDecisions and 81.5 on JevBench v1.7, outperforming Jev by 4.3 and 4.4 Capability points respectively and setting a new state-of-the-art on OpenDecisions. On a suite of public decision datasets, our models achieve accuracy comparable to Jev, with lower calibration error. Our results show that open language models already possess System One capabilities, and with simple fine-tuning they outperform Jev on a wide range of decision tasks. We release our models, serving code, and OpenDecisions.

Abstract

LLM Decision-Making

Explore similar work

Releasing the Inner Jev: A Simple Recipe for Open Decision Models

StreamDecisionBench: Evaluating Decisions in Force on Evolving Language Streams

Seeing the Options: Typed Decisions over Visual Candidates with Reusable Option Memory