Agents end many steps in a typed decision over a set of candidates: which screen element to click, which product to pick. System One models answer in one forward pass, and Jeva showed that four properties the setting demands - a "none of the above" probability, invariance to candidate order, scalability in the number of candidates, and reuse of candidate computation - follow from encoding the request and each candidate separately and letting a request state read a candidate memory. Jeva is text-only, and existing multimodal System One models (OneJev, Jev-Omni) put images in the request while candidates stay text in letter slots. This paper asks what happens to the four properties, and to accuracy, when the candidates themselves are images. Iris admits image, text, or mixed candidates and leaves Jeva's decision module untouched. On a visual-candidate benchmark built from Multimodal-Mind2Web (every candidate is the element's screenshot crop, K=16, three splits), independent encoding reaches 78.4/72.1/74.8% with a flip rate of exactly zero, while joint encoding (OneJev-4B fine-tuned on the same data) reaches 90.6/89.5/91.6% but changes its answer on 8-10% of records when the crops are reordered. Most of the gap is the candidate not seeing the request: a third design that encodes each candidate after the state - but never after another candidate - keeps order invariance exactly, shares the state pass through a forked cache, and reaches 92.1/87.0/90.2% at a third of OneJev's latency with a calibrated null probability (AUROC 0.93-0.97). Letting pooled candidates see each other (set attention, token-level reads, multi-vector memory) buys nothing, because the encoder maps visually distinct elements to vectors within cosine 0.999 of each other. Reusing image candidates across requests cuts latency 37x at K=256 and 191x at K=2048. Code: https://github.com/lllfx/Iris
Figures & tables
Figure 1: Jeva encodes the request and each text candidate independently and lets the state read a candidate memory. Iris keeps that decision module and the four properties it guarantees, and lets the state and every candidate carry images: a screenshot crop, a chart, a product photo. The vision tower that Jeva loads and discards is used; nothing in the decision path changes.
Design
Order-invariant
Cacheable
Null
Encodings / request
K in practice
Letter slots ( OneJev , Liu et al.)
no
no
no
K
≤16 – 256
Runoff (inference-time)
by construction
no
no
K+m2
tens
Bi-encoder (MetaEncoder)
approx.
candidates only ∗
no
(1−r)K
millions (open set)
Jeva
structural
both sides
yes
(1−r)K
thousands (text)
Iris , independent
structural
both sides
yes
(1−r)K
thousands (images)
Iris , state-conditioned
structural
within a request
yes
1+K (state once)
thousands (images)
Table 1: How System One designs handle visual candidates. r : fraction of candidates already cached. ∗ With the candidate list inside the request (needed for closed-set accuracy) the request itself is not reusable. The state-conditioned Iris encoder (§ 3.5 ) shares the state pass across a request’s candidates and questions but not across requests.
Figure 2: What a candidate’s vector may depend on. (a) Joint encoding: the answer depends on the order and number of candidates and nothing is reusable. (b) Independent encoding: ui is a function of ci alone, so the decision is order-invariant, unbounded in K and cacheable across requests (Proposition 1 ). (c) State-conditioned encoding: each candidate continues from a fork of the state’s cache—order invariance and null handling are unchanged, the state pass is shared within a request, and cross-request reuse is given up. (d) Readouts that let the pooled candidate vectors see each other keep every guarantee of (c) and, in our experiments, add nothing (§ 5.5 ).
Figure 3: A Multimodal-Mind2Web step as Iris sees it. Left: the task, the last previous actions and the question form the text state. Right: each candidate (14 here, after deduplication) is the element’s own crop with a short label; the gold element is outlined in red, at a shuffled position.
Accuracy (%)
Flip rate (%)
Model
Candidates
Train data
Task
Website
Domain
Task
Website
Domain
Jeva -4B (reported)
HTML text
Jeva curriculum
79.99
62.25
66.23
0.00
0.00
0.00
OneJev -4B
labels (page in state)
OneJev-Data
26.40
24.88
24.20
60.13
67.75
69.97
OneJev -4B
crops in state
OneJev-Data
37.85
37.46
34.83
50.64
52.80
54.30
OneJev -4B retrained
crops in state
+ Mind2Web crops
90.58
89.48
91.59
7.63
10.13
7.99
Iris -4B zero-shot
crops
–
31.11
25.07
32.73
0.00
0.00
0.00
Table 2: Element selection on Multimodal-Mind2Web with K=16 : first-choice accuracy and decision flip rate over five candidate orders (original, reversed, three seeded random), on the cross-task / cross-website / cross-domain splits. Jeva ’s row uses text candidates and is the reference point for the same positives. OneJev cannot take image candidates: “labels” gives it each candidate’s text label with the page screenshot in the state (its native setting); “crops in state” lists every candidate’s crop inside the state and keeps letter slots for the candidates, the only way a letter-slot model can look at the crops; “retrained” fine-tunes OneJev -4B with LoRA on the same 7,751 training steps in that format (Appendix A ). Trained rows report the final checkpoint; the “selected” row is the periodic checkpoint with the best cross-task accuracy (epoch 2, step 50; Appendix C.2 ), so its cross-task number ( † ) is a selection-split number, not a test number; the “abstention stage” row continues that checkpoint for one epoch on a null-oversampled subset of the training steps, and the released row applies to it an abstention margin selected on cross-task (§ 5.4 ); the last three rows add candidate-side readouts to these models and are discussed in § 5.5 . Chance is 6.25%. For Iris , all-order accuracy equals first-order accuracy on every split: a record answered correctly is answered correctly in every order.
Figure 4: Accuracy against order dependence on Mind2Web, one marker per model and split. The two released Iris models sit at a flip rate of exactly zero; the retrained OneJev buys 1–3 points of accuracy on two splits with an 8–10% flip rate, and the released OneJev answers largely by position.
Figure 5: The collapse diagnosis on cross-website (released conditioned model). Left: cosine between the gold candidate’s option vector and its hardest negative, by outcome (bars: 10th–90th percentile). Right: pixel RMSE between the gold crop and the chosen crop on wrong records, and to the nearest negative on correct ones (bars: quartiles). The crops differ; the vectors do not.
Table 3: Accuracy (%) on the typed-decision benchmarks used by MetaEncoder. Reported numbers are from [ 4 ] , Table 2. ImaJev-Bench: the 154 dev rows with public labels (of 173). Iris ’s flip rate is 0.00 on every split.
MSCOCO i2t
VisNews i2t
VisDial
MSCOCO t2i
VisNews t2i
WebQA
EDIS
Wiki-SS-NQ
CIRR
NIGHTS
OVEN
FashionIQ
mean
first stage (Qwen3-VL-Emb-2B)
74.6
75.4
81.5
78.0
73.1
90.1
92.6
83.7
65.1
69.2
76.5
38.8
74.9
Qwen3-VL-Reranker-8B (supervised) [ 9 ]
77.7
79.6
89.5
82.2
75.1
92.7
92.4
86.4
74.1
70.8
78.6
41.2
78.4
Qwen3.8-27B choice, zero-shot [ 9 ]
70.5
74.2
84.8
80.0
76.2
92.4
95.0
87.3
72.0
68.8
78.0
41.9
76.8
Iris -4B stage 1, zero-shot
41.4
40.2
15.4
21.6
27.2
34.4
42.1
13.5
4.4
4.9
41.6
5.5
24.4
Iris -4B mixture (retrieval-tuned)
47.6
49.4
30.4
44.8
47.0
82.0
69.3
48.8
4.5
6.0
49.3
4.6
40.3
Table 4: Reranking the top 16 of Qwen3-VL-Embedding-2B on MMEB-V2 image retrieval (hit@1, 1,000 queries per task). Iris ’s flip rate is 0.00 on every task. The zero-shot row has never seen a retrieval candidate; the tuned row is the released mixture model (§ 5.8 ), trained on 4,000 M-BEIR train queries of each of ten tasks (all but VisDial and Wiki-SS-NQ, which measure transfer) with hard negatives from the same first stage.
Figure 6: MMEB-V2 reranking per task. Retrieval tuning helps wherever the query or the candidate is text and does nothing where both are images: a per-candidate vector cannot carry a comparison with the query image.
AUROC of p(⊥)
Abstention on null steps (%)
Model
Task
Website
Domain
Task
Website
Domain
Iris -4B mixture (independent)
0.867
0.780
0.785
11.1
13.6
13.1
Iris -4B conditioned, stage 2
0.943
0.919
0.947
40.7
43.2
33.8
Iris -4B conditioned + abstention stage
0.969
0.935
0.949
79.0
77.3
77.5
Table 5: Null detection on the Mind2Web null steps (81 / 44 / 222 steps per split).
Model
Accuracy (all)
noul MAE
noul Brier
score MAE
Iris -4B ( Jeva -4B weights, no training)
53.0
0.386
0.199
0.934
Iris -4B stage 1 (OneJev-Data)
74.1
0.263
0.127
0.597
Iris -4B mixture
67.5
0.312
0.153
0.654
Table 6: Typed outputs on 2,000 held-out OneJev-Data rows.
Figure 7: Latency against K with none, half, or all of the image candidates already cached (solid: measured; dashed: Proposition 2 with the measured constants). At full reuse a request costs the state pass plus host-side assembly whatever K is.
p50 latency, measured / predicted (ms)
K
r=0
r=0.25
r=0.5
r=0.75
r=1
S(1)
16
208 / 201
166 / 164
147 / 127
135 / 90
57 / 54
3.6 ×
64
652 / 628
497 / 485
339 / 341
198 / 197
58 / 54
11 ×
256
2,342 / 2,275
1,768 / 1,720
1,208 / 1,164
645 / 609
64 / 54
37 ×
1,024
12,108 / 11,907
9,118 / 8,944
6,083 / 5,980
3,051 / 3,017
91 / 54
133 ×
2,048
23,894 / 23,894
17,835 / 17,934
11,909 / 11,974
5,933 / 6,014
125 / 54
191 ×
Table 7: Latency under candidate reuse: K image candidates of which a fraction r is already in the hybrid cache, fresh state every request; one A800, 5 repetitions. Predicted is T(r) of Proposition 2 with the measured constants. S(1) is the measured speedup at full reuse.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Backbone
Qwen3.5-4B (vision tower + language model), bf16, SDPA attention
Vision tower
frozen; patch embedding run as a matrix product (§ 3.6 )
LoRA
rank 64, α 128, dropout 0.05, on q/k/v/o and gate/up/down projections of the language model; initialised from Jeva -4B
state 16,384 (stage 1) / 4,096 (stage 2 and evaluation); candidate 2,048 / 1,024
Appendix
Table 8: Settings. Everything not listed is Jeva -4B’s default.
δ
cross-task accuracy
positive accuracy
abstention on null steps
−0.2
91.02
91.64
81.5
0 (saved decision)
91.70
92.52
79.0
+0.1
91.77
92.75
76.5
+0.24 (selected)
92.07
93.23
74.1
+0.4
91.92
93.23
71.6
cross-website at δ=0 / +0.24
86.53 / 87.02
86.9 / 87.5
77.3 / 77.3
Appendix
Table 9: Abstention margin for the released conditioned model (%).
Figure 8: Cross-task accuracy of the conditioned stage-2 checkpoints: total, on steps with a positive, and correct abstention on the 81 null steps.
Checkpoint
Rows seen
Accuracy
Positive
Null
epoch 1, step 100
3,200
86.69
91.5
12.3
epoch 1, step 150
4,800
88.33
92.0
32.1
epoch 1, step 200
6,400
87.51
91.4
27.2
epoch 2, step 50
9,351
89.75
92.4
48.1
epoch 2, step 100
10,951
89.68
91.2
65.4
epoch 2, step 150
12,551
88.71
90.8
55.6
Appendix
Table 10: Cross-task accuracy (%) of the conditioned stage-2 checkpoints, original order. From 1.2 epochs on, accuracy on steps with a positive is stable and the total is driven by the volatile abstention rate on the 81 null steps.
System One models output constrained decisions and probability distributions rather than free-form text generation. While prevailing paradigms rely on structured schema objects to encode state, intent, and candidate choices, we revisit a fully natural language-based System One interface. In this framework, both the user request and each candidate option are expressed in natural language, supported by multimodal (image and video) auxiliary inputs. We introduce MetaEncoder, which fine-tunes a pre-trained Muse-Glimmer 30B decoder into an instruction-following decision-making encoder. To scale effectively across both small closed-set (< 256) and massive open-set (millions) candidate spaces, MetaEncoder employs a bi-encoder architecture trained via unidirectional contrastive learning for request-candidate alignment. We conduct extensive evaluations across 11 benchmark suites and 190 tasks spanning multimodal decision-making, understanding (closed-set) and retrieval (open-set), highlighting where MetaEncoder beats SOTA multimodal encoders, as well as its current limits on reasoning-intensive tasks.
Interactive agents repeatedly choose among a finite set of actions, labels, or ordinal outcomes. Generating a free-form response for each choice adds latency and complicates the interface between a model and its environment. We present Intern-Decision, a family of 0.8B, 2B, and 4B multimodal decision models that return structured choices and candidate probabilities from text and optional images. Our autoregressive masked language modeling objective places one decision placeholder per field in a complete assistant JSON skeleton. Ground-truth symbols appear only in supervision, while all fields are evaluated in a single causal forward pass using the backbone's native language-model head. The 4B model achieves a 90.02% unweighted average accuracy across seven evaluation suites, compared with 88.74% for Jev v1.13.0. For a fixed 289-token request with three fields on one RTX 4090, mean latency is 44.16 ms for the 4B model and approximately 33--34 ms for the smaller models. On a separate 96-case known-distribution pilot, temperature scaling reduces expected multiclass Brier from 0.628 to 0.550 and expected calibration error from 0.213 to 0.089. A confidence-based handoff study on 111 public Jevbench-Hard items improves accuracy from 82/111 (73.87%) to 97/111 (87.39%) by routing 42 items to Atria-Dawn-Preview. We also describe qualitative game, visual interface, and browser demonstrations. These results support compact decision models as a practical component of larger agent systems.
Tong Zhu, Zhe Xu, Zefeng He +7
Shanghai AI Lab · Shanghai Jiao Tong University · Fudan University +1
System one models like Jev enable direct, low-latency decision-making. Yet recent open-source practice of predicting an answer from serialized text does not, by itself, respect the structure of a decision problem. We argue that current designs underemphasize four important structural requirements. (i) null-aware: Models should distinguish answer absence from uncertainty among candidates; (ii) permutation-invariant: Reordering alternatives should not change the decision; (iii) option-scalable: Growing candidate sets should not require redefining the output space; (iv) cacheable: Request-independent computation over recurring alternatives should be reusable across requests. We introduce Jeva, a decision-native architecture that addresses these requirements jointly. A shared encoder independently represents the request and each option, while a compact decision stack refines only the request state against immutable option memory. Symmetric aggregation yields permutation-equivariant probabilities; shared scoring accommodates variable-size candidate sets. A separate coverage estimate distinguishes answer absence from candidate competition, enabling explicit null outcomes. Request-independent option representations and layer projections support reuse across newly assembled sets. These mechanisms unify Choice, Noul, and Score outputs. Across evaluations against Jev and open source models, Jeva combines strong typed-decision performance with near-perfect null detection and no decision flips under any tested permutations. It scales to thousands of candidates and substantially reduces latency through cross-request reuse without changing decisions.