Human intelligence relies heavily on learned intuition: recognising patterns and judging situations without explicitly unfolding every intermediate step. We introduce Bongard, an open-weight System One model that treats machine intuition as an independent capability to design and train. A T5Gemma 2 4B-4B encoder-decoder separates reading the evidence from making judgments. The encoder reads the state bidirectionally together with the question instructions, and separate decoder branches share this encoding, so many judgments about the same situation require only one reading of the state. A trained head returns probabilities over the supplied candidates without generating text. Training proceeds in three stages, from supervised judgments to semantic relationships to action outcomes, and each stage updates all 7.09 billion trainable parameters on one Blackwell GPU. Joint-embedding post-training raises accuracy on held-out rephrasings from 75.7% to 85.9%. A sandbox stage then learns outcome distributions from action rollouts and exact oracles, raising accuracy on a frozen sandbox panel from 50.6% to 64.8%. On DecisionBench, the final model reaches 78.05% accuracy over 23,900 decisions and ranks fourth of 61 systems in the public comparison. On one RTX PRO 6000, its median latency is 36 ms for short requests, and 32 questions about one state take 221 ms. Bongard demonstrates that machine intuition can be systematically trained via representation learning and outcome feedback, providing an open, efficient alternative for high-throughput decision workloads.
Figures & tables
Figure 1: Architecture of Bongard: several typed questions share a single encoding of the state, and no tokens are generated. In this example, three questions of different types concern one checkout page. The encoder reads the state once, and each question is a separate decoder sequence. The decoder processes all sequences in one batched pass, with merged self- and cross-attention over the shared state keys and values. There is no attention between questions. Hidden states at the candidate-end markers ( hi ) and the decision-end marker ( g ) feed a shared bilinear judgment head.
Primitive
Question
Criteria
Answer
Choice
Which named alternative applies?
1–255 options, each with an optional description
A probability per option, the selected option and a confidence
Noul
Is this statement true?
Optional descriptions of the true and false outcomes
The probability that the statement is true
Score
Where does the input fall on an ordered scale?
2–10 ordered level descriptions
A probability per level, the expected level and a confidence
Table 1: The three question primitives. The same head answers all of them in the same forward pass.
Same shape, causal, merged self- and cross-attention
3.21B
Tied embeddings
262,144 tokens × 2,560
0.67B
Vision tower
SigLIP ( Zhai et al., 2023 ) , 896 px, 256 tokens per image, frozen
0.42B
Projector, head
Trained, with w∈R2560 and Wc,Wg∈R256×2560
4.26M
Total
Bongard limit: 32,768 tokens per state plus question
7.51B
Table 2: Bongard 4B-4B configuration. Training updates all components except the vision tower, so 7.09B of the 7.51B parameters are trainable.
(a) Encoder attention mask
Accuracy ↑
NLL ↓
Brier ↓
Fact direction ↑
Effective rank
Bidirectional (native)
69.8%
0.677
0.219
100.0%
49.7
Forward only
64.6%
2.944
0.320
70.8%
37.9
Forward and backward heads
57.3%
1.596
0.363
62.5%
31.7
(b) Decoder modification
Held-out NLL Δ↓
Transfer NLL Δ↓
Latency Δ
Table 3: Architecture probes on T5Gemma 2 1B-1B. (a) Encoder masks with a frozen backbone and a trained head, on synthetic judgments of thresholds, negation and role binding, averaged over two seeds. Fact direction is the share of fact changes that move the truth probability in the correct direction. (b) Decoder modifications after short full-parameter training, as changes relative to the unmodified model for seeds 23 and 37. The value-residual row comes from a longer run and reports cross-task NLL as transfer.
System
Backbone and training
How the state is read
How the answer is read
Encoder–decoder
Bongard
T5Gemma 2 4B-4B; full text-model training on judgments, semantic pairs and action outcomes
Bidirectional, question-aware encoder, once per request, shared by separate decoder branches
Trained bilinear head on candidate and decision readouts, no sampling
Closed
Jev
Undisclosed backbone; RLCD described by TypeSafe
Isolated question branches (observed), state processed once (suggested by latency)
Label-token probabilities at masked slots, noisy re-reads when uncertain
Table 4: Architecture and learning choices in documented System One systems. Sources: Jev ( TypeSafe AI, 2026a ; Hume, 2026 ) , OpenJev ( OpenJev contributors, 2026 ) , Kev ( Palmer and Kev contributors, 2026 ) , JevK5 and Plumb-4B ( allebee, 2026 ; crh225, 2026 ) , AutoJev ( denis-pplx, 2026 ) , MoJev ( MoLeMo Lab, 2026 ) , CLM ( Contrastive-LM, 2026 ) , imajev ( Garg, 2026a ) , and Laya and Verdict ( Convai Innovations, 2026a ; OpenJev contributors, 2026 ) . Jev’s architectural details are inferred from API behaviour.
Figure 2: Training pipeline. A pretrained T5Gemma 2 encoder–decoder receives a judgment head and is then trained in three stages: supervised judgment training, joint-embedding predictive training and sandbox reinforcement learning. Each stage updates all trainable parameters on one GPU.
Family
Raw cosine
Centred cosine
Centred retrieval
paired / unpaired
paired / unpaired
top-1 ( N=256 )
Shared encoded state
Answer state
0.995 / 0.993
0.46 / −0.01
28.9%
Logical complement
1.000 / 0.996
0.94 / 0.05
99.6%
Inherited view
1.000 / 0.979
0.98 / 0.00
71.9%
Separately encoded
Table 5: Geometry of the decision readout g after stage 1, from forward passes over 256 training origins per family. Raw cosine between unrelated readouts is already 0.93–0.996. Centring each side on its mean exposes the true alignment. Logical complements and inherited views are already aligned. Separately encoded content is not, except for paraphrase pairs, whose two sentences share most of their words.
Figure 3: JEPA without a next-token anchor: the stage-2 objective. (a) Separately encoded content and world pairs are centred per pack and trained with a symmetric InfoNCE loss against the other origins in the pack. (b) The anchor is the decision readout of a multiple-choice question. It is contrasted with stop-gradient readouts of a correct “yes” event and a wrong “no” event under the same evidence, so their shared fingerprint cancels.
Measure
Stage 1
Stage 2
Within-source retrieval top-1 on never-paired origins ↑
Query → passage
15%
98%
Question → answer content
4%
86%
Image → content
7%
67%
World prediction / causal prediction
4% / 1%
35% / 31%
Answer state: correct event nearer than wrong event
23%
77%
Table 6: Effect of stage 2. The differences combine the alignment terms, the alternative views and continued training. Retrieval is measured within one source, so matching on source fingerprints alone cannot raise the score.
Component
Implementation
Feed-forward blocks
68 fused Transformer Engine blocks (RMSNorm, gate and up projections, GELU gating, down projection) with NVFP4 matrix multiplications ( NVIDIA, 2026 ; NVIDIA, 2025 )
Attention projections
272 projections as rowwise-scaled FP8 matrix multiplications ( Micikevicius et al., 2022 ; PyTorch Team, 2026 )
Attention
FlashAttention-4 in variable-length mode ( Zadouri et al., 2026 ; Dao et al., 2022 ) , FlexAttention for sliding-window sequences ( Dong et al., 2024 ) , BF16 arithmetic
Storage and optimiser
BF16 parameters and gradients with no FP32 master copy, and 8-bit AdamW ( Dettmers et al., 2022 ) with stochastic rounding ( Gupta et al., 2015 )
Batching and memory
Unpadded packs of 24,576–28,672 tokens (131,072 tokens per update), state keys and values stored once per record, and recomputation ( Chen et al., 2016 ) only for records above the pack limit
Compilation
torch.compile regions ( Ansel and others, 2024 ) , and quantised weights reused until the parameters change
Table 7: Training stack of stages 1 and 2 on a single B200 (PyTorch 2.14, CUDA 13.0).
JevBench v1.4 ( Standhartinger and contributors, 2026 ) , public items
accuracy, easy / standard
100% / 97.2%
Imajev-4B 100% / 99.0%
typed-decisions ( LocalLLaMA, 2026 ) , test
KL from gold, Brier
0.256, 0.132
Jev 1.13 1.442, 0.148
accuracy
0.594
Jev 1.13 0.727, Jeff-Gemma4-E2B 0.561
behavior-benchmark ( Respan AI, 2026 )
F1 of present , core / multilingual
0.458 / 0.668
no public results
Table 8: Public System One benchmarks. DecisionBench, typed-decisions and ImajevBench references come from their leaderboards and dataset cards on 29 September 2026, and JevBench references from its v1.4.2.2 results. On typed-decisions, the gold is a teacher distribution, so the scores measure agreement with that teacher.
Figure 4: Same-item comparison with Jev 1.13. Both models answered the same 49,291 items from 24 public benchmarks, with identical states, instructions and candidates. Jev answered through TypeSafe’s API on 30 September 2026. Dots mark accuracy, and the benchmarks are sorted by Bongard’s lead. Table 12 lists the item sets.
Figure 5: Asked for a fair random choice, chat models repeat one answer, while Bongard returns the uniform distribution. Each chat model answered each question 40 times at its default temperature. For Jev and Bongard, the bar is the largest returned probability. The dashed line marks a fair choice.
Figure 6: Time and cost per decision. Bongard runs on one RTX PRO 6000 in BF16, and its cost assumes $1.79 per GPU hour. Latencies of the other systems come from the DecisionBench leaderboard and include network time. Costs come from the JevBench results, as dollars per 1,000 decisions times 1,000.
Figure 7: Latency of shared-state requests. One request asks N questions about the same JevBench state, and the encoder reads the state once. Ticks mark N separate single-question requests, at N times the single-question latency. Thirty-two questions take 221 ms in one request instead of 1.16 s, so the time per decision falls from 36 to 6.9 ms. One RTX PRO 6000, BF16.
Game
Games
Result
2048 ( Cirulli, 2014 )
12
reaches the 2048 tile in 9 games and 4096 in 2, best score 55,140
Tetris
5
never tops out within the limit of 1,500 pieces, 596–599 rows cleared per game
Othello
32 + 32
wins 24 of 32 against a greedy engine and 4 of 32 against a two-ply positional engine, one of them 13–0 in nine moves
Minesweeper, 8 × 8
64
clears the board in 42 games
Snake, 8 × 8
32
grows to 32 cells, half of the board
Doom
32
10.3 kills per 30-second episode on average, 20 at best
Table 9: Games played by Bongard. The game engine lists the legal moves and states the facts of each move in the state. The question gives a one-line strategy in plain English, and the model chooses every move. For 2048 and Tetris, the facts include the engine’s rating of each move. Doom is ViZDoom Defend the Center ( Kempka et al., 2016 ) , and the model reads a one-line description of the scene every 0.2 s.
Evaluation set
Stage 2
Final
Hard-style decisions (109)
34.9%
50.5%
Sealed-style decisions (200)
35.0%
50.5%
Frontier-style decisions (199)
45.2%
73.9%
DecisionBench, balanced sample (2,580)
64.3%
68.7%
RouterBench routing between cheap and strong models (600)
56.2%
62.5%
JevBench public standard items (72)
93.1%
97.2%
Table 10: Stage 2 (JEPA) and the final model on evaluation sets excluded from stage 3 training. The style-based sets are internal panels, and the public-benchmark samples cover multiple tasks.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Accuracy (%)
Benchmark
Item set
Items
Bongard
Jev
Banking77 ( Casanueva et al., 2020 )
test
3,076
93.1
79.8
Support-ticket triage ( Tobi-Bueck, 2025 )
Laya
400
43.0
35.8
PAWS ( Zhang et al., 2019 )
test
8,000
91.1
84.9
SST-5 ( Socher et al., 2013 )
Laya
600
58.2
57.5
Model routing ( Cobbe et al., 2021 ; Austin et al., 2021 ; Zhang et al., 2015 )
Laya
399
98.0
97.7
Appendix
Table 12: Item sets and accuracy of the same-item comparison, in the order of Figure 4 . Test and validation splits are complete.
Implicit reasoning in language models studies how models solve problems through internal computation rather than, or alongside, written chains of thought.
Top researchers have good intuition, but do language models have as good intuition about model training as top AI researchers? To measure model intuition of LLMs and humans, we introduce the ArchitectureIQ benchmark. Each question presents a synthetic dataset and several training recipes, and the test-taker is asked to predict the recipe yielding the best test metric. Overall, we find that LLMs' model intuition is good but has four limitations: (1) The intuition is imperfect, or even sub-human in some cases. Frontier models achieve around 76% accuracy (random choice 33%) vs best human researcher (66.0%), yet remain far from perfect. For architecture-only questions, best human achieves 65% while GPT-6 Astra only has 38%. (2) The intuition is empirical, not structured, supported by the fact that more CoT compute does not lead to substantial improvement. Unlike math, we still lack a "Science of AI" language that enables structured reasoning on AI. (3) The intuition is not maximally condensed, and can be further compressed into a knoledge base. Our constructed knowledge base with only 20 items yields large gains for weak models: GPT-4o equipped with the accumulated knowledge almost matches the performance of Claude Opus 5. (4) The intuition is insensitive to dataset properties, but the best model should in general depend on data properties. This suggests that data is the real "dark matter" in AI -- LLMs (so do human researchers) understand too little about data, even less than model architectures.
Zirui Ren, Shaoyang Guo, Chencheng Tang +9
MetaCircle (元环智能) · Tsinghua University · Peking University +4
This paper asks whether a bounded neural architecture can exhibit a meaningful division of labor between intuition and deliberation on a classic 64-item syllogistic reasoning benchmark. More broadly, the benchmark is relevant to ongoing debates about world models and multi-stage reasoning in AI. It provides a controlled setting for testing whether a learned system can develop structured internal computation rather than only one-shot associative prediction. Experiment 1 evaluates a direct neural baseline for predicting full 9-way human response distributions under 5-fold cross-validation. Experiment 2 introduces a bounded dual-path architecture with separate intuition and deliberation pathways, motivated by computational mental-model theory (Khemlani & Johnson-Laird, 2022). Under cross-validation, bounded intuition reaches an aggregate correlation of r = 0.7272, whereas bounded deliberation reaches r = 0.8152, and the deliberation advantage is significant across folds (p = 0.0101). The largest held-out gains occur for NVC, Eca, and Oca, suggesting improved handling of rejection responses and c-a conclusions. A canonical 80:20 interpretability run and a five-seed stability sweep further indicate that the deliberation pathway develops sparse, differentiated internal structure, including an Oac-leaning state, a dominant workhorse state, and several weakly used or unused states whose exact indices vary across runs. These findings are consistent with reasoning-like internal organization under bounded conditions, while stopping short of any claim that the model reproduces full sequential processes of model construction, counterexample search, and conclusion revision.
Laurence Anthony
School of Creative Science and Engineering Faculty of Science and Engineering Waseda University
Decision needs vary with each application, language, and deployment environment. The Decision Model Factory turns those needs into specialized, sovereign models on demand. A brief recipe guides the training, calibration, verification, and evaluation of a selected backbone, delivering a deployable decision model under the operator's control. The same workflow creates a new specialist whenever the requirements change. Across Japanese decision tasks, compact deployment, and long-context passage ranking, five factory-built models improve upon their backbones; three surpass the strongest results in a fixed public Japanese benchmark comparison. A dedicated runner computes only permitted-answer probabilities and reuses shared context, more than doubling serving speed on matched multi-question workloads with essentially unchanged decision scores. Together, the factory and runner make fast, task-specific decision models practical to build and deploy whenever needed.