Humans reason about the physical world by flexibly combining different forms of inference as a problem unfolds. Achieving this flexibility is a major open challenge in AI: capabilities often remain separated across specialized models, limiting how they can be composed and jointly adapted to a task. We introduce Ψ-0.6, a unified vision-language world model that adaptively composes perception, language, geometry, and simulation within a shared reasoning process we call streams of thought. These streams interleave images, text, and explicit representations of geometry and motion, all of which the model can generate and interpret through a shared token interface. Ψ-0.6 surpasses leading open-weight models zero-shot on multi-view spatial reasoning, with further gains from reasoning through geometric intermediates. It also supports counterfactual simulation: generating and interpreting imagined scenes to determine, for example, whether moving an object would obstruct a pathway. Reinforcement learning from task feedback jointly optimizes the selection, generation, and use of intermediate representations, improving visual question-answering accuracy with gains that transfer to unseen benchmarks. Ψ-0.6 offers a path toward adaptive physical reasoning -- not only predicting the physical world but also choosing the representations through which to reason.
Figures & tables
Figure 1: Ψ -0.6: representation-adaptive reasoning across vision, language, and world modeling. (A) Capability unification. Diverse representations share a common token interface. Different choices of supplied and predicted tokens express perception, geometry, language, and imagination within a single transformer. (B) Compositional unification. Streams of thought combine model capabilities. (C) Optimization unification. Because intermediate representations are predictions of the same model, choosing which representations to generate and use becomes a learnable action.
Figure 2: Architecture, training recipe, and prompting interface. (A) Ψ -0.6 is a unified model across diverse modalities. Pretraining, mid-training, and post-training sample different sequence families; hollow tokens denote stochastically included sequence elements. Generic prompting interface: prompts specify the available information and requested inference pathway. Ψ -0.6 is interactive: it can generate a view based on a user-specified camera pose, then use that generated view as context for subsequent reasoning. (B) Fine-grained control: each patch is represented by a pointer token and content tokens. Pointers specify modality and spatial and temporal locations, enabling targeted conditioning and scene edits. Probabilistic prediction: Ψ -0.6 represents a distribution over compatible continuations, allowing the same prompt to yield alternative predictions through sampling.
Figure 3: Comparison to related work. (A) Six classes of related models. (B) Frontier assistants pair a VLM with separate tools. (C) Ψ -0.6 reads and writes all of these in one model; each pathway contrasts with a class in (A).
Figure 4: Spatial intelligence reasoning on MindCube . (A) Benchmark categories: Rotation, Around, and Among, adapted from Wang et al. [71] . (B) Two compositional-prompting routes: infer relative poses and select a real input view, or synthesize RGB at a target pose, then read an answer. (C) Accuracy for zero-shot and benchmark-trained systems; some trained comparisons use RL.
Figure 5: Counterfactual imagination of object motion. (A) Answering a counterfactual question requires imagining the outcome of a hypothetical motion. A person, like Ψ -0.6, imagines the box sliding in front of the cup and answers correctly, whereas a VLM that reasons only in text does not. (B) A FlowBench-Real question on real robot data. Ψ -0.6 generalizes to FlowBench-Real after training on FlowBench-Sim (Tab. 3 ). (C) Stream of Thought on FlowBench-Sim: Ψ -0.6 predicts the optical flow of the instructed motion, decoding high- P(motion) patches first (pMotion) with text-enhanced CFG ( 2× image-and-text minus 1× image-only logits), then the future frame, and reasons to its answer. Bottom: FlowBench question types.
Method
Acc ↑
rEPE move ↓
rEPE move med ↓
Blank ↓
Dir ( ∘ ) ↓
Δ PSNR ↑
Oracle Input
Zero flow
0.0%
1.000
1.000
100.0%
–
0.00
–
Modular (Grounding DINO + SAM2 + Depth Pro)
50.8%
0.552
0.358
15.0%
1.4
–
Camera intrinsics
FOFPred [ 57 ]
2.4%
0.915
0.958
0.0%
25.9
–
–
Qwen-Image-Edit [ 77 ]
7.9%
2.154
1.014
2.9%
64.8
-1.63
GT frame-0 mask
IP2P [ 10 ] (plain MSE)
10.5%
2.659
1.221
2.9%
71.0
-3.35
GT frame-0 mask
IP2P [ 10 ] (SoM target)
13.7%
2.938
1.122
3.1%
64.4
-0.96
GT frame-0 mask
Table 1: World modeling ability: expert models vs. Ψ -0.6. “rEPE move ” is the relative endpoint error on moving pixels; static pixels are excluded. “Acc” denotes what percentage of predictions have rEPE lower than a threshold. (Definition in Appendix. A.4 ) “Blank” stands for the percentage of the model predicting zero flow. “Dir” is the 2-D angle between predicted and ground-truth flow on pixels where both move. “ Δ PSNR” shows the improvement of the generated simulated image after the move on moving pixels compared with zero flow. n=622 .
Model
Closer
Closest_3
Contact
Count
Feasibility
L/R
Near
Occl.
Avg
Cat. Avg
Chance
25.0
5.0
50.0
25.0
50.0
50.0
50.0
50.0
35.8
38.1
Qwen3-VL-8B [ 4 ]
47.2
11.9
52.6
24.5
56.6
77.3
58.4
55.0
46.9
47.9
Qwen3.5-9B [ 55 ]
53.5
10.7
52.6
23.3
52.8
68.4
53.3
51.7
44.3
45.8
Qwen3-VL-32B [ 4 ]
41.8
17.0
38.6
20.9
67.6
81.9
55.3
62.5
48.3
48.2
Qwen3-VL-8B FlowBench-FT
89.0
36.3
93.0
57.3
88.4
98.6
88.0
72.5
76.3
77.9
Ψ -0.6 Base
23.4
0.0
24.0
12.1
9.4
45.7
50.4
31.5
24.5
24.6
Table 2: Accuracy on FlowBench-Sim (%). Closest-3 requires the exact set of three. Cat. avg is the unweighted mean over the 8 categories.
Model
L/R
Near
Closer
Depth
Closest-3
Count
Occl.
Avg
Cat. Avg
Chance
50.0
50.0
26.7
50.0
7.0
25.1
50.0
37.6
37.0
Qwen3-VL-8B [ 4 ]
52.2
48.9
42.3
62.1
7.9
38.2
37.2
42.7
41.2
Qwen3.5-9B [ 55 ]
52.2
56.1
40.6
50.0
11.4
30.0
50.7
42.7
41.6
Qwen3-VL-32B [ 4 ]
58.9
49.9
40.7
50.0
19.0
43.5
39.2
44.4
43.0
Qwen-8B FlowBench-FT
70.3
80.0
61.4
62.8
58.0
29.5
31.9
59.6
56.3
Ψ -0.6 Base
50.3
51.3
23.3
50.0
7.4
26.7
50.0
37.5
37.0
Table 3: Accuracy on FlowBench-Real (%). Note: No FlowBench-Real examples are used in training, so this table evaluates the generalization ability of models. Closest-3 requires the exact set of three. Cat. avg is the unweighted mean over the 7 categories.
Figure 6: Learning adaptive stream-of-thought reasoning. (A) Given a video and question, Ψ -0.6 functions as both a policy – emitting a notation naming the stream of thought to use – and as a VLM, answering with those streams in context. (B) The GRPO RL loop: collect notations with the current policy, score each by probability of ground-truth answer relative to RGB-only, and train on the beneficial notations. (C) The learned Ψ -0.6 policy improves video question-answering accuracy, with gains generalizing to unseen benchmarks. (D) These accuracy gains are a result of loss reductions on the ground truth.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Method
Acc ↑
rEPE move ↓
rEPE move med ↓
Blank ↓
Dir ( ∘ ) ↓
Δ PSNR ↑
Oracle Input
Ψ -0.6 Default seq 32
31.8%
0.635
0.639
27.5%
14.1
+1.60
–
+ pMotion
44.2%
0.577
0.474
34.6%
6.6
+1.59
–
+ pMotion + CFG1.5
54.0%
0.463
0.363
5.6%
7.7
+1.49
–
+ pMotion + CFG2.0
56.4%
0.454
0.356
0.3%
8.1
+0.94
–
+ pMotion + CFG2.5
56.4%
0.463
0.347
0.0%
9.1
+0.25
–
+ pMotion + CFG1.5 + seq 128
54.2%
0.462
0.364
5.6%
7.8
+1.45
–
Appendix
Table 4: Ablation study of Ψ -0.6 models: effect of pMotion, CFG scale, and sequence length on optical flow prediction. n=622 .
Element
Serialized size
Content representation
RGB patch
1 pointer + 4 codes
Learned codes for a 16×16 pixel patch.
Depth patch
1 pointer + 2 codes
Learned codes for normalized inverse depth on the same patch grid.
Flow patch
1 pointer + 2 codes
Learned codes for two-dimensional displacement; channels also encode the temporal gap.
Camera pose
1 pointer + 8 values
Two fields of view, three axis–angle rotation components, and three translation components.
Text segment
1 pointer + text tokens
GPT-2 byte-pair encoding, followed by an end-of-text token.
Appendix
Table 5: Token representations. Pointer tokens count toward sequence length but do not receive prediction loss. Patch sizes are in image pixels, whereas the model’s spatial coordinates index the patch grid.
Setting
Value
Transformer blocks / hidden width
32 / 4,096
Query heads / key–value heads
32 / 8
Head dimension / SwiGLU hidden width
128 / 16,384
Token vocabulary / channel embeddings
65,536 / 384
Base training sequence length
4,096 tokens
Normalization
RMSNorm, ϵ=10−5
Appendix
Table 6: Architecture of Ψ -0.6. The channel table reserves capacity beyond the modalities used in this recipe.
Stage
Steps ( ≈ )
Sequence mixture
RGB pretraining
0–500k
Video prediction with 2–8 RGB frames.
Geometric integration
500–700k
RGB, depth, flow, and camera-conditioned video sequences.
Language introduction
700–900k
50% vision; 50% FineWeb-Edu.
Caption alignment
900k–1.03M
25% each: vision, FineWeb-Edu, global captions, and local captions.
Instruction training
1.03–1.10M
Captioning, question answering, text control, and predictive replay; DCLM replaces FineWeb-Edu.
Instruction refinement
1.10–1.20M
More video-question training and revised caption/geometry mixtures (Table 8 ).
Appendix
Table 7: Supervised training stages for Ψ -0.6, with approximate cumulative step boundaries. Every stage uses batches of 512 sequences and a 4,096-token sequence budget. The first approximately 180k steps use TPU v6e; subsequent stages use TPU v5e.
Group
Share ( ≈ )
Data and supervision
Visual prediction
20%
Video corpus; content loss after the first RGB element.
Text-only
20%
DCLM-baseline; language-modeling loss.
Image captions
5%
COCO train and PixMo-Cap; captions and, in most notations, depth targets.
Image questions
5%
PixMo-Cap-QA, VQA v2, LLaVA-Instruct-150K, and the naturalistic Cambrian subset; answer-only loss.
Global video captions
20%
Clip-level captions with visual/geometry sequences; all content elements supervised.
Video questions
15%
OpenVid, PE-Video, and SSv2 question–answer data; answer-only loss.
Appendix
Table 8: Approximate sampling proportions during instruction refinement, rounded to whole percentages. All groups mask pointers and padding.
Component / term
Method and setting
Transformer
Decoder-only; approximately 8.05B parameters; 32 blocks; hidden width 4,096.
RMSNorm
Root mean square normalization; pre-norm residual blocks; ϵ=10−5 .
Memory-efficient causal attention kernel used in TPU training.
SwiGLU
Swish-gated linear unit feed-forward network; hidden width 16,384.
Appendix
Table 9: Technical glossary: architecture, tokenization, and geometric representations. Transformer RMSNorm and quantizer LayerNorm apply to their respective networks.
Component / term
Method and setting
Next-token CE
Cross-entropy on supervised content tokens; token-mean reduction in pretraining/alignment and sequence-mean reduction in instruction training.
AdamW
Adam with decoupled weight decay; β1=0.9 , β2=0.95 , ϵ=10−8 .
Weight decay
0.1 on matrix parameters; 0 on normalization gains.
Gradient clipping
Global gradient norm capped at 1.0.
Learning-rate schedule
Peak 3×10−4 ; 2,000-step linear warmup; constant phase; linear decay toward zero over the final 50,000 steps.
Batch / training duration
512 sequences per global batch; approximately 1.2M supervised steps for the base model.
Appendix
Table 10: Technical glossary: supervised optimization and training infrastructure. Optimizer and schedule settings refer to the transformer; the final two rows describe the independently trained visual quantizers.
Component / term
Method and setting
KV caching
Key–value caching for autoregressive patch decoding.
Sequential / parallel decoding
Inference speedup using an autoregressive patch prefix followed by concurrent decoding of remaining patches; codes within each patch are generated sequentially.
Sampling controls
Temperature, top- k , and nucleus (top- p ) sampling; generic defaults T=1 , k=1,000 , p=1 , with 32 sequential patches.
CFG
Classifier-free guidance; combine reference and conditional logits as zref+s(zcond−zref) .
Motion-based patch order
Prioritize flow patches by 1−∑z∈Zp(z) , where Z contains seven no-motion codes.
Teacher-forced scoring
Mean reference-answer token cross-entropy used to score candidate notations during policy training.
Appendix
Table 11: Technical glossary: decoding controls and adaptive-policy training. Sampling values are the generic predictor defaults; policy settings summarize the procedures in § C.3 .
Reliable physical reasoning from video requires understanding how objects move, interact, and respond to interventions. Existing vision-language models (VLMs) often struggle to interpret these dynamics and reason reliably about future and counterfactual outcomes. We introduce PhysMind, a training-free agentic framework that constructs one reusable, question-agnostic executable world per video. PhysMind recovers a temporally consistent dynamic scene through object segmentation, mesh reconstruction, and 6D pose tracking, then fits analytic continuous-time dynamics and latent physical parameters without unrolling a time-stepped simulator. Given a question, it inspects, continues, or edits the world and answers from the resulting trajectories and interactions. Relative to direct chain-of-thought (CoT) reasoning with the same VLM, PhysMind improves accuracy by 38.23 points on CLEVRER and 8.08 points on Physion++. On counterfactual questions, it exceeds the strongest evaluated VLM baseline, GPT-5.5, by 19.25 points.
Chen Yang, Shenxiang Zeng, Haoyang Zhao +6
1Tsinghua University · 2The University of Hong Kong
We introduce PhiZero, a physical world model built around physical language, a compact discrete representation of world-state transitions. Existing physical world models typically predict future videos directly in pixel space, leaving the underlying world dynamics implicit within high-dimensional visual predictors. Motivated by humans' ability to abstract predictive structure from visual experience and organize it in natural language for explicit reasoning, we learn physical language from in-the-wild videos through self-supervision and use it to explicitly reason about how the physical world evolves. Accordingly, PhiZero adopts a reason-then-render paradigm: it first infers future world evolution as a physical-language sequence and then renders the inferred transitions into videos. Extensive experiments across generation and understanding benchmarks validate the ability of PhiZero to model physically coherent world evolution. We further show its potential for realistic and interactive world modeling, fine-grained action-conditioned simulation, and zero-shot motion transfer.
Shuyao Shang, Yuqi Wang, Ruopeng Gao +4
NLPR, Institute of Automation, Chinese Academy of Sciences (CASIA) · Independent Researcher
Multimodal language models often need to reason about the physical dynamics of the world in order to answer questions appropriately. Various methods have emerged for solving physical reasoning tasks, and in particular, two leading approaches are Chain-of-Thought reasoning and physics-grounded simulation. However, relatively little is known about which particular problems can be solved by each approach. Some tasks may benefit from computationally expensive generative video simulations, whereas others may be solved equally well (or even better) with language-based reasoning. Our work addresses this by systematically comparing simulation and language reasoning across a variety of multimodal tasks involving physical reasoning, such as answering questions about fluid dynamics and elasticity. We draw a parallel with this to resource-rational theories of human cognition, which propose that people optimally trade off simulation and reasoning depending on task demands. We present a novel cognitively-inspired hybrid model which optimally trades off simulation and reasoning to improve model performance and computational efficiency. Our model is able to decide the appropriate method to solve a physical reasoning problem at test time, and finds an optimal balance between accuracy and cost. Further, our model can be trained on a smaller vision-language model that is only 4B parameters and deployed to improve performance on much bigger base models with >3T parameters like Gemini 3 Pro.
YingQiao Wang, Eric Bigelow, Tomer Ullman +2
Department of Psychology, Harvard University · Sakana AI · Goodfire AI