$Ψ$-0.6: Adaptive Streams of Thought in a Unified Vision-Language World Model

Khai Loong Aw; Klemen Kotar; Baihan Zhang; Yifan Liu; Lilian Chen; Atlas Kazemian; Rahul Venkatesh; Daniel Yamins

cs.AIOct 23, 2026

$Ψ$ -0.6: Adaptive Streams of Thought in a Unified Vision-Language World Model

Authors: Khai Loong Aw, Klemen Kotar, Baihan Zhang, Yifan Liu, Lilian Chen, Atlas Kazemian, Rahul Venkatesh, Daniel Yamins

Organizations: Stanford University

Abstract

Humans reason about the physical world by flexibly combining different forms of inference as a problem unfolds. Achieving this flexibility is a major open challenge in AI: capabilities often remain separated across specialized models, limiting how they can be composed and jointly adapted to a task. We introduce $Ψ$ -0.6, a unified vision-language world model that adaptively composes perception, language, geometry, and simulation within a shared reasoning process we call streams of thought. These streams interleave images, text, and explicit representations of geometry and motion, all of which the model can generate and interpret through a shared token interface. $Ψ$ -0.6 surpasses leading open-weight models zero-shot on multi-view spatial reasoning, with further gains from reasoning through geometric intermediates. It also supports counterfactual simulation: generating and interpreting imagined scenes to determine, for example, whether moving an object would obstruct a pathway. Reinforcement learning from task feedback jointly optimizes the selection, generation, and use of intermediate representations, improving visual question-answering accuracy with gains that transfer to unseen benchmarks. $Ψ$ -0.6 offers a path toward adaptive physical reasoning -- not only predicting the physical world but also choosing the representations through which to reason.

Figures & tables

Figure 1: $\Psi$ -0.6: representation-adaptive reasoning across vision, language, and world modeling. (A) Capability unification. Diverse representations share a common token interface. Different choices of supplied and predicted tokens express perception, geometry, language, and imagination within a single transformer. (B) Compositional unification. Streams of thought combine model capabilities. (C) Optimization unification. Because intermediate representations are predictions of the same model, choosing which representations to generate and use becomes a learnable action.

Figure 2: Architecture, training recipe, and prompting interface. (A) $\Psi$ -0.6 is a unified model across diverse modalities. Pretraining, mid-training, and post-training sample different sequence families; hollow tokens denote stochastically included sequence elements. Generic prompting interface: prompts specify the available information and requested inference pathway. $\Psi$ -0.6 is interactive: it can generate a view based on a user-specified camera pose, then use that generated view as context for subsequent reasoning. (B) Fine-grained control: each patch is represented by a pointer token and content tokens. Pointers specify modality and spatial and temporal locations, enabling targeted conditioning and scene edits. Probabilistic prediction: $\Psi$ -0.6 represents a distribution over compatible continuations, allowing the same prompt to yield alternative predictions through sampling.

Figure 3: Comparison to related work. (A) Six classes of related models. (B) Frontier assistants pair a VLM with separate tools. (C)

\Psi

-0.6 reads and writes all of these in one model; each pathway contrasts with a class in (A).

Figure 4: Spatial intelligence reasoning on MindCube . (A) Benchmark categories: Rotation, Around, and Among, adapted from Wang et al. [71] . (B) Two compositional-prompting routes: infer relative poses and select a real input view, or synthesize RGB at a target pose, then read an answer. (C) Accuracy for zero-shot and benchmark-trained systems; some trained comparisons use RL.

Figure 5: Counterfactual imagination of object motion. (A) Answering a counterfactual question requires imagining the outcome of a hypothetical motion. A person, like $\Psi$ -0.6, imagines the box sliding in front of the cup and answers correctly, whereas a VLM that reasons only in text does not. (B) A FlowBench-Real question on real robot data. $\Psi$ -0.6 generalizes to FlowBench-Real after training on FlowBench-Sim (Tab. 3 ). (C) Stream of Thought on FlowBench-Sim: $\Psi$ -0.6 predicts the optical flow of the instructed motion, decoding high- $P(\text{motion})$ patches first (pMotion) with text-enhanced CFG ( $2\times$ image-and-text minus $1\times$ image-only logits), then the future frame, and reasons to its answer. Bottom: FlowBench question types.

Method	Acc $\uparrow$	rEPE move $\downarrow$	rEPE move med $\downarrow$	Blank $\downarrow$	Dir ( ∘ ) $\downarrow$	$\Delta$ PSNR $\uparrow$	Oracle Input
Zero flow	0.0%	1.000	1.000	100.0%	–	0.00	–
Modular (Grounding DINO + SAM2 + Depth Pro)	50.8%	0.552	0.358	15.0%	1.4	–	Camera intrinsics
FOFPred [ 57 ]	2.4%	0.915	0.958	0.0%	25.9	–	–
Qwen-Image-Edit [ 77 ]	7.9%	2.154	1.014	2.9%	64.8	-1.63	GT frame-0 mask
IP2P [ 10 ] (plain MSE)	10.5%	2.659	1.221	2.9%	71.0	-3.35	GT frame-0 mask
IP2P [ 10 ] (SoM target)	13.7%	2.938	1.122	3.1%	64.4	-0.96	GT frame-0 mask

Table 1: World modeling ability: expert models vs.

\Psi

-0.6. “rEPE move ” is the relative endpoint error on moving pixels; static pixels are excluded. “Acc” denotes what percentage of predictions have rEPE lower than a threshold. (Definition in Appendix. A.4 ) “Blank” stands for the percentage of the model predicting zero flow. “Dir” is the 2-D angle between predicted and ground-truth flow on pixels where both move. “

\Delta

PSNR” shows the improvement of the generated simulated image after the move on moving pixels compared with zero flow.

n=622

Model	Closer	Closest_3	Contact	Count	Feasibility	L/R	Near	Occl.	Avg	Cat. Avg
Chance	25.0	5.0	50.0	25.0	50.0	50.0	50.0	50.0	35.8	38.1
Qwen3-VL-8B [ 4 ]	47.2	11.9	52.6	24.5	56.6	77.3	58.4	55.0	46.9	47.9
Qwen3.5-9B [ 55 ]	53.5	10.7	52.6	23.3	52.8	68.4	53.3	51.7	44.3	45.8
Qwen3-VL-32B [ 4 ]	41.8	17.0	38.6	20.9	67.6	81.9	55.3	62.5	48.3	48.2
Qwen3-VL-8B FlowBench-FT	89.0	36.3	93.0	57.3	88.4	98.6	88.0	72.5	76.3	77.9
$\Psi$ -0.6 Base	23.4	0.0	24.0	12.1	9.4	45.7	50.4	31.5	24.5	24.6

Table 2: Accuracy on FlowBench-Sim (%). Closest-3 requires the exact set of three. Cat. avg is the unweighted mean over the 8 categories.

Model	L/R	Near	Closer	Depth	Closest-3	Count	Occl.	Avg	Cat. Avg
Chance	50.0	50.0	26.7	50.0	7.0	25.1	50.0	37.6	37.0
Qwen3-VL-8B [ 4 ]	52.2	48.9	42.3	62.1	7.9	38.2	37.2	42.7	41.2
Qwen3.5-9B [ 55 ]	52.2	56.1	40.6	50.0	11.4	30.0	50.7	42.7	41.6
Qwen3-VL-32B [ 4 ]	58.9	49.9	40.7	50.0	19.0	43.5	39.2	44.4	43.0
Qwen-8B FlowBench-FT	70.3	80.0	61.4	62.8	58.0	29.5	31.9	59.6	56.3
$\Psi$ -0.6 Base	50.3	51.3	23.3	50.0	7.4	26.7	50.0	37.5	37.0

Table 3: Accuracy on FlowBench-Real (%). Note: No FlowBench-Real examples are used in training, so this table evaluates the generalization ability of models. Closest-3 requires the exact set of three. Cat. avg is the unweighted mean over the 7 categories.

Figure 6: Learning adaptive stream-of-thought reasoning. (A) Given a video and question, $\Psi$ -0.6 functions as both a policy – emitting a notation naming the stream of thought to use – and as a VLM, answering with those streams in context. (B) The GRPO RL loop: collect notations with the current policy, score each by probability of ground-truth answer relative to RGB-only, and train on the beneficial notations. (C) The learned $\Psi$ -0.6 policy improves video question-answering accuracy, with gains generalizing to unseen benchmarks. (D) These accuracy gains are a result of loss reductions on the ground truth.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Method	Acc $\uparrow$	rEPE move $\downarrow$	rEPE move med $\downarrow$	Blank $\downarrow$	Dir ( ∘ ) $\downarrow$	$\Delta$ PSNR $\uparrow$	Oracle Input
$\Psi$ -0.6 Default seq 32	31.8%	0.635	0.639	27.5%	14.1	+1.60	–
+ pMotion	44.2%	0.577	0.474	34.6%	6.6	+1.59	–
+ pMotion + CFG1.5	54.0%	0.463	0.363	5.6%	7.7	+1.49	–
+ pMotion + CFG2.0	56.4%	0.454	0.356	0.3%	8.1	+0.94	–
+ pMotion + CFG2.5	56.4%	0.463	0.347	0.0%	9.1	+0.25	–
+ pMotion + CFG1.5 + seq 128	54.2%	0.462	0.364	5.6%	7.8	+1.45	–

Appendix

Table 4: Ablation study of

\Psi

-0.6 models: effect of pMotion, CFG scale, and sequence length on optical flow prediction.

n=622

Element	Serialized size	Content representation
RGB patch	1 pointer + 4 codes	Learned codes for a $16\!\times\!16$ pixel patch.
Depth patch	1 pointer + 2 codes	Learned codes for normalized inverse depth on the same patch grid.
Flow patch	1 pointer + 2 codes	Learned codes for two-dimensional displacement; channels also encode the temporal gap.
Camera pose	1 pointer + 8 values	Two fields of view, three axis–angle rotation components, and three translation components.
Text segment	1 pointer + text tokens	GPT-2 byte-pair encoding, followed by an end-of-text token.

Appendix

Table 5: Token representations. Pointer tokens count toward sequence length but do not receive prediction loss. Patch sizes are in image pixels, whereas the model’s spatial coordinates index the patch grid.

Setting	Value
Transformer blocks / hidden width	32 / 4,096
Query heads / key–value heads	32 / 8
Head dimension / SwiGLU hidden width	128 / 16,384
Token vocabulary / channel embeddings	65,536 / 384
Base training sequence length	4,096 tokens
Normalization	RMSNorm, $\epsilon=10^{-5}$

Appendix

Table 6: Architecture of

\Psi

-0.6. The channel table reserves capacity beyond the modalities used in this recipe.

Stage	Steps ( $\approx$ )	Sequence mixture
RGB pretraining	0–500k	Video prediction with 2–8 RGB frames.
Geometric integration	500–700k	RGB, depth, flow, and camera-conditioned video sequences.
Language introduction	700–900k	50% vision; 50% FineWeb-Edu.
Caption alignment	900k–1.03M	25% each: vision, FineWeb-Edu, global captions, and local captions.
Instruction training	1.03–1.10M	Captioning, question answering, text control, and predictive replay; DCLM replaces FineWeb-Edu.
Instruction refinement	1.10–1.20M	More video-question training and revised caption/geometry mixtures (Table 8 ).

Appendix

Table 7: Supervised training stages for

\Psi

-0.6, with approximate cumulative step boundaries. Every stage uses batches of 512 sequences and a 4,096-token sequence budget. The first approximately 180k steps use TPU v6e; subsequent stages use TPU v5e.

Group	Share ( $\approx$ )	Data and supervision
Visual prediction	20%	Video corpus; content loss after the first RGB element.
Text-only	20%	DCLM-baseline; language-modeling loss.
Image captions	5%	COCO train and PixMo-Cap; captions and, in most notations, depth targets.
Image questions	5%	PixMo-Cap-QA, VQA v2, LLaVA-Instruct-150K, and the naturalistic Cambrian subset; answer-only loss.
Global video captions	20%	Clip-level captions with visual/geometry sequences; all content elements supervised.
Video questions	15%	OpenVid, PE-Video, and SSv2 question–answer data; answer-only loss.

Appendix

Table 8: Approximate sampling proportions during instruction refinement, rounded to whole percentages. All groups mask pointers and padding.

Component / term	Method and setting
Transformer	Decoder-only; approximately 8.05B parameters; 32 blocks; hidden width 4,096.
RMSNorm	Root mean square normalization; pre-norm residual blocks; $\epsilon=10^{-5}$ .
GQA	Grouped-query causal self-attention; 32 query heads, 8 key–value heads; head dimension 128.
Attention scaling	Scaled dot-product attention; logit scale $1/\sqrt{128}$ .
FlashAttention	Memory-efficient causal attention kernel used in TPU training.
SwiGLU	Swish-gated linear unit feed-forward network; hidden width 16,384.

Appendix

Table 9: Technical glossary: architecture, tokenization, and geometric representations. Transformer RMSNorm and quantizer LayerNorm apply to their respective networks.

Component / term	Method and setting
Next-token CE	Cross-entropy on supervised content tokens; token-mean reduction in pretraining/alignment and sequence-mean reduction in instruction training.
AdamW	Adam with decoupled weight decay; $\beta_{1}=0.9$ , $\beta_{2}=0.95$ , $\epsilon=10^{-8}$ .
Weight decay	0.1 on matrix parameters; 0 on normalization gains.
Gradient clipping	Global gradient norm capped at 1.0.
Learning-rate schedule	Peak $3\!\times\!10^{-4}$ ; 2,000-step linear warmup; constant phase; linear decay toward zero over the final 50,000 steps.
Batch / training duration	512 sequences per global batch; approximately 1.2M supervised steps for the base model.

Appendix

Table 10: Technical glossary: supervised optimization and training infrastructure. Optimizer and schedule settings refer to the transformer; the final two rows describe the independently trained visual quantizers.

Component / term	Method and setting
KV caching	Key–value caching for autoregressive patch decoding.
Sequential / parallel decoding	Inference speedup using an autoregressive patch prefix followed by concurrent decoding of remaining patches; codes within each patch are generated sequentially.
Sampling controls	Temperature, top- $k$ , and nucleus (top- $p$ ) sampling; generic defaults $T=1$ , $k=1{,}000$ , $p=1$ , with 32 sequential patches.
CFG	Classifier-free guidance; combine reference and conditional logits as $z_{\mathrm{ref}}+s(z_{\mathrm{cond}}-z_{\mathrm{ref}})$ .
Motion-based patch order	Prioritize flow patches by $1-\sum_{z\in\mathcal{Z}}p(z)$ , where $\mathcal{Z}$ contains seven no-motion codes.
Teacher-forced scoring	Mean reference-answer token cross-entropy used to score candidate notations during policy training.

Appendix

Table 11: Technical glossary: decoding controls and adaptive-policy training. Sampling values are the generic predictor defaults; policy settings summarize the procedures in § C.3 .

VLM Reasoning

Vision-language model reasoning studies how multimodal systems use visual evidence and language to answer questions, plan, and explain.

See all 853 papers

PhysMind: From Video to Executable Worlds for Training-Free Physical Reasoning
Aug 5, 2026Chen Yang, Shenxiang Zeng, Haoyang Zhao +6Counterfactual Reasoning Physical Reasoning in Video
1Tsinghua University · 2The University of Hong Kong
PhiZero: A World Model Built Around Physical Language
Jul 30, 2026Shuyao Shang, Yuqi Wang, Ruopeng Gao +4Action-Conditioned Video Generation Physical Consistency in Video Generation
NLPR, Institute of Automation, Chinese Academy of Sciences (CASIA) · Independent Researcher
Integrating Simulation and Chain-of-thought Reasoning in Multimodal-Language Models For Physical Reasoning
Oct 14, 2026YingQiao Wang, Eric Bigelow, Tomer Ullman +2Multimodal Reasoning CoT Reasoning
Department of Psychology, Harvard University · Sakana AI · Goodfire AI

Aug 5, 2026J/K move · Enter open · S save

PhysMind: From Video to Executable Worlds for Training-Free Physical Reasoning

Chen Yang, Shenxiang Zeng, Haoyang Zhao, Zhouyuan Xu+5

1Tsinghua University · 2The University of Hong Kong

Counterfactual Reasoning Physical Reasoning in Video Embodied World Models Temporal Reasoning in VLMs

Open paper AskPDF arXiv

Reliable physical reasoning from video requires understanding how objects move, interact, and respond to interventions. Existing vision-language models (VLMs) often struggle to interpret these dynamics and reason reliably about future and counterfactual outcomes. We introduce PhysMind, a training-free agentic framework that constructs one reusable, question-agnostic executable world per video. PhysMind recovers a temporally consistent dynamic scene through object segmentation, mesh reconstruction, and 6D pose tracking, then fits analytic continuous-time dynamics and latent physical parameters without unrolling a time-stepped simulator. Given a question, it inspects, continues, or edits the world and answers from the resulting trajectories and interactions. Relative to direct chain-of-thought (CoT) reasoning with the same VLM, PhysMind improves accuracy by 38.23 points on CLEVRER and 8.08 points on Physion++. On counterfactual questions, it exceeds the strongest evaluated VLM baseline, GPT-5.5, by 19.25 points.

Abstract

VLM Reasoning

Explore similar work

PhysMind: From Video to Executable Worlds for Training-Free Physical Reasoning

PhiZero: A World Model Built Around Physical Language

Integrating Simulation and Chain-of-thought Reasoning in Multimodal-Language Models For Physical Reasoning