Phi-4-reasoning-vision-15B Technical Report

Jyoti Aneja; Michael Harrison; Neel Joshi; Tyler LaBonte; John Langford; Eduardo Salinas

cs.AIMar 4, 2026

Phi-4-reasoning-vision-15B Technical Report

Authors: Jyoti Aneja, Michael Harrison, Neel Joshi, Tyler LaBonte, John Langford, Eduardo Salinas

Organizations: Microsoft Research

Abstract

We present Phi-4-reasoning-vision-15B, a compact open-weight multimodal reasoning model, and share the motivations, design choices, experiments, and learnings that informed its development. Our goal is to contribute practical insight to the research community on building smaller, efficient multimodal reasoning models and to share the result of these learnings as an open-weight model that is good at common vision and language tasks and excels at scientific and mathematical reasoning and understanding user interfaces. Our contributions include demonstrating that careful architecture choices and rigorous data curation enable smaller, open-weight multimodal models to achieve competitive performance with significantly less training and inference-time compute and tokens. The most substantial improvements come from systematic filtering, error correction, and synthetic augmentation -- reinforcing that data quality remains the primary lever for model performance. Systematic ablations show that high-resolution, dynamic-resolution encoders yield consistent improvements, as accurate perception is a prerequisite for high-quality reasoning. Finally, a hybrid mix of reasoning and non-reasoning data with explicit mode tokens allows a single model to deliver fast direct answers for simpler tasks and chain-of-thought reasoning for complex problems.

Figures & tables

Figure 1: Phi-4-reasoning-vision-15B can help with a wide range of everyday tasks, from writing travel captions and interpreting receipts to reading garment care instructions.

Figure 2: Phi-4-reasoning-vision-15B presents a compelling option compared to existing models, pushing the Pareto frontier of the trade-off between accuracy and compute costs. We achieve competitive performance with much slower models that require more time and tokens, and higher accuracy than similarly fast models. These values were computed by averaging accuracy, time, and output token counts for a subset of 4 benchmarks: ChartQA TEST , MathVista MINI , MMMU VAL , and ScreenSpot_v2.

Figure 3: Overview of the Phi-4-reasoning-vision-15B mid-fusion architecture. Images are processed by a SigLIP-2 vision encoder and projected into the language embedding space via a cross-modality projector (MLP). The resulting visual “soft” tokens are interleaved with text tokens and fed into the Phi-4-Reasoning language model.

Method	Max Tokens	MathVista	ScreenSpot	ScreenSpot-Pro	V*Bench
Dynamic-S 2	3096	42.9	78.4	9.4	52.9
Multi-crop	3096	43.4	67.8	5.4	51.8
Multi-crop with S 2	2048	43.4	79.1	10.6	57.1
Dynamic resolution	2048	45.2	81.5	9.2	51.3
Dynamic resolution	3600	44.9	79.7	17.5	56.0

Table 1: Results with different resolution handling approaches on MathVista ( Lu et al., 2024 ) , ScreenSpot ( Cheng et al., 2024 ) , ScreenSpot-Pro ( Li et al., 2025 ) , and V*Bench ( Wu and Xie, 2023 ) . We have bolded the top two configurations on each benchmark. These experiments are done on a smaller, 5B, variation of our model created for testing purposes.

Stage	MLP	Vision Encoder	LLM	Data
1. MLP Pretraining	✓	×	×	Image–text alignment
2. Instruction Tuning	✓	✓	✓	Single-image instruction tuning
3. Long Context & RAI	✓	✓	✓	Long content, multi-image, RAI

Table 2: Training recipe for Phi-4-reasoning-vision-15B. Trainable modules are indicated with ✓; frozen modules with ×.

Hyperparameter	Stage 1	Stage 2	Stage 3
Learning rate	$1\times 10^{-3}$	$2\times 10^{-5}$	$7\times 10^{-7}$
LR schedule	Cosine	Cosine w/ min LR	Cosine w/ min LR
Min LR ratio	–	0.1	0.1
Warmup	3% of steps	500 steps	500 steps
Weight decay	0	$10^{-4}$	$10^{-4}$
Adam $(\beta_{1},\beta_{2})$	(0.9, 0.999)	(0.9, 0.95)	(0.9, 0.95)

Table 3: Training hyperparameters by stage.

Figure 4: Training data composition and examples for the Stage 2 training of Phi-4-reasoning-vision-15B. The Stage 3 data is designed to have a similar composition.

General	Math	CUA	Total	MMMU-CoT	MathVista	ScreenSpot-V2
1M	150K	450K	1.6M	44.0	37.4	48.2
1M	150K	850K	2.0M	44.1	37.3	60.0
1M	450K	450K	1.9M	45.3	36.0	48.3
1M	450K	850K	2.3M	43.4	38.9	63.1
1M	150K	150K	1.3M	44.2	36.9	29.8
1M	150K	250K	1.4M	45.4	37.4	37.7

Table 4: Varying the ratios of math and CUA data. Increasing math data by 3

\times

while keeping computer-use data constant improves both math and computer-use benchmarks.

Figure 5: Phi-4-reasoning-vision-15B can interpret sequences of images, here reasoning about the changing appearance of Saturn’s rings across multiple frames.

Figure 6: Phi-4-reasoning-vision-15B excels at math and science reasoning, correctly solving a multi-part spring-mass physics problem presented with diagrams.

Figure 7: Phi-4-reasoning-vision-15B can help with written math problems, identifying a sign error in a handwritten quadratic formula solution and providing a corrected step-by-step derivation.

Panel of Figure 8 — Figure 8: Phi-4-reasoning-vision-15B can help navigate computer UIs, grounding interactive elements on desktop interfaces and localizing objects across dense visual layouts.

Benchmark	Phi-4-reasoning-vision-15B	Phi-4- reasoning- vision-15B –force nothink	Phi-4- mm- instruct	Kimi-VL- A3B- Instruct	gemma- 3- 12b-it	Qwen3- VL-8B- Instruct -4K	Qwen3- VL-8B- Instruct -32K	Qwen3- VL-32B- Instruct -4K	Qwen3- VL-32B- Instruct -32K
AI2D TEST	84.8	84.7	68.6	84.6	80.4	82.7	83	84.8	85
ChartQA TEST	83.3	76.5	23.5	87	39	83.1	83.2	84.3	84
HallusionBench	64.4	63.1	56	65.2	65.3	73.5	74.1	74.4	74.9
MathVerse MINI	44.9	43.8	32.4	41.7	29.8	54.5	57.4	64.2	64.2
MathVision MINI	36.2	34.2	20	28.3	31.9	45.7	50	54.3	60.5
MathVista MINI	75.2	68.7	50.5	67.1	57.4	77.1	76.4	82.5	81.8

Table 5: Accuracy comparisons relative to popular open-weight, non-thinking models.

Benchmark	Phi-4-reasoning-vision-15B	Phi-4- reasoning- vision-15B –force thinking	Kimi-VL- A3B- Thinking	gemma- 3-12b-it	Qwen3- VL-8B- Thinking -4K	Qwen3- VL-8B- Thinking -40K	Qwen3- VL-32B- Thinking -4K	Qwen3- VL-32B- Thinking -40K
AI2D TEST	84.8	79.7	81.2	80.4	83.5	83.9	86.9	87.2
ChartQA TEST	83.3	82.9	73.3	39	78	78.6	78.5	79.1
HallusionBench	64.4	63.9	70.6	65.3	71.6	73	76.4	76.6
MathVerse MINI	44.9	53.1	61	29.8	67.3	73.3	78.3	78.2
MathVision MINI	36.2	36.2	50.3	31.9	43.1	50.7	60.9	58.6
MathVista MINI	75.2	74.1	78.6	57.4	77.7	79.5	83.9	83.8

Table 6: Accuracy comparisons relative to popular open-weight, thinking models.

Evaluation	Description	Defect Rate
Text to Text Safety	Automated content safety evaluation measuring safety policies	1.4%
Image to Text Safety	Automated content safety evaluation measuring safety policies	4.5%

Table 15

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Stage	Category	Datasets
Stage 1: MLP Pretraining	Image–Text Alignment	Bunny [ He et al., 2024 ]
Stage 2: Single-Image Instruction Tuning	Caption	Bunny [ He et al., 2024 ] Recaptioned, Pixmo [ Deitke et al., 2024 ] , LLaVA-OneVision [ Li et al., 2024a ]
	Diagram & Chart QA	LLaVA-OneVision [ Li et al., 2024a ] , CoSyn [ Yang et al., 2025 ]
	Document QA	Docmatix [ HuggingFaceM4 Team, 2024a ] , LLaVA-OneVision [ Li et al., 2024a ]
	Object Detection	Open Images [ Kuznetsova et al., 2020 ]
	OCR	LLaVA-OneVision [ Li et al., 2024a ] , IAM [ Marti and Bunke, 2002 ]

Appendix

Table 7: Open-Source Training Data Sources for Stages 1–3.

VLM Reasoning

Vision-language model reasoning studies how multimodal systems use visual evidence and language to answer questions, plan, and explain.

See all 853 papers

Optical Reasoning: Rethinking Images as an Expressive Reasoning Medium Beyond Text
Jun 8, 2026Yutong Bian, Dongjie Cheng, Heming Xia +2Visual Reasoning Multimodal Reasoning
The Hong Kong Polytechnic University
MMVistaReason: Toward Open-Data and Post-Training Recipes for Multimodal Reasoning
Oct 1, 2026Juekai Lin, Honglin Lin, Yuqian Yuan +7Reinforcement Learning Multimodal Reasoning
Zhejiang University · Shanghai Artificial Intelligence Laboratory, OpenDataLab · Shanghai Jiao Tong University +1
Can MLLMs Reason Beyond Language? VisReason: A Comprehensive Benchmark for Vision-Centric Reasoning
May 25, 2026Longteng Guo, Yifan Wang, Pengkang Huo +4Visually Grounded Reasoning Visual Reasoning
Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences

Jun 8, 2026J/K move · Enter open · S save

Optical Reasoning: Rethinking Images as an Expressive Reasoning Medium Beyond Text

Yutong Bian, Dongjie Cheng, Heming Xia, Yongqi Li+1

The Hong Kong Polytechnic University

Visual Reasoning Multimodal Reasoning Multimodal CoT Reasoning LLM Reasoning MLLM Reasoning

Open paper AskPDF arXiv

Chain-of-Thought (CoT) improves the performance of Large Language Models (LLMs) and has been extended to Multimodal Large Language Models (MLLMs). More recent work further moves from text-based multimodal reasoning toward interleaved-modal reasoning, where intermediate steps can incorporate both textual rationales and visual evidence. In this work, we propose a bolder and more ambitious idea: could images alone serve as the reasoning medium for both language and multimodal tasks? To explore this, we propose optical reasoning, which treats images as a standalone reasoning medium. We instantiate this concept with two variants: typographic-based optical reasoning, which optimizes visual layouts for compact rationale rendering, and graphical-based optical reasoning, which composes text and graphical elements into structured visual rationales. Across mathematical, scientific, and interleaved-modal reasoning benchmarks, optical reasoning can match or even exceed traditional text reasoning while reducing reasoning tokens by an average of 28.57% on language tasks and 16% on multimodal tasks, achieving 1.96 times the token efficiency of text reasoning. These results show that images can effectively and efficiently encode rationales while providing a unified visual canvas for reasoning.

Abstract

VLM Reasoning

Explore similar work

Optical Reasoning: Rethinking Images as an Expressive Reasoning Medium Beyond Text

MMVistaReason: Toward Open-Data and Post-Training Recipes for Multimodal Reasoning

Can MLLMs Reason Beyond Language? VisReason: A Comprehensive Benchmark for Vision-Centric Reasoning