Think 360°: Evaluating the Width-centric Reasoning Capability of MLLMs Beyond Depth

Mingrui Chen; Hexiong Yang; Haogeng Liu; Huaibo Huang; Ran He

cs.CVMar 24, 2026

Think 360°: Evaluating the Width-centric Reasoning Capability of MLLMs Beyond Depth

Authors: Mingrui Chen, Hexiong Yang, Haogeng Liu, Huaibo Huang, Ran He

Organizations: School of Artificial Intelligence, University of Chinese Academy of Sciences · NLPR&MAIS, Institute of Automation, Chinese Academy of Sciences · Zhongguancun Academy · School of Advanced Interdisciplinary Science, University of Chinese Academy of Sciences

Abstract

In this paper, we present a holistic multimodal benchmark that evaluates the reasoning capabilities of MLLMs with an explicit focus on reasoning width, a complementary dimension to the more commonly studied reasoning depth. Specifically, reasoning depth measures the model's ability to carry out long-chain, sequential reasoning in which each step is tightly and rigorously linked to the next. Reasoning width tends to focus more on the model's capacity for broad trial-and-error search or multi-constrained optimization: it must systematically traverse many possible and parallelized reasoning paths, apply diverse constraints to prune unpromising branches, and identify valid solution routes for efficient iteration or backtracking. To achieve it, we carefully curate 1200+ high-quality multimodal cases spanning heterogeneous domains, and propose a fine-grained tree-of-thought evaluation protocol that jointly quantifies reasoning width and depth. We evaluate 12 major model families (over 30 advanced MLLMs) across difficulty tiers, question types, and required skills. Results show that while current models exhibit strong performance on general or common-sense VQA tasks, they still struggle to combine deep sequential thought chains with wide exploratory search to perform genuine insight-based reasoning. Finally, we analyze characteristic failure modes to provide possible directions for building MLLMs that reason not only deeper but also wider.

Figures & tables

Figure 1: The concepts illustration for the width and depth in the information propagation process of neural network and reasoning. Drawing insights from the classical designs in neutral network: shortcut skipping or dropout, pyramid feature, layer stacking and gradient back propagation, we analogize these to the strategies: pruning, divide-and-conquer, trial-and-error and backtracking to distinguish depth versus width in inference processes.

Figure 2: Three‑stage pipeline for constructing Think360 ∘ —beginning with diverse seed data collection, progressing through a two‑step quality filter (rule‑based heuristics and human double‑check), and finalized through targeted annotation & refinement (as demonstrated by proof‑ and game‑based problems).

Figure 4: Tree-of-Thought Evaluation from the perspective of depth and width.

Panel of Table 2 — Table 2: Comparison with existing multimodal math benchmarks. Level: K = K -12, U = U niversity, C = C ompetition. Source: S = S elf-sourced, P =Collected from P ublic Dataset.

Panel of Figure 5 — Figure 5: Frequency distribution and co-occurrence patterns of cognitive skills required for solving problems in Think360 ∘ . The left panel shows the frequency distribution of individual cognitive capabilities across our benchmark, while the right panel presents a chord diagram illustrating the co-occurrence relationships between different cognitive skills. Please zoom in for a better view.

Model	#Para.	ALL			Perceive‑and‑Comprehend			Trial-and-Error			Divide‑and-Conquer			Branch‑and-Bound			Hypothesize-and-Test
		Acc./%	Time/ $s$	Token	Acc./ $\%$	Time/ $s$	Token	Acc./ $\%$	Time/ $s$	Token	Acc./ $\%$	Time/ $s$	Token	Acc./ $\%$	Time/ $s$	Token	Acc./ $\%$	Time/ $s$	Token
$\clubsuit$ Close-source MLLMs
$\clubsuit$ GPT-4o	-	16.0	13.28	309.03	17.2	12.69	287.16	15.3	10.72	268.64	9.9	12.21	331.46	16.8	13.17	322.85	14.3	12.89	313.87
$\clubsuit$ GPT-4v-preview	-	24.0	27.92	898.87	25.9	27.22	861.06	19.4	26.71	812.36	16.2	28.11	964.47	25.2	27.31	1013.58	20.9	28.61	919.47
$\clubsuit$ GPT-4.1	-	24.8	35.32	973.70	26.3	32.69	913.43	20.5	35.89	979.04	17.1	34.55	993.28	26.6	36.05	982.80	21.2	37.44	1015.21
$\clubsuit$ o1	-	36.8	186.81	6537.11	37.9	184.52	6540.38	29.6	170.49	6386.16	32.2	225.06	7214.05	40.6	177.01	6605.14	32.1	192.70	6713.42

Table 3: Reasoning performance evaluation with various closed-source and open-source MLLMs. We highlight the top , second , and third highest results within each column of the two groups. Please zoom in for a better view. Models with the symbol † are evaluated by the implementation with vLLM (Qwen series, MiMo, Kimi, Llama, GLM) or LMDeploy (InternVL series) for acceleration. Please zoom in for a better view.

Model	CoT	ALL
		Acc./ $\%$	Time/ $s$	Token
$\clubsuit$ Close-source MLLMs
$\clubsuit$ GPT-4o	✗	16.0	13.28	309.03
	✓	16.4	28.41	388.91
	$\Delta$	$+$ 0.4	$+$ 15.13	$+$ 79.88
$\clubsuit$ o4-mini	✗	42.1	84.61	6736.37

Table 4: Influence of Chain-of-Thought prompting on model performances.

Model	ToT-Width/%	ToT-Depth/%	Acc./%
$\clubsuit$ GPT-4o	41.4	50.4	16.0
$\clubsuit$ GPT-4.1	50.1	54.8	24.8
$\clubsuit$ Doubao-1.5-vision-pro-250328	47.9	47.5	23.9
$\clubsuit$ Claude-4-Sonnet	53.6	60.9	28.2
$\clubsuit$ Grok-2-vision-1212	40.7	52.0	15.7
$\clubsuit$ Gemini-2.0-flash	40.2	47.8	21.1

Table 5: Model results on ToT width/depth and accuracy. We highlight the top , second , and third highest results within each column.

Model	Max Response Length
$\clubsuit$ Closed-source MLLMs
$\clubsuit$ GPT-4o	16384
$\clubsuit$ GPT-4v-preview	16384
$\clubsuit$ GPT-4.1	32768
$\clubsuit$ o1	100000
$\clubsuit$ o3	100000

Table S1 : Maximum response length settings for different models. Models with † are evaluated using vLLM (Qwen series, MiMo, Kimi, LLama, and GLM-V) or LMDeploy (InternVL series).

Figure S1 : Demonstration of the Scaling of Reasoning Depth & Width

Panel of Figure 13 — Figure S2 : Distribution of error categories for Claude-Opus-4-Thinking (left) and Seed-Thinking-v1.5 (right). The pie charts illustrate that width-centric errors (e.g., incomplete branch expansion and ineffective pruning) dominate the failure modes across different architectures, substantiating that current models are primarily bottlenecked by multi-path reasoning rather than single-step depth.

Figure S3 : Word Cloud and Length Distribution Histogram of Questions in Think360.

Figure S4 : Thinking Mode Ablation. The x-axis shows accuracy improvement, and the y-axis shows token increase. Bubble size visualizes the time cost expansion. For each model, we fix the radius of the non-thinking circle to 1 and scale the radius of the thinking circle by the empirical multiplier of inference time (×Time).

Model	#Para.	CoT	ALL			Perceive‑and‑Comprehend			Trial-and-Error			Divide‑and-Conquer			Branch‑and-Bound			Hypothesize-and-Test
			Acc./%	Time/ $s$	Token	Acc./ $\%$	Time/ $s$	Token	Acc./ $\%$	Time/ $s$	Token	Acc./ $\%$	Time/ $s$	Token	Acc./ $\%$	Time/ $s$	Token	Acc./ $\%$	Time/ $s$	Token
$\clubsuit$ Close-source MLLMs
$\clubsuit$ GPT-4o	-	✗	16.0	13.28	309.03	17.2	12.69	287.16	15.3	10.72	268.64	9.9	12.21	331.46	16.8	13.17	322.85	14.3	12.89	313.87
		✓	16.4	28.41	388.91	17.5	27.53	321.81	15.1	20.88	314.74	12.2	33.59	432.74	16.3	22.56	458.20	13.8	28.07	384.97
		$\Delta$	$+$ 0.4	$+$ 15.1	$+$ 79.9	$+$ 0.3	$+$ 14.8	$+$ 34.6	$-$ 0.2	$+$ 10.2	$+$ 46.1	$+$ 2.3	$+$ 21.4	$+$ 101.3	$-$ 0.5	$+$ 9.4	$+$ 135.4	$-$ 0.5	$+$ 15.2	$+$ 71.1
$\clubsuit$ o4-mini	-	✗	42.1	84.61	6736.37	42.8	81.37	6391.21	34.3	106.52	8067.57	38.7	89.62	7460.26	48.0	76.56	6401.12	37.9	91.37	7195.80

Table S2 : Fine-grained Influence of Chain-of-Thought prompting on model performances.

Figure S5 : Ablation Study of Input Settings.

Panel of Figure 18 — Figure S6 : Reasoning performances comparison across different question types. To account for the substantial disparities in reasoning time and token usage across models, we categorize them using predefined time/token thresholds to better highlight the performance profiles of system-1 and system-2 models. Please zoom in for a better view.

Figure S7 : Workflow for Game-based Questions

Figure S8 : Workflow for Proof-based Questions

Figure S9 : Examples of the Branch-and-Bound main category.

Figure S10 : Examples of the Divide-and-Conquer main category.

Figure S11 : Examples of the Hypothesize-and-Test main category.

Figure S12 : Examples of the Perceive-and-Comprehend main category.

Figure S13 : Examples of the Trial-and-Error main category.

Figure S14 : Examples of the algebra subcategory.

Figure S15 : Examples of the combinatorics subcategory.

Figure S16 : Examples of the number theory subcategory.

Figure S17 : Examples of the geometry subcategory.

Figure S18 : Examples of the probability subcategory.

Figure S19 : Examples of the logic subcategory.

Explore similar work

From Reasoning Depth to Reasoning Breadth: Evaluating Multi-Point Associative Reasoning in Large Language Models
Aug 11, 2026Si'an Xie, Jiaxun Liu, Biao Yang +4LLM Evaluation Multi-Hop Reasoning
Beijing University of Posts and Telecommunications · Peking University · Kuaishou Technology
The Complexity Ceiling Benchmark: A Multi-Domain Evaluation of Sequential Reasoning Under Depth Scaling
Jun 28, 2026Shubh Chapra, Dhruv Kumar, Murari Mandal +1LLM Evaluation Structured Reasoning
BITS Pilani, Pilani Campus · KIIT Bhubaneswar
Think Deep, Not Just Long: Measuring LLM Reasoning Effort via Deep-Thinking Tokens
Feb 13, 2026Wei-Lin Chen, Liqian Peng, Tian Tan +5LLM Inference Efficiency Inference-Time Scaling
University of Virginia · Google

Aug 11, 2026J/K move · Enter open · S save

From Reasoning Depth to Reasoning Breadth: Evaluating Multi-Point Associative Reasoning in Large Language Models

Si'an Xie, Jiaxun Liu, Biao Yang, Wei Yuan+3

Beijing University of Posts and Telecommunications · Peking University · Kuaishou Technology

LLM Evaluation Multi-Hop Reasoning

Open paper AskPDF arXiv

Large language models (LLMs) have made substantial progress on reasoning tasks that require increasingly long and complex inferential chains. This progress primarily reflects reasoning depth. A complementary and comparatively unexamined capability is reasoning breadth: exploring multiple semantic directions in parallel and integrating the resulting clues into one coherent answer. We introduce MPAR-Bench, a bilingual English-Chinese benchmark that isolates reasoning breadth through multi-point associative reasoning. Inspired by the cooperative game Just One, each item asks a model to recover a hidden target from several independently generated, semantically diverse clues. We construct 1,000 items using a multi-agent clue-generation pipeline, embedding-based diversity filtering, and human verification. Only the answer space is drawn from public word lists, whereas every clue set is generated from scratch. Beyond exact-match accuracy, we evaluate models using accuracy, ANLS, embedding similarity, reasoning-trace verification, and four perturbations: clue masking, order shuffling, distractor injection, and multi-step clues. Across evaluated models, perturbations reduce accuracy by 9-18 percentage points in English and 5-12 percentage points in Chinese. Thinking mode improves standard-setting accuracy, especially in English, but does not consistently reduce sensitivity to perturbations. Case-level analysis also shows that extended reasoning can overturn an initially correct hypothesis. These results indicate that greater reasoning depth does not automatically confer robust reasoning breadth, and that reasoning breadth remains largely uncovered by current benchmarks.

Abstract

Explore similar work

From Reasoning Depth to Reasoning Breadth: Evaluating Multi-Point Associative Reasoning in Large Language Models

The Complexity Ceiling Benchmark: A Multi-Domain Evaluation of Sequential Reasoning Under Depth Scaling

Think Deep, Not Just Long: Measuring LLM Reasoning Effort via Deep-Thinking Tokens