GLaQ: Grounding Latent Queries in Visual Evidence for Multimodal Reasoning

Zesheng Yang; Lingling Zhang; Xinyu Zhang; Cheng Zhang; Pengyu Li; Heng Wang; Lin Wu

cs.CVAug 16, 2026

GLaQ: Grounding Latent Queries in Visual Evidence for Multimodal Reasoning

Authors: Zesheng Yang, Lingling Zhang, Xinyu Zhang, Cheng Zhang, Pengyu Li, Heng Wang, Lin Wu

Organizations: Xi’an Jiaotong University · Xidian University

Abstract

Chain-of-thought reasoning has substantially improved the problem-solving capabilities of multimodal large language models. Fine-grained visual evidence, however, remains difficult to preserve and reuse across text-based reasoning steps. To address this limitation, tool-augmented thinking-with-images methods maintain visual access externally by revisiting or manipulating the image, but require predefined tools and additional inference-time processing. As an internal alternative, continuous visual latent reasoning retains intermediate computation in hidden states. However, its prevailing autoregressive construction makes each latent state depend on its predecessors, so later states may repeat information already present in the latent sequence rather than capture complementary visual details. We introduce GLaQ, a grounded latent-query framework that replaces sequential latent rollout with a fixed set of context-conditioned queries grounded in the original visual tokens. The grounded queries are reinjected for answer generation, providing direct and coordinated access to source visual evidence. We train GLaQ with localized-view supervision followed by reinforcement learning under task-level rewards. Across five benchmarks for fine-grained visual understanding and perception, GLaQ-7B gains 5.99--9.66% over its base model and leads all compared visual latent methods, suggesting that direct query-to-image grounding can recover localized evidence from the full image without external visual operations or autoregressive latent rollouts.

Figures & tables

Figure 1: Comparison of visual reasoning paradigms. We compare (a) thinking with images, (b) autoregressive visual latent reasoning, and (c) GLaQ. The heatmaps show within-set pairwise cosine similarities for the three representations; darker indicates higher similarity.

Figure 2: Overview of GLaQ. Left: The first pass through a multimodal large language model (MLLM) contextualizes a fixed query block under the current image–text prefix. Latent Query Former (LQFormer) grounds the resulting queries in source visual tokens, and the grounded block is reinjected into the second pass as soft prompts for answer generation. Right: During supervised fine-tuning, a full-image student and a region-of-interest (ROI) view teacher updated by an exponential moving average (EMA) ground the same contextualized queries. Slot-wise latent alignment transfers localized evidence to the student, while next-token prediction trains response generation; the teacher is removed at inference.

Proprietary Model
Methods	V ∗			HRBench-4K			HRBench-8K			MME-RealWorld-Lite			MMVP
Methods	Overall	Attr.	Spat.	Overall	Single	Cross	Overall	Single	Cross	Overall	Perc.	Reas.	Overall
GPT-4o ( OpenAI 2024 )	62.30	64.35	59.21	62.38	64.75	60.00	58.88	59.50	58.25	48.46	50.90	44.67	84.33
Gemini-3-Flash ( 2025 )	86.91	83.48	92.11	90.75	91.00	90.50	88.38	87.50	89.25	62.22	64.59	58.53	85.33
Open-Source Model
Qwen2.5-VL-7B ( Bai et al. 2025 )	76.44	79.13	72.37	67.38	78.75	56.00	63.75	73.75	53.75	43.82	48.16	37.07	65.67

Table 1: Main results on V ∗ , HRBench-4K, HRBench-8K, MME-RealWorld-Lite, and MMVP. All results are obtained from our reproductions and evaluated with VLMEvalKit. Scores follow official benchmark metrics; higher is better.

\Delta

reports absolute gains over Qwen2.5-VL-7B. The visual latent model with the best performance is highlighted in bold .

Table 4

$K$	V ∗	HR-4K	HR-8K	MME-Lite	MMVP
$8$	78.01	68.75	65.00	47.99	73.33
$16$	82.20	70.13	66.38	49.19	73.67
$32$	78.53	68.50	63.75	49.09	70.00

Table 4: Sensitivity to the number of grounded query slots

K

. Results are obtained under the Stage 1 SFT setting with

\lambda_{\mathrm{latent}}=0.3

and all other settings fixed.

Figure 3: Accuracy–throughput trade-off on V ∗ . All methods are evaluated on the same 191 examples under the same setup.

Multimodal Reasoning

Multimodal reasoning studies how AI systems combine text with images, audio, video, and other signals to solve evidence-dependent tasks.

See all 508 papers

SLVR: Structured Latent Visual Reasoning via Human-like Reasoning Flows
Oct 1, 2026Albert Gao, Bing Xue, Andrea ZanetteVisual Reasoning Latent Visual Reasoning
Carnegie Mellon University · Meta
Rethinking Latent Visual Reasoning: Grounding Latent Reasoning in Visual Evidence
Sep 28, 2026Xi Xiao, Tianchen Zhao, Youngeun Kim +10Large Vision-Language Models Latent Visual Reasoning
University of Alabama at Birmingham · Amazon AGI · Work done during an internship at Amazon AGI.
Latent Visual Contrastive Reasoning with Multi-Branch Exploration
Oct 27, 2026Kegeng Tang, Jingbo Wang, Yangming Zhang +3MLLM Reasoning Latent Visual Reasoning
Department of Computer Science & Engineering, University of Tennessee at Chattanooga, Chattanooga, TN, USA · Department of Computer Science, University of Texas at Arlington, Arlington, TX, USA · Department of Electrical Engineering, The Pennsylvania State University, University Park, PA, USA

Oct 1, 2026J/K move · Enter open · S save

SLVR: Structured Latent Visual Reasoning via Human-like Reasoning Flows

Albert Gao, Bing Xue, Andrea Zanette

Carnegie Mellon University · Meta

Visual Reasoning Latent Visual Reasoning VLM Reasoning Visually Grounded Reasoning Latent Reasoning

Open paper AskPDF arXiv

Multimodal large language models (MLLMs) often answer visual reasoning questions by relying on linguistic priors rather than task-relevant visual evidence. Textual chain-of-thought reasoning can partially mitigate this issue by encouraging models to decompose visual questions into intermediate evidence-seeking steps, but generating these steps autoregressively increases inference cost. Latent reasoning avoids explicit rationale generation, but existing approaches provide limited control over what intermediate states encode, making it difficult to impose separate supervision for planning, grounding, and evidence selection. We propose Structured Latent Visual Reasoning (SLVR), a training framework that bridges explicit chain-of-thought and latent reasoning by organizing multimodal reasoning into typed latent stages for planning, grounding, evidence selection, and reasoning integration. SLVR first trains the model to rely on the image by masking answer-revealing text and contrasting the correct answer with visually plausible distractors. It then organizes reasoning into latent stages for planning, grounding, evidence selection, and integration, supervising each stage with the corresponding signal: plans, boxes, visual evidence, and final rationales. This gives latent reasoning an explicit functional structure while avoiding generated textual chains at inference time. Built on Qwen2.5-VL-7B, SLVR improves consistently across multimodal reasoning benchmarks, with absolute gains of +9.4 on MMVP and +14.2 on BLINK Relation, as well as improvements on V*, MathVista, and ChartQA. These results suggest that structured latent supervision can improve fine-grained visual reasoning without the decoding overhead of textual CoT. Project page is available here.

Abstract

Multimodal Reasoning

Explore similar work

SLVR: Structured Latent Visual Reasoning via Human-like Reasoning Flows

Rethinking Latent Visual Reasoning: Grounding Latent Reasoning in Visual Evidence

Latent Visual Contrastive Reasoning with Multi-Branch Exploration