TikArt: Stabilizing Aperture-Guided Fine-Grained Visual Reasoning with Reinforcement Learning

Hao Ding; Zhichuan Yang; Weijie Ge; Ziqin Gao; Chaoyi Lu; Lei Zhao

cs.CVFeb 16, 2026

TikArt: Stabilizing Aperture-Guided Fine-Grained Visual Reasoning with Reinforcement Learning

Authors: Hao Ding, Zhichuan Yang, Weijie Ge, Ziqin Gao, Chaoyi Lu, Lei Zhao

Organizations: Zhejiang University, Hangzhou, China · Xi’an Jiaotong University, Xi’an, China · Zhejiang University of Science and Technology, Hangzhou, China

Abstract

Fine-grained visual reasoning in multimodal large language models (MLLMs) is bottlenecked by single-pass global image encoding: key evidence often lies in tiny objects, cluttered regions, subtle markings, or dense charts. We present TikArt (Thinking Aperture), an aperture-guided agent that formulates multimodal reasoning as sequential evidence acquisition over regions of interest. TikArt follows a Think--Aperture--Observe (TAO) loop that interleaves language reasoning with two aperture actions: Zoom, which extracts rectangular crops, and Segment, which invokes an off-the-shelf segmenter to produce object-centric mask-based views for irregular targets. A mandatory Observation step after every aperture action writes local evidence back into text, yielding interpretable aperture trajectories and persistent linguistic memory. Built on Qwen3-VL-8B, TikArt is trained with GRPO-style reinforcement learning under a two-stage curriculum. To stabilize long-horizon tool-integrated learning, we introduce Relative Uncertainty Reduction (RUR), a dense reward computed by a frozen evaluator that favors evidence-building trajectories and mitigates degenerate tool use. Experiments on high-resolution reasoning, general multimodal understanding, and both referring and reasoning-oriented segmentation show consistent gains over the backbone, demonstrating that aperture-guided observation improves fine-grained visual reasoning and transfers naturally to pixel-level grounding.

Figures & tables

Figure 1. TikArt alternates between aperture selection and grounded observation across tasks. In this example, one trajectory supports both spatial VQA (blue) and segmentation (yellow). Dashed links indicate RoIs extracted from the same high-resolution image, and SAM2 ( Ravi et al., 2024 ) provides the mask-based crop.

Figure 2. TikArt follows a Think–Aperture–Observe (TAO) loop. Beyond box-based Zoom, TikArt includes a mask-based Segment action to obtain object-centric apertures for irregular or cluttered targets, and explicitly writes observations back into the context after each action.

Figure 3. RUR-augmented GRPO training overview. (a) We generate a group of $G$ rollouts with the current multimodal policy and compute group-normalized advantages for GRPO updates. (b) RUR is computed by a separate frozen evaluator via teacher forcing on the task target $y^{*}$ , comparing the baseline context (input only) against the trajectory context (trajectory prefix; final answer tokens excluded to prevent leakage). (c) The task scorer can combine heterogeneous supervision sources (e.g., rule-based math, segmentation metrics, VQA judging, and format constraints) into task/action rewards, which are then aggregated with RUR using fixed weights.

Proprietary models
Model	Params	V ∗			HR Bench 4K			HR Bench 8K			MME-RW-Lite			MMStar
		Attr	Spatial	Overall	FSP	FCP	Overall	FSP	FCP	Overall	Perc.	Reas.	All.	All.
GPT-4o	-	71.30 ∗	69.74 ∗	70.68 ∗	68.75 ∗	59.00 ∗	63.87 ∗	60.75 ∗	60.00 ∗	60.38 ∗	49.81	53.89	45.73	60.93
GPT-5	-	74.78	78.95	76.44	78.00	77.00	77.50	70.50	72.75	71.62	56.94	55.47	56.22	76.47
GPT-5-nano	-	58.26	72.37	63.87	69.00	61.75	65.38	65.75	61.50	63.63	46.11	49.40	42.82	66.12
Gemini-2.5-pro	-	84.35	71.05	79.06	88.75	87.00	87.87	84.50	82.75	83.62	52.10	50.73	53.48	79.40

Table 1. Results on V ∗ , HR-Bench 4K/8K, MME-RealWorld-Lite, and MMStar. TikArt consistently improves over Qwen3-VL-8B-Instruct and narrows the gap to larger open-source and proprietary models. Values marked with ∗ are from prior work; † denotes MoE models with 235B total and 22B active parameters.

Panel of Figure 4 — Figure 4. Training dynamics of TikArt and GRPO-based variants. We plot (a) policy entropy, (b) the average number of aperture actions per trajectory, and (c) the group reward excluding RUR, i.e., aggregated from task and action terms only. TikArt converges to a stable policy with moderate entropy and reasonable aperture usage, while removing Observation leads to higher entropy, uncontrolled aperture actions, and degraded reward.

Model	Params	RefCOCO			ReasonSeg-test
		val	testa	testb	giou	ciou
Grounded SAM	-	-	-	-	21.3 ∗	16.4 ∗
LISA	13B	74.9 ∗	79.1 ∗	72.4 ∗	61.3 ∗	62.2 ∗
SegR1	7B	74.3 ∗	78.7 ∗	67.6 ∗	56.7 ∗	53.7 ∗
VisionReasoner	7B	-	-	-	63.6 ∗	-
SAM-R1	7B	79.2 ∗	-	-	60.2 ∗	54.3 ∗

Table 2. Results on referring segmentation and reasoning-oriented segmentation benchmarks. TikArt brings ReasonSeg performance up to or beyond RL segmentation baselines while retaining strong RefCOCO scores. We also include w/o RUR Reward to quantify the contribution of the shared trajectory-validity reward in segmentation settings. Values marked with

\ast

are taken from prior work ( Lai et al., 2024 ; You and Wu, 2025 ; Huang et al., 2025 ) . The best result in each column is shown in bold. TikArt w/o Aperture disables aperture control; w/o Zoom Action disallows the zoom action; w/o Observation removes the observation module.

VLM Reasoning

Vision-language model reasoning studies how multimodal systems use visual evidence and language to answer questions, plan, and explain.

See all 853 papers

LookWise: Knowing When and Where to Look for Fine-Grained Visual Reasoning in Multimodal Large Language Models
Feb 26, 2026Yuxiang Shen, Hailong Huang, Zhenkun Gao +6VLM Reasoning Large Vision-Language Models
the Institute of Intelligent Machines, Hefei Institutes of Physical Science, Chinese Academy of Sciences, Hefei 230031, China, and also with the University of Science and Technology of China, Hefei 230026, China · Zhejiang University, Hangzhou 310027, China · East China Normal University, Shanghai 200241, China +3
Skip the Talk, Re-Focus on Vision: Latent Reasoning for Reasoning Segmentation in Multimodal Large Language Models
Sep 25, 2026Tianhang Guo, Yulin He, Wei Chen +3Reasoning Segmentation VLM Reasoning
National University of Defense Technology, Changsha, China
TVI-CoT: Text-Visual Interleaved Chain-of-Thought Reasoning for Multimodal Understanding
Jun 7, 2026Lianyu Hu, Xiaoyu Ma, Zeqin Liao +1Multimodal Large Language Models Multimodal Reasoning
College of Computing and Data Science of Nanyang Technological University, Singapore.

Feb 26, 2026J/K move · Enter open · S save

LookWise: Knowing When and Where to Look for Fine-Grained Visual Reasoning in Multimodal Large Language Models

Yuxiang Shen, Hailong Huang, Zhenkun Gao, Xueheng Li+5

the Institute of Intelligent Machines, Hefei Institutes of Physical Science, Chinese Academy of Sciences, Hefei 230031, China, and also with the University of Science and Technology of China, Hefei 230026, China · Zhejiang University, Hangzhou 310027, China · East China Normal University, Shanghai 200241, China+3

VLM Reasoning Large Vision-Language Models Fine-Grained Visual Perception Efficient VLM Inference

Open paper AskPDF arXiv

Multimodal Large Language Models (MLLMs) are shifting towards "Thinking with Images" by actively exploring image details. While effective, large-scale training is computationally expensive, which has spurred growing interest in lightweight, training-free solutions. However, existing training-free methods suffer from two flaws: perceptual redundancy from indiscriminate cropping, which increases computational cost and introduces noise; and a drift between semantic intent and spatial attention, which prevents accurate localization of user-focused regions. To address these challenges, we propose LookWise, a framework for adaptive visual reasoning. LookWise follows a two-stage pipeline: a confidence-based module decides when to look more carefully, and a semantic-guided localization module determines where to look. This design enables MLLMs to adaptively acquire fine-grained visual evidence without additional training. Experiments on fine-grained and high-resolution visual reasoning benchmarks show that LookWise consistently improves accuracy over strong baselines while achieving an approximately

4.0\times

inference speedup over the search-based method ZoomEye, demonstrating robust cross-model generalization.

Abstract

VLM Reasoning

Explore similar work

LookWise: Knowing When and Where to Look for Fine-Grained Visual Reasoning in Multimodal Large Language Models

Skip the Talk, Re-Focus on Vision: Latent Reasoning for Reasoning Segmentation in Multimodal Large Language Models

TVI-CoT: Text-Visual Interleaved Chain-of-Thought Reasoning for Multimodal Understanding