Fine-Grained Visual Perception

Follow this topic Ask about this topic

Momentum

14 papers in the last four weeks, up 250% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 111

All topics

Beyond the Eye: Efficient Multimodal Reasoning via Self-Regulated Implicit Visual Tools
Jul 13, 2026Xiuwei Chen, Quanlin Chen, Wentao Hu +8Efficient Multimodal Inference Tool Use in VLMs
Sun Yat-sen University, China
SigLIP-HD by Fine-to-Coarse Supervision
Jul 10, 2026Lihe Yang, Zhen Zhao, Hengshuang ZhaoEfficient VLM Inference Large Vision-Language Models
The University of Hong Kong · Shanghai AI Laboratory
Semantic Hardness Is Not Visual Hardness: Sign-Aware Hard Negative Mining for Sign Language Retrieval
Jul 10, 2026Junmyeong Lee, Chan Hur, ChangSu Choi +5Hard Negative Mining Cross-Modal Retrieval
School of Computing1 · ETRI Medical Informatics Laboratory4 · Graduate School of Culture Technology2 +1
RFHNet: Relational and Frequency-Aware Hashing Network for Large-Scale Fine-Grained Food Image Retrieval
Jul 7, 2026Junsong Wang, Weiqing Min, Guorui Sheng +3Image Matching Fine-Grained Image Retrieval
College of Computer Science and Artificial Intelligence, Ludong University Yantai, China · Institute of Computing Technology, Chinese Academy of Sciences Beijing, China · University of Chinese Academy of Sciences Beijing, China
Segmentation before Answering: Pixel Grounding for MLLM Visual Reasoning
Jul 7, 2026Yake Wei, Yuan Wang, Fengyun Rao +2Vision-Language Grounding VLM Reasoning
Gaoling School of Artificial Intelligence, Renmin University of China · Laboratory of Research on Large Models and Intelligent Governance · Tsinghua University +1
Scene Graph Thinking: Reinforcing Structured Visual Reasoning for Multimodal Large Language Models
Jul 7, 2026Zhiwei Yang, Yuanchen Wu, Nan Zhang +3Multimodal Large Language Models Multimodal CoT Reasoning
Tencent Youtu Lab, Shanghai, China · Fudan University, Shanghai, China
BVS: Bayesian Visual Search with Multimodal Large Language Model for Fine-grained Perception
Jul 3, 2026Geng Li, Yuxin PengLarge Vision-Language Models Visual Search
Wangxuan Institute of Computer Technology, Peking University
When Does Resolution Help a Frozen Backbone? Global Attention at Resolution Predicts Scalable Adaptation for Camouflaged and Marine Animal Segmentation
Jul 2, 2026Tyler Rust, Chandra KambhamettuImage Segmentation Vision Foundation Model Adaptation
University of Delaware · University of South Florida
Perceive-to-Reason: Decoupling Perception and Reasoning for Fine-Grained Visual Reasoning
Jul 1, 2026Hongxing Li, Xiufeng Huang, Dingming Li +11Visual Reasoning VLM Reasoning
1Zhejiang University · 2Alibaba Group
Towards High-Resolution Visual Perception via Hierarchical Entity Exploration
Jul 1, 2026Ziyu Ma, Shidong Yang, Yuxiang Ji +5Large Vision-Language Models Fine-Grained Visual Perception
AMAP, Alibaba Group, China · Monash University, Australia
VisReflect: Latent Visual Reflection for Fine-Grained Perception in Long Visual Context
Jun 29, 2026Xiaoqian Shen, Mohamed ElhoseinyEfficient VLM Inference Attention Sinks
King Abdullah University of Science and Technology, Saudi Arabia
Reliability-Prioritized Fine-Grained Generation in Multimodal Large
Jun 28, 2026Xiaomeng Fan, Wei Wu, Yuwei Wu +9Direct Preference Optimization Multimodal Large Language Models
Beijing Key Laboratory of Intelligent Information Technology, School of Computer Science & Technology, Beijing Institute of Technology · Guangdong Laboratory of Machine Perception and Intelligent Computing, Shenzhen MSU-BIT University · Department of Electrical and Computer System Engineering, Monash University
What Color is the Sky (for a non-human) ?
Jun 27, 2026Yair Weiss, Ofer SpringerFine-Grained Visual Perception
School of Computer Science and Engineering and Edmond and Lili Safra Center for Brain Research, Hebrew Univesity of Jerusalem · School of Computer Science and Engineering, Hebrew Univesity of Jerusalem
PerceptionRubrics: Calibrating Multimodal Evaluation to Human Perception
Jun 26, 2026Yana Wei, Hongbo Peng, Yanlin Lai +14Rubric-Based Evaluation Automated Evaluation
Johns Hopkins University · StepFun · Tsinghua University +1
DELTAVID: Enhancing Fine-Grained Spatiotemporal Perception with Cross-Video Differences
Jun 26, 2026Yankai Yang, Yancheng Long, Bin Wen +4Video Understanding Video-Language Models
1Harbin Institute of Technology, Shenzhen · 2Kuaishou Technology
Latent Visual Diffusion Reasoning with Monte Carlo Tree Search
Jun 26, 2026Xirui Teng, Nan Xi, Junsong YuanLatent Visual Reasoning Visually Grounded Reasoning
Beijing Jiaotong University, Beijing, China · Virginia Commonwealth University, Richmond VA, USA · University at Buffalo, Buffalo NY, USA
SAM2Matting: Generalized Image and Video Matting
Jun 25, 2026Ruiqi Shen, Guangquan Jie, Chang Liu +1Fine-Grained Visual Perception
Fudan University · Shanghai University of Finance and Economics
DiCoBench: Benchmarking Multi-Image Fine-Grained Perception via Differential and Commonality Visual Cues
Jun 25, 2026Geng Li, Yuxin PengVisual Reasoning Multimodal Large Language Models
Wangxuan Institute of Computer Technology, Peking University, Beijing, China
ActiveScope: Actively Seeking and Correcting Perception for MLLMs
Jun 23, 2026Yajing Wang, Chao Bi, Junshu Sun +4Multimodal Large Language Models Vision-Language Grounding
University of Chinese Academy of Sciences · State Key Lab of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China. · Harbin Institute of Technology (Weihai).
Look Before You Zoom: Adaptive Routing for the Resolution-Context Trade-off in Visual RAG
Jun 20, 2026Oanh N. Tran, Thanh Quoc Hung Le, Oscar Chew +2Multimodal RAG Efficient VLM Inference
VinUni-Illinois Smart Health Center, VinUniversity · Texas A&M University
Benchmarking Large Vision-Language Models on Fine-Grained Image Tasks: From Evaluation to Diagnosis
Jun 17, 2026Hong-Tao Yu, Chen-Wei Xie, Yuxin Peng +2VLM Evaluation VLM Robustness
School of Computer Science and Engineering, Southeast University, China · Alibaba Group · Wangxuan Institute of Computer Technology, National Key Laboratory for Multimedia Information Processing, Peking University, China +2
LARE: Low-Attention Region Encoding for Text-Image Retrieval
Jun 17, 2026Abdulmalik Alquwayfili, Faisal Almeshal, Jumanah Almajnouni +8Visual Representation Learning Fine-Grained Image Retrieval
Saudi Data and Artificial Intelligence Authority (SDAIA), Riyadh, Saudi Arabia.
MathVis-Fine: Aligning Visual Supervision with Necessity via Progressive Dependency-Guided Training for Multimodal Mathematical Reasoning
Jun 16, 2026Wanshi Xu, Haokun Zhao, Haidong Yuan +2Multimodal Reasoning Multimodal Reward Modeling
School of ECE, Peking University · College of Computer Science and Artificial Intelligence, Fudan University · School of Software and Microelectronics, Peking University +1
LOCUS: Local Visual Cue Search for Enhancing Fine-Grained Perception in Multimodal Large Language Models
Jun 15, 2026Zhou Tao, Fang Zhang, Zewen Ding +5Multimodal Large Language Models Vision-Language Grounding
University of Science and Technology of China · State Key Laboratory of Cognitive Intelligence
City landscape in sight: A crowdsourced framework for unlocking urban-scale window view perceptions from real estate imagery
Jun 13, 2026Chucai Peng, Sijie Yang, Ang Liu +3Human Preference Modeling Urban Planning
Department of Architecture, National University of Singapore, Singapore · College of Horticulture and Forestry Sciences / Hubei Engineering Technology Research Centre for Forestry Information, Huazhong Agricultural University, Wuhan, 430070, China · School of Engineering and Applied Science, University of Pennsylvania, Philadelphia, 19104, USA +2
Trustworthy Visual Predicates for Robust Manipulation Understanding under Degradation
Jun 6, 2026Fatemeh ZiaeetabarRobotic Manipulation Fine-Grained Visual Perception
Department of Computer Science, School of Mathematics, Statistics and Computer Science, College of Science, University of Tehran, Tehran, Iran
The Last Visible Pixel: Probing Fine-Scale Perception in Vision-Language Models
Jun 5, 2026Lujun Li, Lama Sleem, Niccolo Gentile +4VLM Evaluation Large Vision-Language Models
University of Luxembourg · Foyer S.A. · Université Paris-Saclay
Anchored, Not Graded: Vision-Language Models Fail at Slant-from-Texture Perception
Jun 4, 2026Qian Zhang, Michal Golovanevsky, Fulvio Domini +1VLM Evaluation Vision-Language Models
Brown University Computer Science · Harvard University · Brown University Cognitive Science
UltraVR: A Diagnostic Ultra-Resolution Image-VQA Benchmark for Evidence-Grounded Reasoning
Jun 4, 2026Gexin Huang, Yanting Yang, Myeongkyun Kang +6VLM Evaluation Visual Question Answering
University of British Columbia, Vancouver, BC, Canada · Vector Institute, Toronto, ON, Canada · The Hong Kong Polytechnic University, Hong Kong SAR, China +1
An Open-Source Two-Stage Computer Vision Pipeline for Fine-Grained Vehicle Classification using Vision Transformers
Jun 3, 2026Gandhimathi Padmanaban, Fred FengFine-Grained Visual Perception
Department of Industrial Manufacturing and Systems Engineering, University of Michigan-Dearborn, Michigan, USA, 48128

Jul 13, 2026, cs.CVJ/K move · Enter open · S save

Beyond the Eye: Efficient Multimodal Reasoning via Self-Regulated Implicit Visual Tools

Xiuwei Chen, Quanlin Chen, Wentao Hu, Zisheng Chen+7

Sun Yat-sen University, China

Efficient Multimodal Inference Tool Use in VLMs Multimodal Reasoning Fine-Grained Visual Perception Multimodal Tool Use

Open paper AskPDF arXiv

Recent multimodal large language models (MLLMs) have made remarkable progress on fine-grained perception tasks under the "Thinking with Images" (TwI) paradigm by iteratively performing various visual tool operations. However, this paradigm relies heavily on frequent external tool calls and repeated image re-encoding, which leads to substantial computational overhead and inference latency. To address these issues, we propose Beyond the Eye (BEE), a novel implicit visual tool paradigm centered on self-regulated capability. BEE directly incorporates visual tool invocation behaviors into the training objective and encourages the model to develop a self-regulated invocation mechanism. This design enables the model to adaptively balance internal knowledge and implicit tools, avoiding redundant tool usage while substantially reducing inference latency. Specifically, BEE involves a two-stage training process: (1) Formalized Chain-of-Thought (CoT) Supervised Fine-tuning (SFT). We construct CoT trajectories with structured tool slots and mixed invocation states. This stage activates the model's implicit tool representations and adaptive switching capability. (2) Self-regulated Reward-Driven Alignment. To address redundant tool usage caused by ambiguous cognitive boundaries, we first introduce the Net Tool Gain (NTG) metric to quantify this phenomenon. Based on this observation, we further propose a self-regulated reward mechanism. This mechanism penalizes ineffective tool dependency and encourages the model to perform knowledge routing, ensuring that implicit tools are invoked only when the model's internal knowledge is insufficient. BEE achieves state-of-the-art performance in fine-grained visual perception while remaining competitive in general reasoning tasks and achieving substantial gains in inference efficiency.

Momentum

Related topics

Latest papers 111