Fine-Grained Visual Perception

Follow this topic Ask about this topic

Fine-grained visual perception is about recognizing details that broad scene understanding can miss, such as small attributes, object parts, subtle differences between images, and the locations of specific objects. Approaches include part- and attribute-based representations, hierarchical classification, dense visual-language alignment, and adaptive region selection that zooms or searches for task-relevant evidence. Benchmarks test fine-grained recognition, high-resolution question answering, open-vocabulary detection, visual grounding, counting, and controlled image comparisons across domains such as vehicles, food, medicine, and aerial imagery. A central challenge is that strong language and general vision abilities do not guarantee reliable use of local visual evidence: models may overlook details or rely on semantic shortcuts, while added resolution, model scale, or reasoning does not consistently solve the problem.

Latest papers

154 papers16 papers in the last four weeks, up 23% on the four weeks before. 0.1% of all new papers.

Look at Where You Think: Localization-Head Alignment for Fine-Grained Perception in Thinking VLMs
Oct 29, 2026Yuan Wang, Wei Wu, Hongning WangFine-Grained Visual Perception Reasoning for VLMs
Department of Computer Science and Technology, Tsinghua University · Ant Group
WeVisOne-VL: Unified Visual Perception from Objects to Pixels
Oct 22, 2026Tianming Liang, Junyi Pan, Yuxi Li +5Image Matting Fine-Grained Visual Perception
WeChat Vision, Tencent Inc.
VisualWorldBench: A Fine-Grained Multi-Task Benchmark for Evaluating Visual World Knowledge in MLLMs
Oct 14, 2026Chen Chen, Zheming Liang, Jiaqi WangVLM Evaluation Knowledge-Based VQA
University of Science and Technology of China · Shanghai Innovation Institute
From Glance to Focus: Mitigating Recognition Laziness in MLLMs for Fine-Grained Recognition
Oct 9, 2026Yutong Yang, Zhuqi Liu, Mouxing Yang +1Multimodal Large Language Models Fine-Grained Visual Perception
College of Computer Science, Sichuan University, Chengdu 610065, China · School of Artificial Intelligence, Sichuan University, Chengdu 610065, China, and also with Tianfu Jincheng Laboratory, Chengdu 610093, China
ContourVLA: A Closed-Loop Perception-Action Contour Policy for Generalized Referring Expression Segmentation
Oct 8, 2026Ruicheng Zhang, Kaiwen Shen, Jiaqi Hou +6Referring Expression Segmentation Referring Image Segmentation
Sun Yat-sen University · Tsinghua University · The Chinese University of Hong Kong, Shenzhen +1
Mixture of Layers: Dynamic Layer Routing for Visual Reasoning
Oct 7, 2026Jeonghwan Kim, Sofia Stoica, Jiwan Chung +5VLM Reasoning Large Vision-Language Models
University of Illinois Urbana-Champaign · Yonsei University · Meta Reality Labs
A Stevens's Power Law Check-up of GPT-5.5's Implicit Reading of Visual Encoding
Oct 6, 2026Kaichun Yang, Jian ChenVLM Evaluation Chart Understanding
The Ohio State University
Unlocking Fine-Grained Perception in CLIP via Structurally-Aware Latent Masked Modeling
Oct 6, 2026Juntong Li, Lingwei Dang, Haomin Wu +3VLM Adaptation Dense Prediction
School of Software Engineering, South China University of Technology
Revisiting Visual Representation Enhancement of VLMs via Kernel Canonical Correlation Analysis
Oct 2, 2026Peilin Yang, Xiaoyu Liu, Jian Sun +1VLM Adaptation Visual Representation Learning
Beijing Institute of Technology
EviRover: Reinforcing Agentic Perception Beyond a Glance
Sep 30, 2026Kaixuan Fan, Kaituo Feng, Tianshuo Peng +4Tool-Using Vision-Language Agents Active Visual Perception
MMLab, The Chinese University of Hong Kong · Fudan University
GroundingPI: A Grounding Foundation Model towards Physical Intelligence with Visual Primitives
Sep 30, 2026Qize Yu, Lianrui Fan, Boyu Chen +23Vision-Language Grounding VLMs for Robotics
XPeng Inc. · Peking University · The University of Hong Kong +5
Seeing Is Not Addressing: Auditing Linguistic Access to Frozen Visual Geometry
Sep 29, 2026Woosang Jeon, Jiwon Yang, Soo Chung +1Vision-Language Models Vision-Language Grounding
Seoul National University
EviViT: Evidence-Adaptive Vision Transformers for Fine-Grained Perception
Sep 29, 2026Yaoxin Niu, Zhangquan Chen, Yang Zhang +5VLM Adaptation Efficient ViTs
Tsinghua University · Peng Cheng Laboratory · The Hong Kong University of Science and Technology +3
Why MLLMs Struggle to Count: Overcoming Individuation and Aggregation Bottlenecks with ConvStack
Sep 29, 2026Liwei Che, Yihao Quan, Sen Fang +4Object Counting Multimodal Large Language Models
Department of Computer Science, Rutgers University · Department of Computer Science, University of Washington
Hard Vision, Easy Vision: What GPT-6 Astra Reveals Across Computer Vision
Sep 28, 2026Hanoona Rasheed, Mohammed Irfan Kurpath, Bin Ren +3Vision Foundation Models Large Vision-Language Models
Mohamed bin Zayed University of Artificial Intelligence · Apertix
VD-DeepStack: Bridging Visual Comparison and Language Reasoning for Few-Shot Anomaly Detection
Sep 28, 2026Mengyang Zhao, Zhuolin He, Haiyang Yu +9Few-Shot Anomaly Detection Visual Reasoning
Fudan University · University of Southern California · Fuzhou University +1
Evidence-Aligned Multimodal On-Policy Self-Distillation for Fine-Grained Visual Understanding
Sep 28, 2026Nanxing Hu, Qiwei Yan, Jinchao Zhang +1On-Policy Self-Distillation VLM Distillation
Beihang University · University of the Chinese Academy of Sciences
Not All Confusion Is Equal: A Source-Aware Uncertainty Diagnosis for Fine-Grained Aircraft Detection
Sep 24, 2026Hai Huang, Helmut MayerObject Detection Fine-Grained Visual Perception
Chair of Visual Computing, Institute for Applied Computer Science, Universität der Bundeswehr München, Germany
BAS-OPD: Budget-Aware Selective On-Policy Self-Distillation for Fine-Grained Multimodal Perception
Sep 22, 2026Zihan Chen, Hengguang Zhou, Yuan Kang +5VLM Distillation Multimodal Large Language Models
HFIPS, Chinese Academy of Sciences · University of Science and Technology of China · University of California, Los Angeles
Look Where It Counts: A Free, Label-Free Visual Evidence Signal for Fine-Grained Vision-Language Reasoning
Sep 21, 2026Santi Ram Tiwari, Nihal Naik, Devbrat Pandey +1Vision-Language Grounding VLM Reasoning
KGraph AI Solutions Pvt. Ltd. Bangalore, India – 560016
MinCU: A Fine-Grained Benchmark for Grounded Minimal-Change Understanding in Image Pairs
Sep 20, 2026Chaoqian Mu, Wenhao Wu, Zichen Liang +4Multimodal Grounding Fine-Grained Visual Perception
Dalian University of Technology · Independent Researcher
Towards Fine-Grained Object Manipulation: SAM3-Guided Visuomotor Policy with Persistent Memory Learning and Focused Visual Conditioning
Sep 18, 2026Haolong Meng, Fangbo Qin, Mengchen Bai +4Visuomotor Policy Learning Generalization in Robotic Manipulation
Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China. · School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China. · State Key Laboratory of Brain Cognition and Brain-Inspired Intelligence Technology, Shanghai 200031, China +1
Region-Level Policy Optimization for Fine-grained MLLM Perception
Sep 17, 2026Yuheng Shi, Xiaohuan Pei, Minjing Dong +1Efficient VLM Inference Large Vision-Language Models
University of Sydney · City University of Hong Kong
Perceive, Refine, Reason: A Calibrated Pipeline for Measuring Indicators in Strategic Visual Communication on Social Media
Sep 13, 2026Weihong Qi, Chen LingSocial Media Analysis Open-Vocabulary Object Detection
Indiana University Bloomington Bloomington, IN, USA
CoralscapesV2: Panoptic and Fine-Grained Visual Scene Understanding in Coral Reefs
Sep 11, 2026Jonathan Sauder, Thomas Ruckli, Gabrielė Strodomskytė +17Panoptic Segmentation Instance Segmentation
MIT · EPFL · University of Djibouti +7
HiPerViT: A Hierarchical Perceiver-Vision Transformer Architecture for Multi-Scale Texture Recognition
Sep 10, 2026João Pedro C. A. de Sá, Odemir Martinez BrunoVisual Representation Learning Vision Transformer
Institute of Mathematics and Computer Sciences (ICMC), University of São Paulo (USP), Avenida Trabalhador São-carlense, 400, São Carlos, SP 13566-590, Brazil · São Carlos Institute of Physics (IFSC), University of São Paulo (USP), Avenida Trabalhador São-carlense, 400, São Carlos, SP 13566-590, Brazil
VDiff-Bench: A Challenging Benchmark for Fine-Grained Image Difference Identification
Sep 5, 2026Yixin Wan, Tianle Zheng, Kai-Wei ChangVLM Evaluation Vision-Language Models
Department of Computer Science University of California, Los Angeles
FAVE: Foveated Adaptive Visual Encoding for Efficient Fine-Grained Visual Understanding
Sep 3, 2026Amitangshu Mukherjee, Kaushik RoyFine-Grained Visual Perception Efficient ViTs
Elmore Family School of Electrical and Computer Engineering Purdue University
Phase-Aware Spatial-Frequency Fusion for Few-Shot Fine-Grained Image Classification
Sep 3, 2026Ruiling Liu, Linyue Zhang, Wenyi Zeng +5Frequency-Domain Feature Learning Fine-Grained Image Classification
ConCA: Concentration-Aware Channel Attention for Fine-Grained Visual Recognition
Aug 31, 2026Yu-Sheng Liu, Yu-Chen TungChannel Attention Fine-Grained Image Classification
Department of Physics, National Kaohsiung Normal University, Kaohsiung, 824, Taiwan

Oct 29, 2026J/K move · Enter open · S save

Look at Where You Think: Localization-Head Alignment for Fine-Grained Perception in Thinking VLMs

Yuan Wang, Wei Wu, Hongning Wang

Department of Computer Science and Technology, Tsinghua University · Ant Group

Fine-Grained Visual Perception Reasoning for VLMs Visual Evidence Sensitivity in VLMs

Open paper AskPDF arXiv

Thinking vision-language models reason for thousands of tokens, yet still miss small details in high-resolution images. Zooming methods pay for extra inference passes, and outcome rewards never tell the model where to look at. In Qwen3.5-4B, we find that the bottleneck is selection, not seeing: a few attention heads, which we call localization heads, pick the evidence out of thousands of visual tokens; ablating four of them costs 13.2 points more than ablating four random heads, and their focus fades as the reasoning grows. We therefore propose Localization-Head Alignment (LHA), which makes attention a training signal: whenever the reasoning states a visual fact, these heads learn to look at the focal region rather than the background, as an auxiliary loss in both SFT and RL training. On benchmarks of V*, HR-Bench 4K and TreeBench, LHA lifts Qwen3.5-4B from 74.1% to 76.6% accuracy, the best single-pass 4B result we evaluate, ahead of multi-pass zooming and released 7B models. Alignment alone adds 1.8 points over otherwise identical training while shortening reasoning by 20%, and the gains carry over to Qwen2.5-VL-3B.