Fine-Grained Visual Perception

Follow this topic Ask about this topic

Fine-grained visual perception is about recognizing details that broad scene understanding can miss, such as small attributes, object parts, subtle differences between images, and the locations of specific objects. Approaches include part- and attribute-based representations, hierarchical classification, dense visual-language alignment, and adaptive region selection that zooms or searches for task-relevant evidence. Benchmarks test fine-grained recognition, high-resolution question answering, open-vocabulary detection, visual grounding, counting, and controlled image comparisons across domains such as vehicles, food, medicine, and aerial imagery. A central challenge is that strong language and general vision abilities do not guarantee reliable use of local visual evidence: models may overlook details or rely on semantic shortcuts, while added resolution, model scale, or reasoning does not consistently solve the problem.

Latest papers

154 papers16 papers in the last four weeks, up 23% on the four weeks before. 0.1% of all new papers.

UniDoc-RL: Coarse-to-Fine Visual RAG with Hierarchical Actions and Dense Rewards
Apr 16, 2026Jun Wang, Shuo Tan, Zelong Sun +5Document Understanding Multimodal RAG
Glint Lab
Frequency-Enhanced Dual-Subspace Networks for Few-Shot Fine-Grained Image Classification
Apr 16, 2026Meijia Wang, Guochao Wang, Haozhen Chu +4Frequency-Domain Feature Learning Fine-Grained Image Classification
School of Electronic Information and Artificial Intelligence, Shaanxi University of Science and Technology, Xi’an 710000, China
From Attenuation to Attention: Variational Information Flow Manipulation for Fine-Grained Visual Perception
Apr 14, 2026Jilong Zhu, Yang FengVision-Language Models Fine-Grained Visual Perception
Key Laboratory of Intelligent Information Processing, Institute of Computing Technology, Chinese Academy of Sciences (ICT/CAS) · State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences (ICT/CAS) · University of Chinese Academy of Sciences, Beijing, China
Test-time Scaling over Perception: Resolving the Grounding Paradox in Thinking with Images
Apr 13, 2026Zheng Jiang, Yiming Chen, Nan He +4Visually Grounded Reasoning Test-Time Scaling for VLMs
Tsinghua University, Beijing, China · Beijing University of Technology, Beijing, China
Progressive Deep Learning for Automated Spheno-Occipital Synchondrosis Maturation Assessment
Apr 13, 2026Omid Halimi Milani, Amanda Nikho, Marouane Tliba +4Medical Image Classification Curriculum Learning
Department of Electrical and Computer Engineering, University of Illinois Chicago, Chicago, IL, USA · Department of Orthodontics, College of Dentistry, University of Illinois Chicago, Chicago, IL, USA · University of Orléans, Orléans, France
DiningBench: A Hierarchical Multi-view Benchmark for Perception and Reasoning in the Dietary Domain
Apr 12, 2026Song Jin, Juntian Zhang, Xun Zhang +6VLM Evaluation Fine-Grained Visual Perception
Gaoling School of Artificial Intelligence, Renmin University of China · Meituan · Wuhan University
Toward Unified Fine-Grained Vehicle Classification and Automatic License Plate Recognition
Apr 7, 2026Gabriel E. Lima, Valfride Nascimento, Eduardo Santos +3Fine-Grained Image Classification Fine-Grained Visual Perception
Department of Informatics, Federal University of Paraná, R. Evaristo F. Ferreira da Costa 391, Jardim das Américas, Curitiba, PR, 81530-090, Brazil.
Grid2Matrix: Revealing Digital Agnosia in Vision-Language Models
Apr 6, 2026Yunkai Zhang, Linda Li, Yingxin Cui +5VLM Evaluation Fine-Grained Visual Perception
BAIR, UC Berkeley · UC Santa Cruz · Analogy AI
VLMs Need Words: Vision Language Models Ignore Visual Detail In Favor of Semantic Anchors
Apr 2, 2026Haz Sameen Shahgir, Xiaofu Chen, Yu Fu +4VLM Reasoning Fine-Grained Visual Perception
University of California, Riverside 1 · MBZUAI2
V-Reflection: Transforming MLLMs from Passive Observers to Active Interrogators
Mar 31, 2026Jiazhou Zhou, Yucheng Chen, Hongyang Li +4VLM Hallucination Mitigation Latent Visual Reasoning
AI Thrust, The Hong Kong University of Science and Technology (Guangzhou) · International Digital Economy Academy · MedVisAI Lab, Lee Kong Chian School of Medicine, Nanyang Technological University, and Centre of AI in Medicine +2
Learning to Focus and Precise Cropping: A Reinforcement Learning Framework with Information Gaps and Grounding Loss for MLLMs
Mar 29, 2026Xuanpu Zhao, Zhentao Tan, Dianmo Sheng +6Visual Question Answering RL for VLMs
School of Cyber Science and Technology, University of Science and Technology of China · Anhui Province Key Laboratory of Digital Security · Independent Researcher
GUIDED: Granular Understanding via Identification, Detection, and Discrimination for Fine-Grained Open-Vocabulary Object Detection
Mar 27, 2026Jiaming Li, Zhijia Liang, Weikai Chen +2Open-Vocabulary Object Detection Fine-Grained Visual Perception
School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou, China · Meituan, Shenzhen, China · GuangDong Province Key Laboratory of Information Security Technology, China +1
SARE: Sample-wise Adaptive Reasoning for Training-free Fine-grained Visual Recognition
Mar 18, 2026Jingxiao Yang, DaLin He, Miao Pan +7Fine-Grained Visual Perception Adaptive Inference
Zhejiang University · Netease Yidun AI Lab
FineViT: Progressively Unlocking Fine-Grained Perception with Dense Recaptions
Mar 18, 2026Peisen Zhao, Xiaopeng Zhang, Mingxing Xu +10Fine-Grained Visual Perception Vision-Language Pretraining
Huawei Inc.
Panoramic Affordance Prediction
Mar 16, 2026Zixin Zhang, Chenfei Liao, Hongfei Zhang +10Robotic Perception Fine-Grained Visual Perception
HKUST(GZ) · SJTU · MBZUAI +3
FOCUS: Bridging Fine-Grained Recognition and Open-World Discovery across Domains
Mar 15, 2026Vaibhav Rathore, Divyam Gupta, Moloud Abdar +2Generalized Category Discovery Domain Generalization
Indian Institute of Technology Bombay, Mumbai, India. · Northeastern University, Boston, USA. · The University of Queensland, Brisbane, Australia.
CodePercept: Code-Grounded Visual STEM Perception for MLLMs
Mar 11, 2026Tongkun Guan, Zhibo Yang, Jianqiang Wan +10Image-to-Code Generation Visually Grounded Reasoning
MoE Key Lab of Artificial Intelligence, AI Institute, School of Computer Science, Shanghai Jiao Tong University · Qwen Team · Beijing Institute of Technology +2
OddGridBench: Exposing the Lack of Fine-Grained Visual Discrepancy Sensitivity in Multimodal Large Language Models
Mar 10, 2026Tengjin Weng, Wenhao Jiang, Jingyi Wang +3Fine-Grained Visual Perception Multimodal Large Language Models
College of Computer Science and Software Engineering, Shenzhen University · Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ) · Tsinghua Shenzhen International Graduate School +2
Granulon: Awakening Pixel-Level Visual Encoders with Adaptive Multi-Granularity Semantics for MLLM
Mar 9, 2026Junyuan Mao, Qiankun Li, Linghao Meng +5Large Vision-Language Models Fine-Grained Visual Perception
National University of Singapore · Nanyang Technological University
VLM-SubtleBench: How Far Are VLMs from Human-Level Subtle Comparative Reasoning?
Mar 9, 2026Minkyu Kim, Sangheon Lee, Dongmin ParkVLM Evaluation VLM Reasoning
KRAFTON · KAIST
Seeing as Experts Do: A Knowledge-Augmented Agent for Open-Set Fine-Grained Visual Understanding
Mar 4, 2026Junhan Chen, Zilu Zhou, Yujun Tong +3Fine-Grained Visual Perception Visually Grounded Reasoning
School of Artificial Intelligence, Beijing University of Posts and Telecommunications, China
Taxonomy-Aware Representation Alignment for Hierarchical Visual Recognition with Large Multimodal Models
Feb 28, 2026Hulingxiao He, Zhi Tan, Yuxin PengHierarchical Classification Multimodal Large Language Models
Wangxuan Institute of Computer Technology, Peking University
APPO: Attention-guided Perception Policy Optimization for Video Reasoning
Feb 27, 2026Henghui Du, Chang Zhou, Xi Chen +1Video Reasoning Fine-Grained Visual Perception
Gaoling School of Artificial Intelligence, Renmin University of China, Beijing · AI Technology Center, Online Video Business Unit, Tencent PCG
LookWise: Knowing When and Where to Look for Fine-Grained Visual Reasoning in Multimodal Large Language Models
Feb 26, 2026Yuxiang Shen, Hailong Huang, Zhenkun Gao +6VLM Reasoning Large Vision-Language Models
the Institute of Intelligent Machines, Hefei Institutes of Physical Science, Chinese Academy of Sciences, Hefei 230031, China, and also with the University of Science and Technology of China, Hefei 230026, China · Zhejiang University, Hangzhou 310027, China · East China Normal University, Shanghai 200241, China +3
Dr. Seg: Revisiting GRPO Training for Visual Large Language Models through Perception-Oriented Design
Feb 25, 2026Haoxiang Sun, Tao Wang, Chenwei Tang +2Large Vision-Language Models RL for VLMs
College of Computer Science, Sichuan University · School of Electronic and Computer Engineering, Peking University
Evaluating Graphical Perception Capabilities of Vision Transformers
Feb 20, 2026Poonam Poonam, Pere-Pau Vázquez, Timo RopinskiVision Transformer Data Visualization
Ulm University, Ulm, 89081, Germany · Viscom Group, Institute of Media Informatics, Ulm University · ViRVIG Group, Universitat Politècnica de Catalunya - BarcelonaTech
Understanding the Fine-Grained Knowledge Capabilities of Vision-Language Models
Feb 19, 2026Dhruba Ghosh, Yuhui Zhang, Ludwig SchmidtFine-Grained Visual Perception VLM Evaluation
Stanford University
Coverage-aware Semantic Representation Learning for Underrepresented Visual Concepts
Feb 18, 2026Sakib Ahammed, Xia Cui, Wenqi Lu +4Representation Learning Visual Representation Learning
Department of Computing and Mathematics, Manchester Metropolitan University The Dalton Building, Chester Street, M1 5GD Manchester
Visual Self-Refine: A Pixel-Guided Paradigm for Accurate Chart Parsing
Feb 18, 2026Jinsong Li, Xiaoyi Dong, Yuhang Zang +3Chart Understanding Fine-Grained Visual Perception
The Chinese University of Hong Kong · Shanghai AI Laboratory · Shanghai Innovation Institute +1
TikArt: Stabilizing Aperture-Guided Fine-Grained Visual Reasoning with Reinforcement Learning
Feb 16, 2026Hao Ding, Zhichuan Yang, Weijie Ge +3VLM Reasoning Multimodal Reasoning
Zhejiang University, Hangzhou, China · Xi’an Jiaotong University, Xi’an, China · Zhejiang University of Science and Technology, Hangzhou, China

Apr 16, 2026J/K move · Enter open · S save

UniDoc-RL: Coarse-to-Fine Visual RAG with Hierarchical Actions and Dense Rewards

Jun Wang, Shuo Tan, Zelong Sun, Tiancheng Gu+4

Glint Lab

Document Understanding Multimodal RAG VLM Reasoning Visually Grounded Reasoning Fine-Grained Visual Perception RL for VLMs

Open paper AskPDF arXiv

Retrieval-Augmented Generation (RAG) extends Large Vision-Language Models (LVLMs) with external visual knowledge. However, existing visual RAG systems typically rely on generic retrieval signals that overlook the fine-grained visual semantics essential for complex reasoning. To address this limitation, we propose UniDoc-RL, a unified reinforcement learning framework in which an LVLM agent jointly performs retrieval, reranking, active visual perception, and reasoning. UniDoc-RL formulates visual information acquisition as a sequential decision-making problem with a hierarchical action space. Specifically, it progressively refines visual evidence from coarse-grained document retrieval to fine-grained image selection and active region cropping, allowing the model to suppress irrelevant content and attend to information-dense regions. For effective end-to-end training, we introduce a dense multi-reward scheme that provides task-aware supervision for each action. Based on Group Relative Policy Optimization (GRPO), UniDoc-RL aligns agent behavior with multiple objectives without relying on a separate value network. To support this training paradigm, we curate a comprehensive dataset of high-quality reasoning trajectories with fine-grained action annotations. Experiments on three benchmarks demonstrate that UniDoc-RL consistently surpasses state-of-the-art baselines, yielding up to 17.7% gains over prior RL-based methods.