Robotic Perception

Follow this topic Ask about this topic

Robotic perception studies how robots use sensory data to estimate what is around them, how it is arranged, and how it may change. Approaches include neural networks and vision-language models for recognition and segmentation; SLAM, depth estimation, and 3D maps for geometry; and fusion of cameras with LiDAR, radar, tactile, or proprioceptive sensors. Active viewpoint selection and interaction help resolve occlusion and ambiguity, while scene graphs and affordance models connect perception to navigation and manipulation. A central challenge is making systems reliable outside curated benchmarks—under changing lighting, clutter, unfamiliar environments, and tight limits on latency and onboard computing—while representing uncertainty well enough for safe action.

Latest papers 304

64 papers in the last four weeks, up 191% on the four weeks before. 0.4% of all new papers.

All topics

RoboStressBench: Benchmarking VLM Robustness to Physical Visual Stress in Embodied Scenes
May 30, 2026Leyi Wu, Yifan Zhao, Jinjie Zhang +11VLM Evaluation VLM Robustness
HKUST(GZ) · Knowin · HKUST
ROG-Grasp: Root-Oriented Geometry for Robotic Grasping and Placement
May 30, 2026Zijian An, Augustus Sroka, Ran Yang +8Robotic Grasping Agricultural Robotics
Department of Electrical and Computer Engineering, Drexel University, 3141 Chestnut St, Philadelphia, PA, 19104, USA · Virginia Seafood Agricultural Research and Extension Center, and Department of Biological Systems Engineering, Virginia Tech, 15 Rudd Ln, Hampton, VA 23669, USA · Amazon Store Foundation AI (SFAI), 12 W 39th St, New York, NY 10018, USA
Seeing Fast and Slow: Bimodal 3D Scene Graphs for Open-set Tasks
May 29, 2026Marcel Bartholomeus Prasetyo, Shrutika Vishal Thengane, A Manicka Praveen +2Scene Graph Generation 3D SGG
Singapore University of Technology and Design (SUTD), Singapore.
DynaFLIP: Rethinking Robotics Perception via Tri-Modal-Dynamics Guided Representation
May 28, 2026Jusuk Lee, Seungjae Lee, Jonghun Shin +6Cross-Modal Learning Cross-Modal Representation Learning
Seoul National University · University of Maryland, College Park · Georgia Institute of Technology
From General Vision to Reliable Traversability Estimation: Adapting Vision Foundation Models for Unstructured Outdoor Environments
May 28, 2026Ji-Hoon Hwang, Jisung Bae, Dong-Wook Kim +2Vision Foundation Model Adaptation Robotic Perception
Self-Supervised Online Robot-Agnostic Traversability Estimation for Open-World Environments
May 27, 2026Julia Hindel, Simon Bultmann, Houman Masnavi +2Continual Robot Learning Robot Navigation
Department of Computer Science, University of Freiburg, Germany.
RepSAM: Bridging Foundation Models to Robotic Vision via Representation-Guided Adaptation
May 25, 2026Wenhui ChuVision Foundation Model Adaptation Robotic Manipulation
Department of Computer Science and Engineering, Texas A&M University, College Station, TX, USA
FOUND-IT: Foundation-model-first Task-driven 3D Scene Graphs with Granularity on Demand
May 25, 2026Dominic Maggio, Nicolas Gorlo, Kris Hauser +13D Foundation Models Scene Graph Generation
Laboratory for Information & Decision Systems, Massachusetts Institute of Technology Cambridge, MA, USA · Samsung Research America
GreenSeg: Ground Segmentation Algorithm for Agricultural Robots in Mediterranean Greenhouses using RGB-D Point Clouds
May 24, 2026Fernando Cañadas-Aránega, José C. Moreno, José L. Blanco-Claraco3D Point Cloud Segmentation Robot Navigation
Department of Informatics, CIESOL, ceiA3, Universidad de Almería, Ctra. Sacramento s/n, Almería, 04120, Spain · Department of Engineering, CIESOL, ceiA3, Universidad de Almería, Ctra. Sacramento s/n, Almería, 04120, Spain
WideDepth: Millimeter-Accurate Benchmark for Fisheye Depth Estimation
May 22, 2026Ilia Indyk, Ignat Penshin, Ivan Sosin +3Metric Depth Estimation Depth Estimation
Robotics Center, Moscow, Russia · AXXX, Moscow, Russia · Trusted AI Research Center, RAS, Moscow, Russia
Temporal Coding as a Substrate for Sensorimotor Object Inference: A Spiking Reinterpretation of Thousand Brains Architecture
May 21, 2026Joy BoseNeural Processes Active Sensing
Independent Researcher, Bengaluru, India
Learning to Evolve: Multi-modal Interactive Fields for Robust Humanoid Navigation in Dynamic Environments
May 21, 2026Peifeng Jiang, Hong Liu, Jin Jin +23D Gaussian Splatting Safe Robot Navigation
State Key Laboratory of General Artificial Intelligence, Peking University, Shenzhen Graduate School · Oxford Robotics Institute, University of Oxford · Institute for Machine Learning, Department of Computer Science, ETH Zurich
Probing an Embodied LLM: When Higher Observation Fidelity Hurts Problem Solving
May 19, 2026Oussama Zenkri, Oliver BrockLarge Language Model-Based Robot Planning LLM Agent Evaluation
Robotics and Biology Laboratory, Technische Universität Berlin, Germany · Science of Intelligence, Research Cluster of Excellence, Berlin, Germany · Robotics Institute Germany (RIG)
RGB-only Active 3D Scene Graph Generation for Indoor Mobile Robots
May 18, 2026Giorgia Modi, Davide Buoso, Giuseppe Averta +1Active Perception Scene Graph Generation
Mobile Robotics Group (MRG), University of Oxford, UK · Visual and Multimodal Applied Learning Lab (VANDAL), Politecnico di Torino, Italy
Fixed External Cameras as Common Prior Maps for Active 3D Scene Graph Generation
May 18, 2026Giorgia Modi, Davide Buoso, Giuseppe Averta +13D SGG Robotic Perception
Mobile Robotics Group (MRG), University of Oxford, UK · Visual and Multimodal Applied Learning Lab (VANDAL), Politecnico di Torino, Italy
Mono-Hydra++: Real-Time Monocular Scene Graph Construction with Multi-Task Learning for 3D Indoor Mapping
May 17, 2026U. V. B. L. Udugama, George Vosselman, Francesco Nex3D SGG Visual-Inertial SLAM
Department of Earth Observation Science, University of Twente, Enschede, 7522 NH, The Netherlands
Let Robots Feel Your Touch: Visuo-Tactile Cortical Alignment for Embodied Mirror Resonance
May 14, 2026Tianfang Zhu, Ning An, Rui Wang +4Vision-Based Tactile Sensing Cross-Modal Alignment
Institute for AI Industry Research, Tsinghua University, Beijing, China. · MoE Key Laboratory for Biomedical Photonics, Wuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology, Wuhan, China. · School of New Media Art and Design, Beihang University, Beijing, China. +1
CUBic: Coordinated Unified Bimanual Perception and Control Framework
May 13, 2026Xingyu Wang, Pengxiang Ding, Jingkai Xu +2Bimanual Robotic Manipulation Visuomotor Policy Learning
Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing, School of Artificial Intelligence, Beihang University · Westlake University · Zhejiang University +1
The Unified Autonomy Stack: Toward a Blueprint for Generalizable Robot Autonomy
May 12, 2026Mihir Dharmadhikari, Nikhil Khedekar, Mihir Kulkarni +7Robotics Safe Robot Navigation
Autonomous Robots Lab, Norwegian University of Science and Technology (NTNU), Trondheim, Norway · Department of Engineering Cybernetics, Norwegian University of Science and Technology, Høgskoleringen 1, Trondheim 7034, Norway
Learning Action Manifold with Multi-view Latent Priors for Robotic Manipulation
May 12, 2026Junjin Xiao, Dongyang Li, Yandan Yang +9Multi-View Learning Robotic Manipulation
Shenzhen University of Advanced Technology
Learning Point Cloud Geometry as a Statistical Manifold: Theory and Practice
May 11, 2026Jinwoo Lee, Jiwoo Kim, Woojae Shin +2Geometric Representation Learning Manifold Learning
Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Republic of Korea · Daegu Gyeongbuk Institute of Science and Technology (DGIST), Daegu, Republic of Korea
Initiation of Interaction Detection Framework using a Nonverbal Cue for Human-Robot Interaction
May 11, 2026Guhnoo Yun, Juhan Yoo, Kijung Kim +1Human-Robot Interaction Robotic Perception
Korea Institute of Science and Technology, Seoul, 02792, Republic of Korea · Department of Computer Science, Semyung University, Jecheon-si, 27136, Republic of Korea
Search-based Robustness Testing of Laptop Refurbishing Robotic Software
May 8, 2026Erblin Isaku, Hassan Sartaj, Shaukat Ali +2Automated Software Testing Adversarial Robustness
Simula Research Laboratory and University of Oslo · Simula Research Laboratory · Danish Technological Institute
GA3T: A Ground-Aerial Terrain Traversability Dataset for Heterogeneous Robot Teams in Unstructured Environments
May 7, 2026Siwei Cai, Knut Peterson, Quan Tran +7Multi-Robot Systems Robotic Data Collection
Drexel University, Philadelphia, USA
Contact-Free Grasp Stability Prediction with In-Hand Time-of-Flight Sensors
May 6, 2026Kyle DuFrene, Cindy GrimmRobotic Grasping Robotic Perception
Collaborative Robotics and Intelligent Systems (CoRIS) Institute, Oregon State University, Corvallis, OR 97331
Task-Aware Scanning Parameter Configuration for Robotic Inspection Using Vision Language Embeddings and Hyperdimensional Computing
May 5, 2026Zhiling Chen, David Gorsich, Matthew P. Castanier +3Robotic Inspection Robotics
aSchool of Mechanical, Aerospace, and Manufacturing Engineering, University of Connecticut, Storrs, Connecticut, USA · bUS Army DEVCOM Ground Vehicle Systems Center, Warren, Michigan, USA
Hybrid Visual Telemetry for Bandwidth-Constrained Robotic Vision: A Pilot Study with HEVC Base Video and JPEG ROI Stills
May 3, 2026Natalia Trukhina, Vadim VashkelisRobotic Perception
Embedded Intelligence Lab
SixthSense: Task-Agnostic Proprioception-Only Whole-Body Wrench Estimation for Humanoids
May 2, 2026Xingzhou Chen, Xiayan Xu, Yan Ning +8Robotic Perception Humanoid Robotics
The Hong Kong University of Science and Technology, Hong Kong SAR, China · Zhejiang University, Hangzhou, China · Tencent Robotics X, Shenzhen, China
Terrain Perception for Agricultural UAVs in Complex Farmland via Rotating mmWave Radar
May 2, 2026Zhihao Zhan, Le Tao, Shaobin Li +5Aerial Robotics 3D Reconstruction
School of Computer Science, Hangzhou Dianzi University, Hangzhou, 310018, China · TopXGun Robotics, Nanjing, 211100, China · CARDC, Mianyang, 621000, China +2
Ablation Study of Multimodal Perception, Language Grounding, and Control for Human-Robot Interaction in an Object Detection and Grasping Task
May 1, 2026Zi Tian, Guanting ShenMultimodal Grounding Robotic Grasping
Dalian University of Technology, Dalian, China

May 30, 2026J/K move · Enter open · S save

RoboStressBench: Benchmarking VLM Robustness to Physical Visual Stress in Embodied Scenes

Leyi Wu, Yifan Zhao, Jinjie Zhang, Suzeyu Chen+10

HKUST(GZ) · Knowin · HKUST

VLM Evaluation VLM Robustness Robotic Perception

Open paper AskPDF arXiv

Vision-Language Models (VLMs) have shown strong visual understanding and are increasingly deployed in embodied AI systems, where reliable perception under real conditions is essential. However, existing benchmarks assess VLMs using clean images or isolated perturbations rather than stresses caused by physical scene formation. This design has two limitations: it covers only a narrow subset of everyday visual stresses, and some perturbations rarely appear in realistic embodied scenes. This gap raises a fundamental question: how can we define visual stress in a principled way that captures the diverse factors encountered in physical environments? To address this question, we formulate visual perception from an inverse graphics perspective and introduce RoboStressBench, a benchmark for evaluating VLM robustness to physical visual stress in embodied scenes. Inspired by the physical rendering equation, RoboStressBench decomposes visual stress into four physically grounded dimensions: Material (M), Viewpoint (V), Lighting (L), and Geometry (G). This design enables RoboStressBench to cover a broad range of visual stresses in real-world environments, while allowing controlled analysis of their effects on VLM capabilities such as visual recognition, reasoning, and planning. Through comprehensive evaluations of state-of-the-art VLMs, we identify stress-specific failure modes and reveal that different physical factors degrade different embodied capabilities, which are often obscured by aggregate accuracy. We further introduce a stress-aware agentic solver that detects visual stressors and invokes visual-editing skills before reasoning, improving robustness in high-stress scenarios. Overall, RoboStressBench provides a principled evaluation framework for diagnosing and improving VLM perception under real-world physical stress, supporting the development of more reliable embodied AI systems.