Embodied Reasoning

Follow this topic Ask about this topic

Embodied reasoning connects what an agent sees with what it knows about space, time, objects, and physical cause and effect, so it can choose actions and adapt when the world changes. Approaches include vision-language and vision-language-action models, structured scene graphs and world models, language-based reasoning, memory and reflection, and reinforcement learning. Benchmarks test navigation, manipulation, and long-horizon tasks in simulators such as ALFRED and BEHAVIOR as well as on real robots. Models are improving, but fluent plans and strong benchmark scores do not always translate into grounded, reliable action, especially under changing conditions or partial observation.

Latest papers

78 papers12 papers in the last four weeks, up 100% on the four weeks before. 0.1% of all new papers.

HY-Embodied-0.5: Embodied Foundation Models for Real-World Agents
Apr 8, 2026Tencent Robotics X, HY Vision Team, : +20Mixture-of-Transformers Robot Foundation Models
FlueBricks: A Construction Kit of Flute-like Instruments for Acoustic Reasoning
Apr 4, 2026Bo-Yu Chen, Chiao-Wei Huang, Lung-Pan ChengAudio Reasoning Embodied Reasoning
National Taiwan University, Taipei, Taiwan · Independent Researcher, Taipei, Taiwan
Why Cognitive Robotics Matters: Lessons from OntoAgent and LLM Deployment in HARMONIC for Safety-Critical Robot Teaming
Mar 20, 2026Sanjay Oruganti, Sergei Nirenburg, Marjorie McShane +5Cognitive Architectures for AI Agents Large Language Model-Based Robot Planning
Cognitive Science Department, Rensselaer Polytechnic Institute, Troy, NY 12180, USA. · University of Georgia, Athens, GA, 30605, USA. · Aerospace Engineering and Engineering Mechanics, University of Texas at Austin, TX, US.
Grid Spatial Understanding: A Dataset for Textual Spatial Reasoning over Grids, Embodied Settings, and Coordinate Structures
Mar 18, 2026Risham Sidhu, Julia HockenmaierSpatial Reasoning in Language Models Spatial Reasoning
University of Illinois at Urbana-Champaign
Recurrent Reasoning with Vision-Language Models for Estimating Long-Horizon Embodied Task Progress
Mar 18, 2026Yuelin Zhang, Sijie Cheng, Chen Li +4Robot Task Progress Estimation VLMs for Robotics
Gaoling School of Artificial Intelligence, Renmin University of China · Beijing Key Laboratory of Research on Large Models and Intelligent Governance · Engineering Research Center of Next-Generation Intelligent Search and Recommendation, MOE +2
When Should a Robot Think? Resource-Aware Reasoning via Reinforcement Learning for Embodied Robotic Decision-Making
Mar 17, 2026Jun Liu, Pu Zhao, Zhenglun Kong +12Embodied AI Embodied Reasoning
Robotics Institute, Carnegie Mellon University · Northeastern University · Harvard University +9
RenderMem: Rendering as Spatial Memory Retrieval
Mar 15, 2026JooHyun Park, HyeongYeop KangLong-Term Memory for Embodied Agents Agent Memory
Korea University, South Korea
ICLR: In-Context Imitation Learning with Visual Reasoning
Mar 8, 2026Toan Nguyen, Weiduo Yuan, Songlin Wei +3Robot Imitation Learning ICL for Robotics
University of Southern California · Autodesk Research
ManipulationNet: An Infrastructure for Benchmarking Real-World Robot Manipulation with Physical Skill Challenges and Embodied Multimodal Reasoning
Mar 4, 2026Yiting Chen, Kenneth Kimble, Edward H. Adelson +20Robotic Manipulation Robot Manipulation Benchmarks
Rice University · U.S. National Institute of Standards and Technology · Massachusetts Institute of Technology +12
ACE-Brain-0: Spatial Intelligence as a Shared Scaffold for Universal Embodiments
Mar 3, 2026Ziyang Gong, Zehang Luo, Anke Tang +21Embodied Spatial Reasoning Embodied AI
ACE Robotics · Shanghai Jiao Tong University · The University of Hong Kong +7
From Language to Action: Can LLM-Based Agents Be Used for Embodied Robot Cognition?
Mar 3, 2026Shinas Shaji, Fabian Huppertz, Alex Mitrevski +1Large Language Model-Based Robot Planning Language Model-Based Control
Institute of AI and Autonomous Systems (A2S), Hochschule Bonn-Rhein-Sieg, Sankt Augustin, Germany · Fraunhofer Institute for Intelligent Analysis and Information Systems, Sankt Augustin, Germany
OOWM: Structuring Embodied Reasoning and Planning via Object-Oriented Programmatic World Modeling
Feb 25, 2026Hongyu Chen, Liang Lin, Guangrun WangLarge Language Model-Based Robot Planning World Models
Sun Yat-sen University · Guangdong Key Laboratory of Big Data Analysis and Processing · X-Era AI Lab
Learning from Trials and Errors: Reflective Test-Time Planning for Embodied LLMs
Feb 24, 2026Yining Hong, Huang Huang, Manling Li +4Large Language Model-Based Robot Planning LLM Agent Planning
Stanford University · Northwestern University
Grounding LLMs in Scientific Discovery via Embodied Actions
Feb 24, 2026Bo Zhang, Jinfeng Zhou, Yuxuan Chen +3AI Agents for Scientific Discovery LLM Grounding
Tsinghua University, Beijing, China.
PhysMem: Scaling Test-Time Memory for Embodied Physical Reasoning
Feb 23, 2026Haoyang Li, Yang You, Hao Su +1Memory-Augmented Agents Embodied Reasoning
UC San Diego · Stanford University
SplatReasoner: Enhancing Embodied Reasoning and Grounding by Novel View Synthesis
Jan 19, 2026Kim Yu-Ji, Dahye Lee, Kim Jun-Seong +6VLM Reasoning Visually Grounded Reasoning
POSTECH · KAIST · ETRI +1
Robobench: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models as Embodied Brain
Oct 20, 2025Yulin Luo, Chun-Kai Fan, Menghang Dong +19Long-Horizon Robotic Manipulation Multimodal Large Language Models
State Key Laboratory of Multimedia Information Processing, School of Computer2026 Science, Peking University · Institute for Brain and Intelligence, Fudan University · University of Science andJul Technology Beijing +2
BuilderBench: The Building Blocks of Intelligent Agents
Oct 7, 2025Raj Ghugare, Roger Creus Castanyer, Catherine Ji +4Long-Horizon Agent Evaluation Robot Manipulation Benchmarks
Princeton University · Mila – Quebec AI Institute · Université de Montréal

Apr 8, 2026J/K move · Enter open · S save

HY-Embodied-0.5: Embodied Foundation Models for Real-World Agents

Tencent Robotics X, HY Vision Team, :, Xumin Yu+19

Mixture-of-Transformers Robot Foundation Models VLMs for Robotics Embodied Reasoning Spatial Reasoning

Open paper AskPDF arXiv

We introduce HY-Embodied-0.5, a family of foundation models specifically designed for real-world embodied agents. To bridge the gap between general Vision-Language Models (VLMs) and the demands of embodied agents, our models are developed to enhance the core capabilities required by embodied intelligence: spatial and temporal visual perception, alongside advanced embodied reasoning for prediction, interaction, and planning. The HY-Embodied-0.5 suite comprises two primary variants: an efficient model with 2B activated parameters designed for edge deployment, and a powerful model with 32B activated parameters targeted for complex reasoning. To support the fine-grained visual perception essential for embodied tasks, we adopt a Mixture-of-Transformers (MoT) architecture to enable modality-specific computing. By incorporating latent tokens, this design effectively enhances the perceptual representation of the models. To improve reasoning capabilities, we introduce an iterative, self-evolving post-training paradigm. Furthermore, we employ on-policy distillation to transfer the advanced capabilities of the large model to the smaller variant, thereby maximizing the performance potential of the compact model. Extensive evaluations across 22 benchmarks, spanning visual perception, spatial reasoning, and embodied understanding, demonstrate the effectiveness of our approach. Our MoT-2B model outperforms similarly sized state-of-the-art models on 16 benchmarks, while the 32B variant achieves performance comparable to frontier models such as Gemini 3.0 Pro. In downstream robot control experiments, we leverage our robust VLM foundation to train an effective Vision-Language-Action (VLA) model, achieving compelling results in real-world physical evaluations. Code and models are open-sourced at https://github.com/Tencent-Hunyuan/HY-Embodied.