Robot Foundation Models

Ask about this topic

Overview

Robot foundation models aim to transfer capabilities across tasks, environments, and robot bodies by learning from large collections of robot demonstrations, human videos, and other multimodal data. Common approaches include vision-language-action policies, world-action models that predict how actions change scenes, and motion or tactile models, trained with imitation and self-supervision and often adapted through reinforcement learning or human feedback. Researchers evaluate them on manipulation and navigation benchmarks such as LIBERO and RoboTwin, as well as on real robots and humanoids, with growing attention to long-horizon behavior and cross-embodiment transfer. Broad generalization remains difficult: data must align with each robot’s actions and sensors, while robustness, safety, latency, and the work needed for deployment are not yet resolved by strong benchmark results alone.

Latest papers

CardsList

138 papers35 papers in the last four weeks, up 75% on the four weeks before. 0.2% of all new papers.

Apr 16, 2026

Foundation Models in Robotics: A Comprehensive Review of Methods, Models, Datasets, Challenges and Future Research Directions

Over the recent years, the field of robotics has been undergoing a transformative paradigm shift from fixed, single-task, domain-specific solutions towards adaptive, multi-function, general-purpose agents, capable of operating in complex, open-world, and dynamic environments. This tremendous advancement is primarily driven by the emergence of Foundation Models (FMs), i.e., large-scale neural-network architectures trained on massive, heterogeneous datasets that provide unprecedented capabilities in multi-modal understanding and reasoning, long-horizon planning, and cross-embodiment generalization. In this context, the current study provides a holistic, systematic, and in-depth review of the research landscape of FMs in robotics. In particular, the evolution of the field is initially delineated through five distinct research phases, spanning from the early incorporation of Natural Language Processing (NLP) and Computer Vision (CV) models to the current frontier of multi-sensory generalization and real-world deployment. Subsequently, a highly-granular taxonomic investigation of the literature is performed, examining the following key aspects: a) the employed FM types, including LLMs, VFMs, VLMs, and VLAs, b) the underlying neural-network architectures, c) the adopted learning paradigms, d) the different learning stages of knowledge incorporation, e) the major robotic tasks, and f) the main real-world application domains. For each aspect, comparative analysis and critical insights are provided. Moreover, a report on the publicly available datasets used for model training and evaluation across the considered robotic tasks is included. Furthermore, a hierarchical discussion on the current open challenges and promising future research directions in the field is incorporated.

Aggelos Psiris, Vasileios Argyriou, Evangelos K. Markakis +4

Department of Informatics and Telematics, Harokopio University of Athens, Athens, Greece · Department of Networks and Digital Media, Kingston University, London, United Kingdom · Department of Electrical and Computer Engineering, Hellenic Mediterranean University, Heraklion, Greece +4

Large Language Model-Based Robot PlanningRobot Foundation ModelsCross-Embodiment Robot LearningVision-Language-Action ModelsMultimodal Foundation ModelsVLMs for Robotics
AskPDF
Apr 13, 2026

StarVLA-αα: Reducing Complexity in Vision-Language-Action Systems

Vision-Language-Action (VLA) models have recently emerged as a promising paradigm for building general-purpose robotic agents. However, the VLA landscape remains highly fragmented and complex: as existing approaches vary substantially in architectures, training data, embodiment configurations, and benchmark-specific engineering. In this work, we introduce StarVLA-αα, a simple yet strong baseline designed to study VLA design choices under controlled conditions. StarVLA-αα deliberately minimizes architectural and pipeline complexity to reduce experimental confounders and enable systematic analysis. Specifically, we re-evaluate several key design axes, including action modeling strategies, robot-specific pretraining, and interface engineering. Across unified multi-benchmark training on LIBERO, SimplerEnv, RoboTwin, and RoboCasa, the same simple baseline remains highly competitive, indicating that a strong VLM backbone combined with minimal design is already sufficient to achieve strong performance without relying on additional architectural complexity or engineering tricks. Notably, our single generalist model outperforms π0.5π_{0.5} by 20% on the public real-world RoboChallenge benchmark. We expect StarVLA-αα to serve as a solid starting point for future research in the VLA regime. Code will be released at https://github.com/starVLA/starVLA.

Jinhui Ye, Ning Gao, Senqiao Yang +7

HKUST · XJTU · CUHK +3

Robot Foundation ModelsRobotic ManipulationVision-Language-Action ModelsRobot Learning
AskPDF
Apr 8, 2026

HY-Embodied-0.5: Embodied Foundation Models for Real-World Agents

We introduce HY-Embodied-0.5, a family of foundation models specifically designed for real-world embodied agents. To bridge the gap between general Vision-Language Models (VLMs) and the demands of embodied agents, our models are developed to enhance the core capabilities required by embodied intelligence: spatial and temporal visual perception, alongside advanced embodied reasoning for prediction, interaction, and planning. The HY-Embodied-0.5 suite comprises two primary variants: an efficient model with 2B activated parameters designed for edge deployment, and a powerful model with 32B activated parameters targeted for complex reasoning. To support the fine-grained visual perception essential for embodied tasks, we adopt a Mixture-of-Transformers (MoT) architecture to enable modality-specific computing. By incorporating latent tokens, this design effectively enhances the perceptual representation of the models. To improve reasoning capabilities, we introduce an iterative, self-evolving post-training paradigm. Furthermore, we employ on-policy distillation to transfer the advanced capabilities of the large model to the smaller variant, thereby maximizing the performance potential of the compact model. Extensive evaluations across 22 benchmarks, spanning visual perception, spatial reasoning, and embodied understanding, demonstrate the effectiveness of our approach. Our MoT-2B model outperforms similarly sized state-of-the-art models on 16 benchmarks, while the 32B variant achieves performance comparable to frontier models such as Gemini 3.0 Pro. In downstream robot control experiments, we leverage our robust VLM foundation to train an effective Vision-Language-Action (VLA) model, achieving compelling results in real-world physical evaluations. Code and models are open-sourced at https://github.com/Tencent-Hunyuan/HY-Embodied.

Tencent Robotics X, HY Vision Team, : +20

Mixture-of-TransformersRobot Foundation ModelsVLMs for RoboticsEmbodied ReasoningSpatial Reasoning
AskPDF
Mar 27, 2026

ROSClaw: An OpenClaw ROS 2 Framework for Agentic Robot Control and Interaction

Foundation models can endow robots with open-ended reasoning, language understanding, and adaptive planning, yet connecting a model to a physical robot today requires bespoke integration that couples perception, actuation, and safety to a single model and platform. We present ROSClaw, a model-agnostic executive layer that integrates the OpenClaw agent runtime with ROS 2, enabling any foundation model to perceive, reason about, and act on any ROS-enabled robot through (i) dynamic capability discovery with standardized affordance injection, (ii) multimodal observation normalization, (iii) pre-execution action validation within a configurable safety envelope, and (iv) structured audit logging. Swapping model backends or robot platforms is a configuration change; tool schemas, safety enforcement, and provenance logging remain invariant. We deploy ROSClaw on three platforms (wheeled, quadruped, humanoid) with four foundation-model backends. Under this controlled substrate, models exhibit up to 4.8 x differences in out-of-policy action proposal rates (3.4 x among frontier models alone) and produce qualitatively distinct physical behaviors from identical commands. A cross-framework parity protocol against ROSA confirms that executive-layer design, not just prompt wording, significantly affects both task completion and safety behavior, establishing ROSClaw as both practical agentic-robot infrastructure and a reproducible measurement instrument for embodied AI.

Irvin Steve Cardenas, Marcus Anthony Arnett, Natalie Catherine Yeo +2

Advanced Telerobotics Research Lab, Kent State University, Kent, OH, USA · OpenDive Technologies, New York City, NY, USA · PlaiPin Inc, Singapore

RoboticsRobot Foundation ModelsRobot Safety
AskPDF
Mar 18, 2026

Specification-Aware Distribution Shaping for Robotics Foundation Models

Robotics foundation models have demonstrated strong capabilities in executing natural language instructions across diverse tasks and environments. However, they remain largely data-driven and lack formal guarantees on safety and satisfaction of time-dependent specifications during deployment. In practice, robots often need to comply with operational constraints involving rich spatio-temporal requirements such as time-bounded goal visits, sequential objectives, and persistent safety conditions. In this work, we propose a specification-aware action distribution optimization framework that enforces a broad class of Signal Temporal Logic (STL) constraints during execution of a pretrained robotics foundation model without modifying its parameters. At each decision step, the method computes a minimally modified action distribution that satisfies a hard STL feasibility constraint by reasoning over the remaining horizon using forward dynamics propagation. We validate the proposed framework in simulation using a state-of-the-art robotics foundation model across multiple environments and complex specifications.

Sadık Bera Yüksel, Derya Aksaray

Department of Electrical and Computer Engineering at Northeastern University.

Robotic ControlRobot SafetyRobot Foundation Models
AskPDF
Mar 16, 2026

Embodied Foundation Models at the Edge: A Survey of Deployment Constraints and Mitigation Strategies

Deploying foundation models in embodied edge systems is fundamentally a systems problem, not just a problem of model compression. Real-time control must operate within strict size, weight, and power constraints, where memory traffic, compute latency, timing variability, and safety margins interact directly. The Deployment Gauntlet organizes these constraints into eight coupled barriers that determine whether embodied foundation models can run reliably in practice. Across representative edge workloads, autoregressive Vision-Language-Action policies are constrained primarily by memory bandwidth, whereas diffusion-based controllers are limited more by compute latency and sustained execution cost. Reliable deployment therefore depends on system-level co-design across memory, scheduling, communication, and model architecture, including decompositions that separate fast control from slower semantic reasoning.

Utkarsh Grover, Ravi Ranjan, Mingyang Mao +9

University of South Florida, USA · Florida International University, USA · Johns Hopkins University, USA +2

Robot Foundation ModelsEdge ComputingEfficient VLA Model Inference
AskPDF
Mar 12, 2026

Ψ0Ψ_0: An Open Foundation Model Towards Universal Humanoid Loco-Manipulation

We introduce Ψ0Ψ_0 (Psi-Zero), an open foundation model to address challenging humanoid loco-manipulation tasks. While existing approaches often attempt to address this fundamental problem by co-training on large and diverse human and humanoid data, we argue that this strategy is suboptimal due to the fundamental kinematic and motion disparities between humans and humanoid robots. Therefore, data efficiency and model performance remain unsatisfactory despite the considerable data volume. To address this challenge, \ours;decouples the learning process to maximize the utility of heterogeneous data sources. Specifically, we propose a staged training paradigm with different learning objectives: First, we autoregressively pre-train a VLM backbone on large-scale egocentric human videos to acquire generalizable visual-action representations. Then, we post-train a flow-based action expert on high-quality humanoid robot data to learn precise robot joint control. Our research further identifies a critical yet often overlooked data recipe: in contrast to approaches that scale with noisy Internet clips or heterogeneous cross-embodiment robot datasets, we demonstrate that pre-training on high-quality egocentric human manipulation data followed by post-training on domain-specific real-world humanoid trajectories yields superior performance. Extensive real-world experiments demonstrate that \ours\ achieves the best performance using only about 800 hours of human video data and 30 hours of real-world robot data, outperforming baselines pre-trained on more than 10×\times as much data by over 40% in overall success rate across multiple tasks. We will open-source the entire ecosystem to the community, including a data processing and training pipeline, a humanoid foundation model, and a real-time action inference engine.

Songlin Wei, Hongyi Jing, Boqian Li +12

USC Physical Superintelligence (PSI) Lab · NVIDIA · WorldEngine

Robot Foundation ModelsRobot LearningHumanoid Loco-ManipulationRobotic Loco-ManipulationLoco-ManipulationRobotic Control+2
AskPDF
Mar 11, 2026

Thousand-GPU Large-Scale Training and Optimization Recipe for AI-Native Cloud Embodied Intelligence Infrastructure

Embodied intelligence is a key step towards Artificial General Intelligence (AGI), yet its development faces multiple challenges including data, frameworks, infrastructure, and evaluation systems. To address these issues, we have, for the first time in the industry, launched a cloud-based, thousand-GPU distributed training platform for embodied intelligence, built upon the widely adopted LeRobot framework, and have systematically overcome bottlenecks across the entire pipeline. At the data layer, we have restructured the data pipeline to optimize the flow of embodied training data. In terms of training, for the GR00T-N1.5 model, utilizing thousand-GPU clusters and data at the scale of hundreds of millions, the single-round training time has been reduced from 15 hours to just 22 minutes, achieving a 40-fold speedup. At the model layer, by combining variable-length FlashAttention and Data Packing, we have moved from sample redundancy to sequence integration, resulting in a 188% speed increase; π-0.5 attention optimization has accelerated training by 165%; and FP8 quantization has delivered a 140% speedup. On the infrastructure side, relying on high-performance storage, a 3.2T RDMA network, and a Ray-driven elastic AI data lake, we have achieved deep synergy among data, storage, communication, and computation. We have also built an end-to-end evaluation system, creating a closed loop from training to simulation to assessment. This framework has already been fully validated on thousand-GPU clusters, laying a crucial technical foundation for the development and application of next-generation autonomous intelligent robots, and is expected to accelerate the arrival of the era of human-machine integration.

Yongjian Guo, Yunxuan Ma, Haoran Sun +22

Distributed TrainingEmbodied AIRobot Foundation Models
AskPDF
Mar 10, 2026

TiPToP: A Modular Open-Vocabulary Robot Manipulation System That Plans

We present TiPToP, a modular manipulation system that integrates pretrained foundation models with a GPU-accelerated Task and Motion Planner to solve tasks directly from RGB images and natural language. TiPToP composes perception, planning, and execution modules and requires no robot training data. It can be deployed on a standard DROID setup in under an hour and adapted to new embodiments with minimal effort. We evaluate TiPToP against π0.5-DROIDπ_{0.5}\text{-DROID}, a state-of-the-art VLA fine-tuned on 350 hours of demonstrations, across two real-world DROID setups (one operated by an external team) and simulation, where TiPToP attains a higher average success rate and faster average completion time. We also evaluate on the MolmoSpaces benchmark, where TiPToP ranks first overall on pick and pick-and-place tasks among methods not trained on in-distribution data. We further show that TiPToP's modularity enables us to trace failures to specific components, revealing where to target improvements. We release TiPToP open-source to serve as a reproducible baseline and to enable further research on modular manipulation systems. Project website and code: https://tiptop-robot.github.io

William Shen, Nishanth Kumar, Sahit Chintalapudi +8

MIT CSAIL · University of Pennsylvania

Robotic ManipulationRobot Task PlanningZero-Shot Robotic ManipulationMotion PlanningRobot Foundation Models
AskPDF
Mar 6, 2026

Robotic Foundation Models for Industrial Control: A Comprehensive Survey and Readiness Assessment Framework

Robotic foundation models (RFMs) are emerging as a promising route towards flexible, instruction- and demonstration-driven robot control, however, a critical investigation of their industrial applicability is still lacking. This survey gives an extensive overview over the RFM-landscape and analyses, driven by concrete implications, how industrial domains and use cases shape the requirements of RFMs, with particular focus on collaborative robot platforms, heterogeneous sensing and actuation, edge-computing constraints, and safety-critical operation. We synthesise industrial deployment perspectives into eleven interdependent implications and operationalise them into an assessment framework comprising a catalogue of 149 concrete criteria, spanning both model capabilities and ecosystem requirements. Using this framework, we evaluate 324 manipulation-capable RFMs via 48,276 criterion-level decisions obtained via a conservative LLM-assisted evaluation pipeline, validated against expert judgements. The results indicate that industrial maturity is limited and uneven: even the highest-rated models satisfy only a fraction of criteria and typically exhibit narrow implication-specific peaks rather than integrated coverage. We conclude that progress towards industry-grade RFMs depends less on isolated benchmark successes than on systematic incorporation of safety, real-time feasibility, robust perception, interaction, and cost-effective system integration into auditable deployment stacks.

David Kube, Simon Hadwiger, Tobias Meisen

Factory Automation, Siemens AG, Gleiwitzer Straße 555, Nuremberg, 90475, Germany. · Institute for Technologies and Management of Digital Transformation, Bergische Universität Wuppertal, Gaußstraße 20, Wuppertal, 42119, Germany.

Robot Foundation ModelsIndustrial RoboticsRobotic Manipulation
AskPDF
Mar 3, 2026

ACE-Brain-0: Spatial Intelligence as a Shared Scaffold for Universal Embodiments

Universal embodied intelligence demands robust generalization across heterogeneous embodiments, such as autonomous driving, robotics, and unmanned aerial vehicles (UAVs). However, existing embodied brain in training a unified model over diverse embodiments frequently triggers long-tail data, gradient interference, and catastrophic forgetting, making it notoriously difficult to balance universal generalization with domain-specific proficiency. In this report, we introduce ACE-Brain-0, a generalist foundation brain that unifies spatial reasoning, autonomous driving, and embodied manipulation within a single multimodal large language model~(MLLM). Our key insight is that spatial intelligence serves as a universal scaffold across diverse physical embodiments: although vehicles, robots, and UAVs differ drastically in morphology, they share a common need for modeling 3D mental space, making spatial cognition a natural, domain-agnostic foundation for cross-embodiment transfer. Building on this insight, we propose the Scaffold-Specialize-Reconcile~(SSR) paradigm, which first establishes a shared spatial foundation, then cultivates domain-specialized experts, and finally harmonizes them through data-free model merging. Furthermore, we adopt Group Relative Policy Optimization~(GRPO) to strengthen the model's comprehensive capability. Extensive experiments demonstrate that ACE-Brain-0 achieves competitive and even state-of-the-art performance across 24 spatial and embodiment-related benchmarks.

Ziyang Gong, Zehang Luo, Anke Tang +21

ACE Robotics · Shanghai Jiao Tong University · The University of Hong Kong +7

Embodied Spatial ReasoningEmbodied AICross-Embodiment TransferEmbodied ReasoningUnified Multimodal ModelsRobot Foundation Models
AskPDF
Feb 28, 2026

I-Perceive: A Foundation Model for Vision-Language Active Perception

Active perception - the ability of a robot to proactively select viewpoints to acquire task-relevant information - is essential for robust operation in real-world environments. However, existing approaches are typically limited to fixed objectives or constrained settings, and struggle to generalize to open-ended perception intents specified in natural language. We propose I-Perceive, a foundation model for language-conditioned active perception in large-scale indoor environments. Given a query image, a set of context images, and a natural language instruction, I-Perceive predicts a 6D camera pose that fulfills the specified perception intent. The model integrates a vision-language pathway for semantic grounding with a geometric reasoning pathway for multi-view 3D understanding, connected via multi-layer semantic fusion to enable language-conditioned geometric reasoning. To support scalable training, we construct a large-scale dataset of language-viewpoint pairs from both real-world scene-scanning data and simulated environments using an automated pipeline. Extensive experiments demonstrate that I-Perceive significantly outperforms strong baselines on prediction accuracy, viewpoint feasibility, and instructions alignment. The model exhibits strong zero-shot generalization to unseen scenes and instructions, and enables closed-loop active perception, progressively refining viewpoints over sequential interactions.

Yongxi Huang, Zhuohang Wang, Wenjing Tang +3

Shanghai Jiao Tong University · Shanghai Innovation Institute · Beihang University

3D Scene UnderstandingRobot Foundation ModelsActive SensingActive PerceptionActive View SelectionVLMs for Robotics+1
AskPDF
Feb 25, 2026

Are Foundation Models the Route to Full-Stack Transfer in Robotics?

In humans and robots alike, transfer learning occurs at different levels of abstraction, from high-level linguistic transfer to low-level transfer of motor skills. In this article, we provide an overview of the impact that foundation models and transformer networks have had on these different levels, bringing robots closer than ever to "full-stack transfer". Considering LLMs, VLMs and VLAs from a robotic transfer learning perspective allows us to highlight recurring concepts for transfer, beyond specific implementations. We also consider the challenges of data collection and transfer benchmarks for robotics in the age of foundation models. Are foundation models the route to full-stack transfer in robotics? Our expectation is that they will certainly stay on this route as a key technology.

Freek Stulp, Samuel Bustamante, João Silvério +3

Institute of Robotics and Mechatronics, German Aerospace Center (DLR), Wessling, Germany. · Stanford AI Lab, Stanford University, Palo Alto, USA.

Transfer LearningRobot Foundation ModelsVLMs for Robotics
AskPDF
Feb 22, 2026

WildOS: Open-Vocabulary Object Search in the Wild

Autonomous navigation in complex, unstructured outdoor environments requires robots to operate over long ranges without prior maps and limited depth sensing. In such settings, relying solely on geometric frontiers for exploration is often insufficient. In such settings, the ability to reason semantically about where to go and what is safe to traverse is crucial for robust, efficient exploration. This work presents WildOS, a unified system for long-range, open-vocabulary object search that combines safe geometric exploration with semantic visual reasoning. WildOS builds a sparse navigation graph to maintain spatial memory, while utilizing a foundation-model-based vision module, ExploRFM, to score frontier nodes of the graph. ExploRFM simultaneously predicts traversability, visual frontiers, and object similarity in image space, enabling real-time, onboard semantic navigation tasks. The resulting vision-scored graph enables the robot to explore semantically meaningful directions while ensuring geometric safety. Furthermore, we introduce a particle-filter-based method for coarse localization of the open-vocabulary target query, that estimates candidate goal positions beyond the robot's immediate depth horizon, enabling effective planning toward distant goals. Extensive closed-loop field experiments across diverse off-road and urban terrains demonstrate that WildOS enables robust navigation, significantly outperforming purely geometric and purely vision-based baselines in both efficiency and autonomy. Our results highlight the potential of vision foundation models to drive open-world robotic behaviors that are both semantically informed and geometrically grounded. Project Page: https://leggedrobotics.github.io/wildos/

Hardik Shah, Erica Tevere, Deegan Atha +6

Jet Propulsion Laboratory (JPL), California Institute of Technology (Caltech), Pasadena, CA 91011, USA · Swiss Federal Institute of Technology (ETH Zürich), Robotic Systems Lab, Zürich 8092, Switzerland · FieldAI Inc. 3 Morgan, Irvine, CA, USA. Work Performed at NASA JPL +2

Visual Robot NavigationRobotic Object SearchOpen-Vocabulary Object NavigationAutonomous Robot ExplorationOff-Road Robot NavigationSafe Robot Navigation+1
AskPDF
Feb 17, 2026

Designing Social Robots with Ethical, User-Adaptive Explainability in the Era of Foundation Models

Foundation models are increasingly embedded in social robots, mediating not only what they say and do but also how they adapt to users over time. This shift renders traditional ``one-size-fits-all'' explanation strategies especially problematic: generic justifications are now wrapped around behaviour produced by models trained on vast, heterogeneous, and opaque datasets. We argue that ethical, user-adapted explainability must be treated as a core design objective for foundation-model-driven social robotics. We first identify open challenges around explainability and ethical concerns that arise when both adaptation and explanation are delegated to foundation models. Building on this analysis, we propose four recommendations for moving towards user-adapted, modality-aware, and co-designed explanation strategies grounded in smaller, fairer datasets. An illustrative use case of an LLM-driven socially assistive robot demonstrates how these recommendations might be instantiated in a sensitive, real-world domain.

Fethiye Irmak Dogan, Alva Markelius, Hatice Gunes

University of Cambridge Cambridge, United Kingdom

Explainable Artificial IntelligenceSocial RoboticsHuman-Robot InteractionRobot Foundation Models
AskPDF
Jan 26, 2026

DeFM: Learning Foundation Representations from Depth for Robotics

Depth sensors are widely deployed across robotic platforms, and advances in fast, high-fidelity depth simulation have enabled robotic policies trained on depth observations to achieve robust sim-to-real transfer for a wide range of tasks. Despite this, representation learning for depth modality remains underexplored compared to RGB, where large-scale foundation models now define the state of the art. To address this gap, we present DeFM, a self-supervised foundation model trained entirely on depth images for robotic applications. Using a DINO-style self-distillation objective on a curated dataset of 60M depth images, DeFM learns geometric and semantic representations that generalize to diverse environments, tasks, and sensors. To retain metric awareness across multiple scales, we introduce a novel input normalization strategy. We further distill DeFM into compact models suitable for resource-constrained robotic systems. When evaluated on depth-based classification, segmentation, navigation, locomotion, and manipulation benchmarks, DeFM achieves state-of-the-art performance and demonstrates strong generalization from simulation to real-world environments. We release all our pretrained models, which can be adopted off-the-shelf for depth-based robotic learning without task-specific fine-tuning. Webpage: https://de-fm.github.io/

Manthan Patel, Jonas Frey, Mayank Mittal +5

Robotic Systems Lab (RSL), ETH Zurich, Switzerland · Stanford University, Stanford, USA · UC Berkeley, Berkeley, USA +1

Representation LearningRobot Foundation ModelsSelf-Supervised Visual Representation LearningRobotic Perception
AskPDF
Dec 10, 2025

Safe Learning for Contact-Rich Robot Tasks: A Survey from Classical Learning-Based Methods to Safe Foundation Models

Contact-rich tasks pose significant challenges for robotic systems due to inherent uncertainty, complex dynamics, and the high risk of damage during interaction. Recent advances in learning-based control have shown great potential in enabling robots to acquire and generalize complex manipulation skills in such environments, but ensuring safety, both during exploration and execution, remains a critical bottleneck for reliable real-world deployment. This survey provides a comprehensive overview of safe learning-based methods for robot contact-rich tasks. We categorize existing approaches into two main domains: safe exploration and safe execution. We review key techniques, including constrained reinforcement learning, risk-sensitive optimization, uncertainty-aware modeling, control barrier functions, and model predictive safety shields, and highlight how these methods incorporate prior knowledge, task structure, and online adaptation to balance safety and efficiency. A particular emphasis of this survey is on how these safe learning principles extend to and interact with emerging robotic foundation models, especially vision-language models (VLMs) and vision-language-action models (VLAs), which unify perception, language, and control for contact-rich manipulation. We discuss both the new safety opportunities enabled by VLM/VLA-based methods, such as language-level specification of constraints and multimodal grounding of safety signals, and the amplified risks and evaluation challenges they introduce. Finally, we outline current limitations and promising future directions toward deploying reliable, safety-aligned, and foundation-model-enabled robots in complex contact-rich environments. More details and materials are available at our \href{ https://github.com/jack-sherman01/Awesome-Learning4Safe-Contact-rich-tasks}{Project GitHub Repository}.

Heng Zhang, Rui Dai, Gokhan Solak +4

Human-Robot Interfaces and Interaction Lab, Istituto Italiano di Tecnologia, Genova, Italy. · Ph.D. program of national interest in Robotics and Intelligent Machines (DRIM), Universit`a di Genova, Genoa, Italy. · Edwardson School of Industrial Engineering, Purdue University, West Lafayette, IN, USA. +3

Safe Robot ManipulationContact-Rich Robotic ManipulationSafe RLRobot Foundation ModelsVLMs for Robotics
AskPDF
Nov 6, 2025

BFM-Zero: A Promptable Behavioral Foundation Model for Humanoid Control Using Unsupervised Reinforcement Learning

Building Behavioral Foundation Models (BFMs) for humanoid robots has the potential to unify diverse control tasks under a single, promptable generalist policy. However, existing approaches are either exclusively deployed on simulated humanoid characters, or specialized to specific tasks such as tracking. We propose BFM-Zero, a framework that learns an effective shared latent representation that embeds motions, goals, and rewards into a common space, enabling a single policy to be prompted for multiple downstream tasks without retraining. This well-structured latent space in BFM-Zero enables versatile and robust whole-body skills on a Unitree G1 humanoid in the real world, via diverse inference methods, including zero-shot motion tracking, goal reaching, and reward optimization, and few-shot optimization-based adaptation. Unlike prior on-policy reinforcement learning (RL) frameworks, BFM-Zero builds upon recent advancements in unsupervised RL and Forward-Backward (FB) models, which offer an objective-centric, explainable, and smooth latent representation of whole-body motions. We further extend BFM-Zero with critical reward shaping, domain randomization, and history-dependent asymmetric learning to bridge the sim-to-real gap. Those key design choices are quantitatively ablated in simulation. A first-of-its-kind model, BFM-Zero establishes a step toward scalable, promptable behavioral foundation models for whole-body humanoid control.

Yitang Li, Zhengyi Luo, Tonghe Zhang +10

1Carnegie Mellon University · 2Meta

Humanoid Robot ControlBehavioral Foundation ModelsRobot Foundation ModelsRepresentation Learning for RLUnsupervised RL
AskPDF