We introduce HY-Embodied-0.5, a family of foundation models specifically designed for real-world embodied agents. To bridge the gap between general Vision-Language Models (VLMs) and the demands of embodied agents, our models are developed to enhance the core capabilities required by embodied intelligence: spatial and temporal visual perception, alongside advanced embodied reasoning for prediction, interaction, and planning. The HY-Embodied-0.5 suite comprises two primary variants: an efficient model with 2B activated parameters designed for edge deployment, and a powerful model with 32B activated parameters targeted for complex reasoning. To support the fine-grained visual perception essential for embodied tasks, we adopt a Mixture-of-Transformers (MoT) architecture to enable modality-specific computing. By incorporating latent tokens, this design effectively enhances the perceptual representation of the models. To improve reasoning capabilities, we introduce an iterative, self-evolving post-training paradigm. Furthermore, we employ on-policy distillation to transfer the advanced capabilities of the large model to the smaller variant, thereby maximizing the performance potential of the compact model. Extensive evaluations across 22 benchmarks, spanning visual perception, spatial reasoning, and embodied understanding, demonstrate the effectiveness of our approach. Our MoT-2B model outperforms similarly sized state-of-the-art models on 16 benchmarks, while the 32B variant achieves performance comparable to frontier models such as Gemini 3.0 Pro. In downstream robot control experiments, we leverage our robust VLM foundation to train an effective Vision-Language-Action (VLA) model, achieving compelling results in real-world physical evaluations. Code and models are open-sourced at https://github.com/Tencent-Hunyuan/HY-Embodied.
Figures & tables
Figure 1: Performance of HY-Embodied-0.5 MoT-2B on spatial and embodied benchmarks as well as downstream robot control tasks. HY-Embodied-0.5 pushes the frontier of embodied VLMs, while excelling in downstream real-world robot evaluations.
Figure 2: HY-Embodied-0.5 Mixture-of-Transformers Architecture. The MoT design decouples the processing of visual and textual tokens by employing modality-specific QKV and FFN layers, alongside distinct attention mechanisms. Visual latent tokens and mixed optimization loss are employed to bridge and stress the relationships between modalities during large-scale training.
Figure 3: Attention Computation of our Modality-Adaptive MoT. We visualize the attention computation under actual interleaved multi-modal sequences with distinct colors.
Figure 4: Data Distribution for Pre-training and Mid-training Stages. We conduct large-scale embodied pre-training and mid-training to establish foundational and advanced physical-world competencies. The pre-training mixture encompasses over 200B tokens based on spatial, robotics, and visual perception tasks. The mid-training stage leverages over 12M high-quality QA pairs for complex real-world execution based on diverse spatial and embodied domains.
Figure 5: Training Pipeline for HY-Embodied-0.5 Series. Large-scale pre-training establishes the models’ foundational multi-modal representations and robust spatial-embodied perception. The subsequent Embodied Post-training phase explicitly enhances complex reasoning capabilities through iterative self-evolution and reinforcement learning. Finally, we employ on-policy distillation to effectively transfer the knowledge from large variants to edge deployment.
Figure 6: Reward Designs for Embodied Reinforcement Learning. To accommodate diverse embodied tasks during RL optimization, we systematically formulate reward functions into four categories: Grounding-Based for spatial localization, Regression-Based for numerical estimation, Trajectory-Based for motion and planning, and Textual-Based for general and semantic reasoning.
Figure 7: Performance on General Understanding Benchmark. Comparison of HY-Embodied-0.5 MoT-2B with size-matched general VLMs. The results demonstrate that while our model is specifically optimized for spatial and embodied reasoning, it successfully maintains comparable and highly competitive performance across diverse general visual understanding tasks.
Figure 8: Visualization Results on Visual Perception Tasks. Empowered by our comprehensive visual perception training, HY-Embodied-0.5 MoT-2B demonstrates superior proficiency across foundational vision tasks, including depth estimation, object detection, and complex counting, outperforming competing embodied-specific and general VLMs.
Figure 9: Visualization Results on Embodied Tasks. Our model demonstrates comprehensive proficiency across embodied tasks, including precise visual grounding, logical action planning, and scene understanding.
Figure 10: Illustration of Chain-of-Thought Process. Our HY-Embodied series demonstrates exceptional long-chain reasoning capabilities when tackling complex visual and embodied challenges. Regarding the specific thought process, rather than simply guessing outcomes, the models systematically analyze spatial relationships and affordances step-by-step, exhibiting advanced self-reflection and error-correction mechanisms within the thinking phase.
Figure 11: MoT architecture enables faster convergence than the standard transformers (left), while delivering comparable inference speed (right). (a) presents the training loss curves, and (b) details the inference efficiency by comparing the total inference time, theoretical total FLOPs, prefill time, and decode time across different models.
Figure 12: Attention Visualizations for Visual Latent Tokens. Visual attention accurately localizes salient objects and key spatial regions, while language attention concurrently focuses on the corresponding core semantic entities, states, and action instructions.
Figure 13: Robot Experimental Setup and Success Rates for the Evaluated Tasks. (a) Real-world setup of three representative tasks. Our platform employs a dual-arm Xtrainer equipped with head-mounted and wrist-mounted cameras. The benchmark includes: (1) Precision Plug-in Packing, (2) Tableware Stacking, and (3) Mug Hanging. Success rates are evaluated over 20 trials per task. (b) Object poses are randomly initialized at the start of each trial. We conduct 20 real-robot trials per task per model. The success rates (%) are summarized above.
Building capable embodied agents requires not only multimodal perception and understanding, but also agentic capabilities for reasoning about actions, adapting to evolving situations, and interacting with the physical world. In this report, we introduce Hy-Embodied-VLM-1.0, an efficient and powerful embodied foundation model specifically designed for embodied agents operating in the physical world. To cultivate such capabilities from the pre-training stage onward, we define an action-centric capability taxonomy comprising three progressive dimensions: Action-Relevant State Understanding, Action-Transition Reasoning, and Sequential and Adaptive Reasoning. Guided by this taxonomy, we develop a systematic data pipeline and curate data mixtures spanning both pre-training and post-training. To deliver strong physical-world understanding and interaction capabilities while supporting latency-sensitive deployment, we build our model on the Hy3-A3B language backbone and the Hy-ViT2 vision encoder. Its efficient Mixture-of-Experts architecture combines strong model capacity with high inference efficiency. We evaluate Hy-Embodied-VLM-1.0 on a comprehensive suite of 38 benchmarks covering embodied perception, physical-world understanding, and embodied reasoning. The model achieves the best performance among similarly sized models on 19 of the 38 benchmarks and substantially outperforms strong competitors, including Qwen3.6-A3B and Cosmos 3. Compared with the previous-generation Hy-Embodied-0.5 MoT-2B, Hy-Embodied-VLM-1.0 improves average performance by 8.4%. Despite activating only 3B parameters, it achieves performance close to that of the previous-generation model with 32B activated parameters. Beyond static benchmark evaluation, Hy-Embodied-VLM-1.0 also demonstrates strong performance on embodied agentic tasks requiring multi-turn interaction and long-horizon reasoning.
Ziyi Wang, Xumin Yu, Yongming Rao +19
Tencent Robotics X Hy Vision Team Futian Laboratory
Physical AI requires models to ground visual and linguistic understanding in real-world environments while accounting for environmental constraints and execution feedback. We introduce MachEmbodied-VLM (ME-VLM), a unified vision-language model with two variants, 4B and 35B-A3B, that brings together embodied cognition and multimodal agent capabilities. Our work emphasizes physical perception and spatiotemporal reasoning, together with planning, interaction, and outcome assessment in both digital and physical environments. We construct training data spanning embodied and multimodal agent tasks, including execution observations and feedback to support outcome assessment and decision refinement. The training pipeline comprises embodied capability injection, separate reinforcement learning of embodied and multimodal-agent experts, and multi-teacher on-policy distillation that consolidates their complementary capabilities into a single model. Experiments show competitive performance on both embodied and agent benchmarks, as well as on autonomous-driving and embodied-navigation tasks. For edge deployment, visual token compression, W4A8 quantization, and hardware-software co-optimization enable on-device inference of the 4B variant on the M100, reducing prefill latency from 400 ms to 188 ms. Project Page: https://machembodied.com/ME-Brain/ME-VLM.html Code Repository: https://github.com/MachEmbodied/ME-VLM
We introduce Embodied-R1.5, a unified Embodied Foundation Model (EFM) that integrates comprehensive embodied reasoning capabilities, spanning embodied cognition, task planning, correction, and pointing, within a single architecture toward general physical intelligence. Leveraging three automated data construction pipelines to significantly expand the data coverage of critical capabilities, we build a large-scale data system of over 15B tokens, and design a multi-task balanced RL recipe to alleviate heterogeneous task conflicts. We further introduce a Planner-Grounder-Corrector (PGC) closed-loop framework that enables a single model to autonomously execute and self-correct over long-horizon tasks. With only 8B parameters, Embodied-R1.5 achieves SOTA on 16 out of 24 embodied VLM benchmarks, surpassing leading models like Gemini-Robotics-ER-1.5 and GPT-5.4. Benefiting from the internalized embodied capabilities, Embodied-R1.5 can be fine-tuned into a VLA with only a small amount of data, outperforming leading VLA models like π0.5 across 4 popular manipulation benchmark suites. We further conduct extensive zero-shot real-robot experiments, validating performance in instruction following, affordance grounding, articulated object manipulation, and long-horizon complex tasks, demonstrating strong generalization to the physical world. We open-source model weights, datasets, training code, and EmbodiedEvalKit, an evaluation framework tailored for embodied tasks, to facilitate future research in EFMs.