We introduce HY-Embodied-0.5, a family of foundation models specifically designed for real-world embodied agents. To bridge the gap between general Vision-Language Models (VLMs) and the demands of embodied agents, our models are developed to enhance the core capabilities required by embodied intelligence: spatial and temporal visual perception, alongside advanced embodied reasoning for prediction, interaction, and planning. The HY-Embodied-0.5 suite comprises two primary variants: an efficient model with 2B activated parameters designed for edge deployment, and a powerful model with 32B activated parameters targeted for complex reasoning. To support the fine-grained visual perception essential for embodied tasks, we adopt a Mixture-of-Transformers (MoT) architecture to enable modality-specific computing. By incorporating latent tokens, this design effectively enhances the perceptual representation of the models. To improve reasoning capabilities, we introduce an iterative, self-evolving post-training paradigm. Furthermore, we employ on-policy distillation to transfer the advanced capabilities of the large model to the smaller variant, thereby maximizing the performance potential of the compact model. Extensive evaluations across 22 benchmarks, spanning visual perception, spatial reasoning, and embodied understanding, demonstrate the effectiveness of our approach. Our MoT-2B model outperforms similarly sized state-of-the-art models on 16 benchmarks, while the 32B variant achieves performance comparable to frontier models such as Gemini 3.0 Pro. In downstream robot control experiments, we leverage our robust VLM foundation to train an effective Vision-Language-Action (VLA) model, achieving compelling results in real-world physical evaluations. Code and models are open-sourced at https://github.com/Tencent-Hunyuan/HY-Embodied.
Figures & tables
Figure 1: Performance of HY-Embodied-0.5 MoT-2B on spatial and embodied benchmarks as well as downstream robot control tasks. HY-Embodied-0.5 pushes the frontier of embodied VLMs, while excelling in downstream real-world robot evaluations.
Figure 2: HY-Embodied-0.5 Mixture-of-Transformers Architecture. The MoT design decouples the processing of visual and textual tokens by employing modality-specific QKV and FFN layers, alongside distinct attention mechanisms. Visual latent tokens and mixed optimization loss are employed to bridge and stress the relationships between modalities during large-scale training.
Figure 3: Attention Computation of our Modality-Adaptive MoT. We visualize the attention computation under actual interleaved multi-modal sequences with distinct colors.
Figure 4: Data Distribution for Pre-training and Mid-training Stages. We conduct large-scale embodied pre-training and mid-training to establish foundational and advanced physical-world competencies. The pre-training mixture encompasses over 200B tokens based on spatial, robotics, and visual perception tasks. The mid-training stage leverages over 12M high-quality QA pairs for complex real-world execution based on diverse spatial and embodied domains.
Figure 5: Training Pipeline for HY-Embodied-0.5 Series. Large-scale pre-training establishes the models’ foundational multi-modal representations and robust spatial-embodied perception. The subsequent Embodied Post-training phase explicitly enhances complex reasoning capabilities through iterative self-evolution and reinforcement learning. Finally, we employ on-policy distillation to effectively transfer the knowledge from large variants to edge deployment.
Figure 6: Reward Designs for Embodied Reinforcement Learning. To accommodate diverse embodied tasks during RL optimization, we systematically formulate reward functions into four categories: Grounding-Based for spatial localization, Regression-Based for numerical estimation, Trajectory-Based for motion and planning, and Textual-Based for general and semantic reasoning.
Figure 7: Performance on General Understanding Benchmark. Comparison of HY-Embodied-0.5 MoT-2B with size-matched general VLMs. The results demonstrate that while our model is specifically optimized for spatial and embodied reasoning, it successfully maintains comparable and highly competitive performance across diverse general visual understanding tasks.
Figure 8: Visualization Results on Visual Perception Tasks. Empowered by our comprehensive visual perception training, HY-Embodied-0.5 MoT-2B demonstrates superior proficiency across foundational vision tasks, including depth estimation, object detection, and complex counting, outperforming competing embodied-specific and general VLMs.
Figure 9: Visualization Results on Embodied Tasks. Our model demonstrates comprehensive proficiency across embodied tasks, including precise visual grounding, logical action planning, and scene understanding.
Figure 10: Illustration of Chain-of-Thought Process. Our HY-Embodied series demonstrates exceptional long-chain reasoning capabilities when tackling complex visual and embodied challenges. Regarding the specific thought process, rather than simply guessing outcomes, the models systematically analyze spatial relationships and affordances step-by-step, exhibiting advanced self-reflection and error-correction mechanisms within the thinking phase.
Figure 11: MoT architecture enables faster convergence than the standard transformers (left), while delivering comparable inference speed (right). (a) presents the training loss curves, and (b) details the inference efficiency by comparing the total inference time, theoretical total FLOPs, prefill time, and decode time across different models.
Figure 12: Attention Visualizations for Visual Latent Tokens. Visual attention accurately localizes salient objects and key spatial regions, while language attention concurrently focuses on the corresponding core semantic entities, states, and action instructions.
Figure 13: Robot Experimental Setup and Success Rates for the Evaluated Tasks. (a) Real-world setup of three representative tasks. Our platform employs a dual-arm Xtrainer equipped with head-mounted and wrist-mounted cameras. The benchmark includes: (1) Precision Plug-in Packing, (2) Tableware Stacking, and (3) Mug Hanging. Success rates are evaluated over 20 trials per task. (b) Object poses are randomly initialized at the start of each trial. We conduct 20 real-robot trials per task per model. The success rates (%) are summarized above.