ACE-Brain-0: Spatial Intelligence as a Shared Scaffold for Universal Embodiments

Ziyang Gong; Zehang Luo; Anke Tang; Zhe Liu; Shi Fu; Zhi Hou; Ganlin Yang; Weiyun Wang

cs.ROMar 3, 2026

ACE-Brain-0: Spatial Intelligence as a Shared Scaffold for Universal Embodiments

Authors: Ziyang Gong, Zehang Luo, Anke Tang, Zhe Liu, Shi Fu, Zhi Hou, Ganlin Yang, Weiyun Wang,

Organizations: ACE Robotics · Shanghai Jiao Tong University · The University of Hong Kong · Nanyang Technological University ·

Abstract

Universal embodied intelligence demands robust generalization across heterogeneous embodiments, such as autonomous driving, robotics, and unmanned aerial vehicles (UAVs). However, existing embodied brain in training a unified model over diverse embodiments frequently triggers long-tail data, gradient interference, and catastrophic forgetting, making it notoriously difficult to balance universal generalization with domain-specific proficiency. In this report, we introduce ACE-Brain-0, a generalist foundation brain that unifies spatial reasoning, autonomous driving, and embodied manipulation within a single multimodal large language model~(MLLM). Our key insight is that spatial intelligence serves as a universal scaffold across diverse physical embodiments: although vehicles, robots, and UAVs differ drastically in morphology, they share a common need for modeling 3D mental space, making spatial cognition a natural, domain-agnostic foundation for cross-embodiment transfer. Building on this insight, we propose the Scaffold-Specialize-Reconcile~(SSR) paradigm, which first establishes a shared spatial foundation, then cultivates domain-specialized experts, and finally harmonizes them through data-free model merging. Furthermore, we adopt Group Relative Policy Optimization~(GRPO) to strengthen the model's comprehensive capability. Extensive experiments demonstrate that ACE-Brain-0 achieves competitive and even state-of-the-art performance across 24 spatial and embodiment-related benchmarks.

Figures & tables

Figure 1: Cross-Embodiment Learning Paradigm of ACE-Brain-0 and Performance Comparison with other Embodied Brains. ACE-Brain-0 unifies tasks from four domains, Spatial Cognition, Autonomous Driving, Low-Altitude Sensing, and Embodied Manipulation. We hope to answer: “How can we instill and unify these capabilities within a single embodied foundation brain?” Conventional joint training mixes multi-domain data with shared parameters, which often causes gradient interference across tasks; sequential training accumulates skills via stage-wise fine-tuning, but tends to overwrite previously learned capabilities and leads to catastrophic forgetting. In contrast, we propose our Scaffold-Specialize-Reconcile paradigm: We first construct a Spatial Expert as a universal foundational model, then train the AD and UAV experts separately to acquire domain-specific skills while enabling coarse-grained spatial reasoning, and subsequently combine their expertise into a unified model via data-free expert merging. We further perform Embodied SFT, optionally followed by GRPO-based RFT for reward-guided post-training alignment. This pipeline delivers consistent and stable improvements across all four domains. The radar chart on the right further compares ACE-Brain-0 against representative embodied brains across multiple benchmarks, showing stronger overall performance on a broader set of tasks, and validating the unified cross-embodiment capability of ACE-Brain-0.

Figure 2: Overview of ACE-Brain-0 Capabilities. ACE-Brain-0 is a spatial-centric foundation brain that supports Spatial Intelligence, Embodied Manipulation, Low-Altitude Sensing, and Autonomous Driving. Specifically, ACE-Brain-0 is evaluated on 7 benchmarks for Spatial Cognition, 6 benchmarks for Autonomous Driving, 5 benchmarks for Low-Altitude Sensing, and 6 benchmarks for Embodied Interaction. ACE-Brain-0’s ability to integrate perception, decision, and planning across diverse real-world embodied scenarios, highlighting its generalization capability as a universal embodied intelligence model.

Figure 3: ACE-Brain-0’s unified multimodal architecture and cross-domain capability coverage. ACE-Brain-0 supports inputs including single-view images, multi-view images, and videos; the instruction examples illustrate that the model can perform Q&A-style tasks across domains (General/Spatial/Driving/Aerial/Embodied). The top row summarizes ACE-Brain-0’s core capability spectrum for cross-embodiment scenarios, such as Spatial Perception and Temporal Modeling, enabling unified representation and compositional generalization across domains.

	Stage-1	Stage-2	Stage-3	Stage-4	Stage-5
Target Objective	Scaffold SFT	Specialize SFT	Expert Reconcile	Embodied SFT	RTF
Data Domain	Spatial	AD,UAV	- (Data-Free)	Embodied	Mixed
Base Model	$\theta_{\text{base}}$	$\theta_{\text{spatial}}$	$\theta$ , $\theta_{\text{spatial}}$ , $\theta_{\text{ad}}$ , $\theta_{\text{uav}}$	$\theta_{\text{merge}}$	$\theta_{\text{embodied}}$
Trainable Part	MLP, LLM	MLP, LLM	MLP, LLM	MLP, LLM	MLP, LLM
Per-device Batch Size	8	8	-	8	8 / 256
Gradient Accumulation	4	4	0	4	8

Table 1: Details of Scaffold-Specialize-Reconcile training strategy of ACE-Brain-0.

Model	VSI	MMSI	BLINK	SITE	SAT	Mindcube	Multi3DRef
Closed-source MLLMs
GPT-4o [ 87 ]	43.6	30.3	77.9	37.8	66.7	46.1	8.1
Gemini-2.5-Pro [ 26 ]	47.8	38.0	81.8	57.0	79.3	57.6	–
Claude-4-Sonnet [ 88 ]	47.0	–	78.1	–	75.3	36.6	–
Qwen-VL-Max [ 71 ]	41.8	–	–	–	56.7	–	–
Open-source general-purpose MLLMs

Table 2: Performance Comparison on Spatial Benchmarks. Bold numbers indicate the best results, underlined numbers indicate the second-best results, and results marked with ∗ are obtained using our evaluation framework.

Model	MME-RealWorld	MAPLM	DriveAction	NuscenesQA	NuPlanQA	LingoQA
Closed-source MLLMs
GPT-4o [ 87 ]	58.0	26.6	72.5	34.3	81.5	56.0
Gemini2.5-pro [ 26 ]	67.0	26.1	73.5	16.1	–	64.1
Claude-4-Sonnet [ 88 ]	–	–	–	–	–	–
Qwen-VL-Max [ 71 ]	61.7	24.8	72.6	6.7	–	58.8
Open-source general-purpose MLLMs

Table 3: Performance Comparison on Autonomous Driving Benchmarks.

Model	UrbanVideo-Bench	AircopBench	Avi-Math	Airspatial-VQA	HRVQA
Closed-source MLLMs
GPT-4o [ 87 ]	43.6	51.8	33.5	192.4	36.9
Gemini2.5-pro [ 26 ]	–	49.1	–	–	–
Claude-4-Sonnet [ 88 ]	–	50.7	–	–	–
Qwen-VL-Max [ 71 ]	45.5	50.5	–	–	–
Open-source general-purpose MLLMs

Table 4: Performance Comparison on Low-Altitude Benchmarks.

Model	ERQA	RoboVQA	OpenEQA	EmbSpatial	Ego-Plan2	EB-Habitat
Closed-source MLLMs
GPT-4o [ 87 ]	47.0	3.3	56.4	71.9	41.8	59.0
Gemini2.5-pro [ 26 ]	48.3	–	–	78.7	42.9	–
Claude-4-Sonnet [ 88 ]	–	–	–	64.3	41.3	–
Qwen-VL-Max [ 71 ]	–	–	–	–	44.7	45.3
Open-source general-purpose MLLMs

Table 5: Performance Comparison on Embodied Benchmarks.

Initialization / Route	AD		UAV		Embodied
Initialization / Route	Avg.	$\Delta$	Avg.	$\Delta$	Avg.	$\Delta$
Qwen3-VL-8B-Instruct ( $\theta$ )	47.0	–	37.8	–	52.7	–
AD Experts ( $\theta\rightarrow\theta_{A}$ )	58.1	+11.1	–	–	–	–
UAV Experts ( $\theta\rightarrow\theta_{U}$ )	–	–	48.8	+11.0	–	–
Embodied Experts ( $\theta\rightarrow\theta_{E}$ )	–	–	–	–	50.8	-1.9
Spatial $\rightarrow$ AD Expert ( $\theta_{S}\rightarrow\theta_{A}$ )	72.6	+25.6	–	–	–	–

Table 6: Spatial knowledge consistently improves expert performance. We compare different pretraining routes for adapting to three domains (AD, UAV, and Embodied). The average score of AD is computed over NuscenesQA, NuPlanQA, and LingoQA benchmarks; the average score of UAV is computed over UrbanVideoBench, AircopBench, and Avi-Math; the average score of Embodied is computed over RoboVQA and EgoPlan. Bold indicates the best score, and improvements (

\Delta

) are reported over Qwen3-VL-8B-Instruct.

Training / Synthesis Strategy	Spatial		AD		UAV
Training / Synthesis Strategy	Avg.	$\Delta$	Avg.	$\Delta$	Avg.	$\Delta$
Qwen3-VL-8B-Instruct ( $\theta$ )	51.6	–	47.0	–	37.8	–
Spatial ( $\theta_{S}$ )	72.5	+20.9	–	–	–	–
Spatial $\rightarrow$ AD ( $\theta_{S}\rightarrow\theta_{A}$ )	–	–	72.6	+25.6	–	–
Spatial $\rightarrow$ UAV ( $\theta_{S}\rightarrow\theta_{U}$ )	–	–	–	–	54.3	+16.5
Merging weights of $\theta_{0}$ , $\theta_{0\rightarrow S}$ , $\theta_{S\rightarrow A}$ , $\theta_{S\rightarrow U}$

Table 7: Optimized Merging effectively combines each domain. The average score of Spatial is computed over VSI, SAT, and MindCube benchmarks.

Training Paradigm	Spatial		AD		UAV		Embodied
Training Paradigm	Avg.	$\Delta$	Avg.	$\Delta$	Avg.	$\Delta$	Avg.	$\Delta$
Spatial ( $\theta_{S}$ )	72.5	–	–	–	–	–	–	–
Spatial $\rightarrow$ AD Expert ( $\theta_{S}\rightarrow\theta_{A}$ )	–	–	72.6	–	–	–	–	–
Spatial $\rightarrow$ UAV Expert ( $\theta_{S}\rightarrow\theta_{U}$ )	–	–	–	–	54.3	–	–	–
Spatial $\rightarrow$ Embodied Expert ( $\theta_{S}\rightarrow\theta_{E}$ )	–	–	–	–	–	–	58.1	–
Joint Training	68.0	-4.5	65.3	-7.3	45.7	-8.6	56.8	-1.3

Table 8: Training paradigm comparison. We compare four paradigms: Joint trains on mixed {Spatial, AD, UAV, Embodied} data; Sequential performs stage-wise adaptation Spatial

\rightarrow

\rightarrow

UAV

\rightarrow

Embodied; SR follows Spatial

\rightarrow

Merging

\rightarrow

Embodied.

Figure 4: Domain distribution and Token count. This nested pie chart illustrates the proportion of tokens contributed by each domain. The distribution exhibits a long-tailed characteristic, where UAV data constitutes a relatively small proportion of the corpus.

Appendix figures & tables34 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 10: Example 1-2 of MMSI Benchmark.

Figure 11: Example 1-4 of BLINK Benchmark.

Figure 12: Example 1-2 of SAT Benchmark.

Figure 13: Example 3-4 of SAT Benchmark.

Figure 16: Example 1 of MindCube Benchmark.

Figure 17: Example 2 of MindCube Benchmark.

Figure 18: Example 3 of MindCube Benchmark.

Figure 19: Example 1 of NuscenesQA Benchmark.

Figure 20: Example 2 of NuscenesQA Benchmark.

Figure 21: Example 3 of NuscenesQA Benchmark.

Figure 22: Example 1 of NuPlanQA Benchmark.

Figure 23: Example 1 of MAPLM Benchmark.

Figure 24: Example 2 of MAPLM Benchmark.

Figure 25: Example 1 of LingoQA Benchmark.

Figure 26: Example 2 of LingoQA Benchmark.

Figure 27: Example 1 and 2 of UrbanVideo Benchmark.

Figure 28: Example 1 and 2 of AirCopBench.

Figure 29: Example 3 and 4 of AirCopBench.

Figure 32: Example 1 of RoboVQA Benchmark.

Figure 33: Example 2 of RoboVQA Benchmark.

Figure 34: Example 2 of RoboVQA Benchmark.

Figure 35: Example 1 of OpenEQA Benchmark.

Figure 36: Example 2 of OpenEQA Benchmark.

Figure 37: Example 1 of EgoPlan-Bench2 Benchmark.

Figure 38: Example 2 of EgoPlan-Bench2 Benchmark.

Embodied Spatial Reasoning

See all 20 papers

ACE-Brain-0.5: A Unified Embodied Foundational Model for Physical Agentic AI
Jul 5, 2026ACE-Brain Team, :, Ziyang Gong +29Robot Foundation Models Model Merging
ACE-Brain Team
RynnBrain 1.1: Towards More Capable and Generalizable Embodied Foundation Model
Jul 20, 2026Kehan Li, Bohan Hou, Minghao Zhu +283D Visual Grounding Cross-Embodiment Robot Learning
1DAMO Academy, Alibaba Group
Vesta: A Generalist Embodied Reasoning Model
Jun 18, 2026Johan Bjorck, Zhiqi Li, Yunze Man +29Embodied Reasoning Robot Foundation Models
Co-first authors (alphabetical order). · University of Illinois Urbana-Champaign · Core authors (alphabetical order). +8

Jul 5, 2026J/K move · Enter open · S save

ACE-Brain-0.5: A Unified Embodied Foundational Model for Physical Agentic AI

ACE-Brain Team, :, Ziyang Gong, Haoming Gu+28

ACE-Brain Team

Robot Foundation Models Model Merging Embodied AI Robotic Perception Embodied Reasoning Agentic AI for Robotics

Open paper AskPDF arXiv

Embodied AI is moving from isolated perception or action modules toward physical agents that understand, plan under goals, act through robot bodies, monitor progress, and improve from experience. Existing systems address this loop only in parts: end-to-end policies generate actions but often lack spatial reasoning, planning, and execution assessment, while robot-agent systems orchestrate tools or specialists but do not learn a shared representation. This fragmentation limits general Physical Agentic AI. We present ACE-Brain-0.5, a unified embodied foundation model that organizes robot intelligence into five coupled functions: spatial perception, decision making, embodied interaction, self-monitoring, and self-improvement. Built on ACE-Brain-0, which established spatial intelligence as a shared scaffold across robot platforms, ACE-Brain-0.5 extends an understanding-centric model into a closed-loop foundation model. A single 8B backbone instantiates the first four functions: grounding objects and affordances, reasoning over 3D and egocentric spatial relations, decomposing instructions into subgoals, generating navigation and manipulation actions, and estimating progress for verification and recovery. To unify these capabilities without cross-task interference, we introduce SSR+, which extends Scaffold-Specialize-Reconcile with a Reactivate stage after task-vector merging. The fifth function, self-improvement, is realized by a companion framework that updates external execution state, including task schemas, spatial memory, and failure-recovery cases, from rollouts. Across fifteen benchmarks, ACE-Brain-0.5 improves over ACE-Brain-0 on 14 of 18 spatial perception and grounding benchmarks, achieves competitive navigation and manipulation performance, and provides strong progress estimation in ID and OOD settings. Together, these results mark an early step toward general Physical Agentic AI.

Abstract

Embodied Spatial Reasoning

Explore similar work

ACE-Brain-0.5: A Unified Embodied Foundational Model for Physical Agentic AI

RynnBrain 1.1: Towards More Capable and Generalizable Embodied Foundation Model

Vesta: A Generalist Embodied Reasoning Model