Embodied AI has emerged as a key paradigm for autonomous systems operating in complex physical environments. While end-to-end models have established strong local capabilities for perception and control, scaling these capabilities to extended, multi-stage tasks remains difficult. Agentic embodied systems address this challenge by introducing high-level planners that organize perception, reasoning, and executable skills over longer interaction horizons. However, existing agentic embodied frameworks typically generate actions for direct execution without explicitly simulating how alternative action sequences shape future observations and task progress, leading agents to commit to suboptimal interactions before considering their long-term consequences. To address this limitation, we propose ParallelWorld, an adaptive-horizon test-time scaling framework that grounds agentic embodied reasoning in the prospective simulation of action outcomes. Specifically, we design an agent system to simulate and evaluate alternative trajectories in parallel before committing to an action. A verifier assesses intermediate state transitions with respect to the task objective, dynamically pruning unpromising branches and guiding subsequent expansion. Based on the evaluated long-horizon outcomes, the agent selects a trajectory, executes only its first action, and replans as new observations become available. Experiments on ESI-Bench and LIBERO-Pro demonstrate improvements in active spatial reasoning and language-conditioned manipulation, respectively, highlighting the effectiveness of prospective planning for both understanding and manipulation tasks.
Figures & tables
Figure 1: Comparison of embodied paradigms. End-to-end systems directly map observations and instructions to actions, while conventional agentic systems introduce a high-level planner to organize executable skills but still commit to actions without explicit prospective evaluation. ParallelWorld augments agentic planning with multi-step world imagination: the planner proposes alternative interaction trajectories, a verifier evaluates their prospective outcomes, and the selected action is executed before replanning from the next observation.
Figure 2: Overview of ParallelWorld. Given a task instruction and the current observation, an agentic planner explores alternative actions through simulator-based or world-model-based imagination. A verifier selects promising branches, while the planner determines whether to continue imagination or commit to execution. Only the first action of the selected trajectory is executed, followed by replanning from the resulting observation. Dashed and solid arrows denote imagination and environment interaction, respectively.
Category
Subcategory
Agent Evaluation
Passive
Active
ParallelWorld
View Hallucination
53.32%
66.82%
71.40%
Partial Occlusion
36.84%
57.89%
77.89%
Perceptual Grounding
Material Transparency
69.72%
71.10%
73.85%
Rigid Containment
55.00%
60.00%
50.00%
Physical Structure
Deformable Objects
37.76%
40.82%
42.86%
Table 1: Quantitative results on ESI-Bench [ 10 ] . We report accuracy for Passive, Active Exploration, and ParallelWorld. All active methods use the same answer-model. Row colors follow the ESI-Bench taxonomy: yellow denotes object representation, purple denotes layout and geometry, blue denotes number representation, and green denotes agents and goal-directed actions.
Method
Spatial
Object
Goal
LIBERO-10
T
S
T
S
T
S
T
S
OpenVLA [ 13 ]
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
π0 [ 1 ]
0.0
0.0
0.0
2.0
0.0
0.0
0.0
0.0
π0.5 [ 12 ]
1.0
20.0
1.0
17.0
2.0
38.0
1.0
8.0
MolmoAct [ 15 ]
0.0
0.0
0.0
6.0
0.0
0.0
6.0
0.0
NORA [ 11 ]
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
Table 2: Quantitative results on LIBERO-Pro [ 40 ] . We report success rates (%) under instruction-redirection (T) and position-swap (S) perturbations across the Spatial , Object , Goal , and LIBERO-10 suites. Each setting contains 100 trials (10 tasks × 10 seeds).
Figure 3: Qualitative analysis of ParallelWorld. (a) Prospective exploration. Active Exploration selects an uninformative view and fails, while ParallelWorld discovers the discriminative observation through future interaction. (b) Prospective manipulation. HarnessVLA [ 36 ] fails with direct execution, whereas ParallelWorld selects an imagined outcome that guides the robot toward a better grasping configuration and achieves success. (c) Limitation in geometric reasoning. ParallelWorld obtains sufficient observations, but the model still fails to infer precise metric relationships between objects. (d) Limitation in physical prediction. The imagined outcome appears plausible, but visual imagination cannot fully capture contact dynamics required for successful physical execution.
Method
Success Rate (%) ↑
Total Time (s) ↓
Direct Execution [ 36 ]
23.3
417.4
Fixed Search Width ( Kh=2 )
10.0
2930.0
Fixed Search Width ( Kh=4 )
36.7
4889.5
Adaptive Search Width (Ours)
73.3
2402.2
Table 3: Ablation of adaptive search width on Spatial-S. Results are averaged over three tasks with 10 seeds each. Imagination Time and Total Time represent the average time cost of imagination and the entire episode, respectively.
Test-time scaling improves the reasoning performance of large language models but often results in token-inefficient overthinking, where models continue reasoning beyond what is necessary for a correct answer. Existing dynamic early-exit methods typically rely on single-step confidence signals, which are often unreliable for detecting reasoning convergence in multi-step settings. To mitigate this limitation, we propose TRACE, a training-free framework for efficient test-time scaling that determines when to terminate reasoning based on temporal aggregation of multi-step evidence rather than instantaneous signals. TRACE detects reasoning convergence over time by aggregating two complementary signals across recent reasoning steps: answer consistency, capturing the persistence of predicted answers, and confidence trajectory, modeling the temporal evolution of model confidence. Benefiting from these two factors, TRACE can accurately determine whether the reasoning process has converged, thereby promptly halting inference and effectively avoiding redundant reasoning steps. Extensive experiments on multiple challenging benchmarks show that TRACE reduces reasoning token usage by 25-30% on average while maintaining accuracy within 1-2% of full-length reasoning, consistently outperforming existing dynamic reasoning methods.
Jiakun Li, Xingwei He, Kefan Li +3
School of Computer Science and Engineering, Beihang University · 3Hangzhou Innovation Institute, Beihang University · 2Qingdao Research Institute, Beihang University
Embodied robotic systems increasingly rely on large language model (LLM)-based agents to support high-level reasoning, planning, and decision-making during interactions with the environment. However, invoking LLM reasoning introduces substantial computational latency and resource overhead, which can interrupt action execution and reduce system reliability. Excessive reasoning may delay actions, while insufficient reasoning often leads to incorrect decisions and task failures. This raises a fundamental question for embodied agents: when should the agent reason, and when should it act? In this work, we propose RARRL (Resource-Aware Reasoning via Reinforcement Learning), a hierarchical framework for resource-aware orchestration of embodied agents. Rather than learning low-level control policies, RARRL learns a high-level orchestration policy that operates at the agent's decision-making layer. This policy enables the agent to adaptively determine whether to invoke reasoning, which reasoning role to employ, and how much computational budget to allocate based on current observations, execution history, and remaining resources. Extensive experiments, including evaluations with empirical latency profiles derived from the ALFRED benchmark, show that RARRL consistently improves task success rates while reducing execution latency and enhancing robustness compared with fixed or heuristic reasoning strategies. These results demonstrate that adaptive reasoning control is essential for building reliable and efficient embodied robotic agents.
Jun Liu, Pu Zhao, Zhenglun Kong +12
Robotics Institute, Carnegie Mellon University · Northeastern University · Harvard University +9
Test-time scaling has become an effective paradigm for improving the reasoning ability of large language models by allocating additional computation during inference. Recent structured approaches have further advanced this paradigm by organizing inference across multiple trajectories, refinement rounds, and verification-based feedback. However, existing structured test-time scaling methods either weakly coordinate parallel reasoning trajectories or rely on noisy historical information without explicitly deciding what should be retained and reused, limiting their ability to balance exploration and exploitation. In this work, we propose TMAS, a framework for scaling test-time compute via multi-agent synergy. TMAS organizes inference as a collaborative process among specialized agents, enabling structured information flow across agents, trajectories, and refinement iterations. To support effective cross-trajectory collaboration, TMAS introduces hierarchical memories: the experience bank reuses low-level reliable intermediate conclusions and local feedback, while the guideline bank records previously explored high-level strategies to steer subsequent rollouts away from redundant reasoning patterns. Furthermore, we design a hybrid reward reinforcement learning scheme tailored to TMAS, which jointly preserves basic reasoning capability, enhances experience utilization, and encourages exploration beyond previously attempted solution strategies. Extensive experiments on challenging reasoning benchmarks show that TMAS achieves stronger iterative scaling than existing test-time scaling baselines, with hybrid reward training further improving scaling effectiveness and stability across iterations. Code and data are available at https://github.com/IQuestLab/tmas.