From Kinematics to Dynamics: Learning to Refine Hybrid Plans for Physically Feasible Execution
Authors: Lidor Erez, Shahaf S. Shperberg, Ayal Taitler
Organizations: Department of Industrial Engineering and Management, Ben-Gurion University of the Negev, Be’er Sheva, Israel · Stein Faculty of Computer and Information Science, Ben-Gurion University of the Negev, Be’er Sheva, Israel
In many robotic tasks, agents must traverse a sequence of spatial regions to complete a mission. Such problems are inherently mixed discrete-continuous: a high-level action sequence and a physically feasible continuous trajectory. The resulting trajectory and action sequence must also satisfy problem constraints such as deadlines, time windows, and velocity or acceleration limits. While hybrid temporal planners attempt to address this challenge, they typically model motion using linear (first-order) dynamics, which cannot guarantee that the resulting plan respects the robot's true physical constraints. Consequently, even when the high-level action sequence is fixed, producing a dynamically feasible trajectory becomes a bi-level optimization problem. We address this problem via reinforcement learning in continuous space. We define a Markov Decision Process that explicitly incorporates analytical second-order constraints and use it to refine first-order plans generated by a hybrid planner. Our results show that this approach can reliably recover physical feasibility and effectively bridge the gap between a planner's initial first-order trajectory and the dynamics required for real execution.
Figures & tables
Figure 1: The refinement framework. A hybrid planner generates a first-order plan, which is converted to a graph. A GNN-based RL agent adjusts velocity bounds, and an SOCP solver generates a candidate trajectory validated by MTV.
Figure 2: Shared-encoder actor-critic architecture. Region, node, and edge features are encoded separately and concatenated before being processed by a message-passing GNN. The resulting node embeddings are used by separate actor and critic heads to produce Gaussian policy parameters and value estimates.
Hyperparameter
Value
Hyperparameter
Value
Learning Rate
10−4
Target KL Div
0.015
Clip Ratio ( ϵ0 )
0.2
Discount ( γ )
0.99
Min Clip Ratio ( ϵmin )
0.05
GAE λ
0.95
Batch Size
16
Horizon
8
Entropy Coeff.
0.01
Rollout Size
64
Value Loss Coeff.
0.5
Epochs
1
Table 1: PPO Hyperparameters
Figure 3: Training curves across domains. The agent consistently improves its return and reaches a stable regime.
Figure 4: Per-instance makespan comparison. Blue points show refined makespan with ±1 std error bars. Orange and green markers denote the baseline makespan with aˉ={0.9,0.995} .
Domain
Instances
Makespan Ratio
AUV-2D
20
1.20±0.008
Norm-AUV-2D
20
1.23±0.006
OnAir-Refuel
10
1.15±0.007
Sailing
10
1.14±0.006
Table 2: Performance statistics. We report the number of problem instances and the average makespan ratio of the final refined feasible solution relative to the initial infeasible plan. Values are mean ± std. dev. Initial plan feasibility was 0% for all domains; refined plan feasibility was 100%.
Physical-world interaction is inherently dynamic, as environments can evolve during execution, requiring agents to adapt their plans under non-stationary conditions. We study this challenge through long-horizon embodied planning under environment deviations and execution uncertainty. Existing embodied-task benchmarks can expose such failures, but these failures are usually treated as evaluation outcomes instead of learnable signals for training agents to recover. In this work, we introduce DynamicEnvPlan, a closed-loop framework for high-level planning in dynamic environments. It extends embodied task execution with humanoid agents, high-level primitive skills, structured semantic memory, and controllable perturbations. Our data synthesis design consists of planning, perturbation, and guarded correction modules that turn dynamic execution states into recovery-oriented traces. The resulting traces are used for staged supervised fine-tuning, enabling the planner to learn from both nominal execution and perturbed recovery trajectories. Using 104 task-scene combinations spanning i.i.d., compositional generalization, and out-of-distribution settings for fine-tuning and evaluation, DynamicEnvPlan boosts success rate from 33.3% for the base planner to 76.2%, while improving across all seven evaluation metrics critical to physical-world interaction, including safety and affordance compliance.
Hao Yuan, Yuxin Wang, Lei Ji +1
University of Chinese Academy of Sciences · †Independent researchers. · 3Beijing Academy of Artificial Intelligence (BAAI)
In this work, a novel method for planar task and motion planning based on hybrid modeling is proposed. By virtue of a discrete variable which models local constraint satisfaction and enables local feasibility analysis, the proposed control architecture unifies planning with control design. Concurrently, control barrier functions are designed on a transformed disk version of the original nonconvex and geometrically complex robotic workspace, thus amending the issue of deadlocks. Simulations of the proposed method indicate effective handling of multiple overlapping spatio-temporal tasks even in the face of input saturation.
Panagiotis Rousseas, Dimos V. Dimarogonas
∗KTH Royal Institute of Technology, School of Electrical Engineering and Computer Science, Division of Decision and Control Systems
Operating constrained dynamical systems requires controllers to efficiently solve complex tasks while enforcing recursive feasibility and physical constraints. To address these competing requirements, we present Feasible Action for Optimal Control (FAOC), a novel control framework integrating Reinforcement Learning (RL) and Optimal Control (OC). The core contribution is a computationally efficient, optimization-based mapping algorithm that transforms the RL agent's action from a static abstract set into a state-dependent feasible parameter set of the Optimal Control Problem (OCP), guaranteeing instantaneous parameter feasibility. When paired with invariant terminal sets, FAOC guarantees strict recursive feasibility and safe operation, effectively combining the predictable safety of OC with the behavioral flexibility of RL. Unlike prior work, the abstract action space does not require expert tuning, nor is the OCP formulation compromised by the inability of RL to guarantee feasibility. We evaluate FAOC on real-time motion planning for robot table tennis, where simulated experiments demonstrate superior sample efficiency and closed-loop performance compared to state-of-the-art baselines. We open-source the used implementation of the mapping algorithm and OCP for motion planning https://github.com/SonyResearch/feasible_action_for_optimal_control.
Stefan Richter, Alberto Giammarino, Guillem Torrente +2