STL-SVPIO: Signal Temporal Logic guided Stein Variational Path Integral Optimization
Authors: Hongrui Zheng, Zirui Zang, Ahmad Amine, Cristian Ioan Vasile, Rahul Mangharam
Organizations: Department of Electrical and Systems Engineering, University of Pennsylvania, Philadelphia, PA, USA. · Mechanical Engineering and Mechanics Department, Lehigh University, Bethlehem, PA, USA.
Signal Temporal Logic (STL) enables formal specification of complex spatiotemporal constraints for robotic task planning. However, synthesizing long-horizon continuous control trajectories from complex STL specifications is fundamentally challenging due to the nested structure of STL robustness objectives. Existing solver-based methods, such as Mixed-Integer Linear Programming (MILP), suffer from exponential scaling, whereas sampling methods, such as Model-Predictive Path Integral control (MPPI), struggle with sparse, long-horizon costs. We introduce Signal Temporal Logic guided Stein Variational Path Integral Optimization (STL-SVPIO), which reframes STL as a globally informative, differentiable reward-shaping mechanism. By leveraging Stein Variational Gradient Descent and differentiable physics engines, STL-SVPIO transports a mutually repulsive swarm of control particles toward high robustness regions. Our method transforms sparse logical satisfaction into tractable variational inference, mitigating the severe local minima traps of standard gradient-based methods. We demonstrate that STL-SVPIO significantly outperforms existing methods in both robustness and efficiency for traditional STL tasks. Moreover, it solves complex long-horizon tasks, including multi-agent coordination with synchronization and queuing while baselines either fail to discover feasible solutions, or become computationally intractable. Finally, we use STL-SVPIO in agile robotic motion planning tasks with nonlinear dynamics, such as 7-DoF manipulation and half cheetah back flips to show the generalizability of our algorithm.
Figures & tables
Fig. 1: Overview of the STL-SVPIO Optimization Algorithm.
Fig. 2: Selected trajectories of each method through iterations.
Method
# Samples
# Iterations
Runtime (ms)
Robustness
STL-SVPIO
10
20
19.152
0.108
MPPI
10
200
187.125
0.005
SVMPC
64
200
178.389
-0.098
DPI
1024
200
194.492
0.179
STLCG++
N/A
200
926.704
-0.068
MILP
N/A
N/A
324.578
0.495
TABLE I: Reach avoid task parameters and robustness
Fig. 3: Aggregated average robustness and satisfaction rate of methods. All algorithms besides MILP were tested across 100 different random sampling seeds. Each solution with positive robustness counts as a satisfaction. Multiagent Button corresponds to the scenario in Section V-B2 , Multiagent Corridor corresponds to the scenario in Section V-B4 , Multiagent Sync Goals corresponds to the scenario in Section V-B3 , and Long Horizon corresponds to the scenario in Section V-B1 . Gurobi was given a 10-hour time limit to solve the MILP.
Fig. 4: Long horizon trajectory synthesized by STL-SVPIO through a cluttered obstacle field. Dark green and red markers indicate the start and final states, respectively.
Fig. 5: STL-SVPIO synthesizes a multi-agent trajectory that satisfies strict ordering constraints between task events.
Fig. 6: A solution generated by STL-SVPIO in which all agents reach their assigned goals within a shared, unspecified time window.
Fig. 7: STL-SVPIO plans collision-free trajectories that enforce sequential corridor occupancy among multiple agents.
Fig. 8: Rollout snapshots of a torque-controlled Franka Panda trajectory synthesized by STL-SVPIO. The end effector first reaches the yellow goal and subsequently reaches the green goal.
Signal Temporal Logic (STL) is a powerful language for specifying temporally structured robotic tasks. Planning executable trajectories under STL constraints remains difficult when system dynamics and environment structure are not analytically available. Existing methods typically either assume explicit models or learn task-specific behaviors, limiting zero-shot generalization to unseen STL tasks. In this work, we study offline STL planning under unknown dynamics using only task-agnostic trajectory data. Our central design philosophy is to separate logical reasoning from trajectory realization. We instantiate this idea in DAG-STL, a hierarchical framework that converts long-horizon STL planning into three stages. It first decomposes an STL formula into reachability and invariance progress conditions linked by shared timing constraints. It then allocates timed waypoints using learned reachability-time estimates. Finally, it synthesizes trajectories between these waypoints with a diffusion-based generator. This decomposition--allocation--generation pipeline reduces global planning to shorter, better-supported subproblems. To bridge the gap between planning-level correctness and execution-level feasibility, we further introduce a rollout-free dynamic consistency metric, an anytime refinement search procedure for improving multiple allocation hypotheses under finite budgets, and a hierarchical online replanning mechanism for execution-time recovery. Experiments in Maze2D, OGBench AntMaze, and the Cube domain show that DAG-STL substantially outperforms direct robustness-guided diffusion on complex long-horizon STL tasks and generalizes across navigation and manipulation settings. In a custom environment with an optimization-based reference, DAG-STL recovers most model-solvable tasks while retaining a clear computational advantage over direct optimization based on the explicit system model.
Ruijia Liu, Ancheng Hou, Xiao Yu +1
School of Automation and Intelligent Sensing, Shanghai Jiao Tong University, Shanghai, China. · Institute of Artificial Intelligence, Xiamen University, Xiamen, China.
Learning robot policies for tasks with sparse success signals is challenging when completion depends on coordinated actions, precise contact outcomes, or satisfying several conditions together. Intricate physical interactions with the world further complicate these requirements. Prior work using conventional reward shaping mechanisms provides dense feedback but local progress might not translate into eventual task completion. We present Signal Temporal Logic-guided Stein Variational Policy Gradient (STL-SVPG), a population-based method that uses smooth STL robustness as a trajectory-level training objective. Differentiating this objective through the dynamics assigns credit to policy actions according to their effect on the complete task specification, rather than local progress alone. We evaluate the approach on six quadcopter and manipulator tasks that involves event-triggered responses, strictly ordered behavior, responses within specified deadlines, and physical interaction with the world. STL-SVPG achieves the highest mean success rate among the compared methods on five of six benchmarks. Simulation-trained policies trained in simulation transfer temporal and contact task behavior to the real world.
Hongrui Zheng, Cristian Ioan Vasile, Antonio Loquercio +1
Department of Electrical and Systems Engineering, University of Pennsylvania, Philadelphia, PA, USA · Mechanical Engineering and Mechanics Department, Lehigh University, Bethlehem, PA, USA
Sampling-based motion planning has emerged as a powerful approach for robotics, enabling exploration of complex, high-dimensional configuration spaces. When combined with Signal Temporal Logic (STL), a temporal logic widely used for formalizing interpretable robotic tasks, these methods can address complex spatiotemporal constraints. However, traditional approaches rely on min-max robustness measures that focus only on critical time points and subformulae, creating non-smooth optimization landscapes with sharp decision boundaries that hinder efficient tree exploration. We propose RRTη, a sampling-based planning framework that integrates the Arithmetic-Geometric Mean (AGM) robustness measure to evaluate satisfaction across all time points and subformulae. Our key contributions include: (1) AGM robustness interval semantics for reasoning about partial trajectories during tree construction, (2) an efficient incremental monitoring algorithm computing these intervals, and (3) enhanced Direction of Increasing Satisfaction vectors leveraging Fulfillment Priority Logic (FPL) for principled objective composition. Our framework synthesizes dynamically feasible control sequences satisfying STL specifications with high robustness while maintaining the probabilistic completeness and asymptotic optimality of RRT∗. We validate our approach on three robotic systems. A double integrator point robot, a unicycle mobile robot, and a 7-DOF robot arm, demonstrating superior performance over traditional STL robustness-based planners in multi-constraint scenarios with limited guidance signals.
Ahmad Ahmad, Shuo Liu, Roberto Tron +1
Division of Systems Engineering, Boston University, Boston, MA, USA. · Department of Mechanical Engineering, Boston University, Boston, MA, USA. · Department of Electrical and Computer Engineering and the Department of Computer Science at the University of Maryland, College Park, MD, USA.