PlatoLTL: Scaling LTL-Guided Multi-Task RL

Jacques Cloete; Mathias Jackermeier; Ioannis Havoutis; Alessandro Abate

cs.LGJan 30, 2026

PlatoLTL: Scaling LTL-Guided Multi-Task RL

Authors: Jacques Cloete, Mathias Jackermeier, Ioannis Havoutis, Alessandro Abate

Organizations: Oxford Robotics Institute University of Oxford Oxford, United Kingdom · Department of Computer Science University of Oxford Oxford, United Kingdom

Abstract

Linear temporal logic (LTL) has emerged as a powerful formalism for specifying structured, temporally extended tasks in multi-task reinforcement learning (RL). However, while existing approaches in LTL-guided multi-task RL demonstrate success in simple environments, they suffer from challenges in representation learning and exploration efficiency when applied to high-dimensional environments with parameterized specifications. We present PlatoLTL, which elevates state-of-the-art methods to address both challenges. We model atomic propositions as instances of atomic predicates and inject task parameters directly into the goal embedding to enable efficient generalization. We also leverage simple priors on closeness to predicate satisfaction to enable and accelerate learning of complex tasks without biasing the optimal policy. We validate our approach on challenging environments including robotic manipulation and multi-drone navigation.

Figures & tables

Figure 1: During training, we convert LTL specifications into LDBAs and use depth-first search to extract reach-avoid sequences corresponding to LTL satisfaction. P 2 BRS combines these sequences with per-predicate robustness metrics to construct dense potential functions to guide policy learning.

Figure 2: Evaluation curves during training for finite -horizon specifications.

		Success Rate (%) ( $\uparrow$ )					Average Episode Length on Success ( $\downarrow$ )
	$\varphi$	Discretize	Concatenate	SIAMS	Plato	Plato + P 2 BRS	Discretize	Concatenate	SIAMS	Plato	Plato + P 2 BRS
Go1RGBZoneEnv	$\varphi_{1}$	$72.4^{+2.2}_{-2.0}$	$97.9^{+0.5}_{-0.7}$	$96.4^{+1.0}_{-1.4}$	$99.0^{+0.2}_{-0.3}$	$\mathbf{99.1}^{+0.3}_{-0.3}$	$441.4^{+28.3}_{-23.4}$	$178.9^{+11.5}_{-9.6}$	$242.5^{+32.2}_{-28.7}$	$117.9^{+4.0}_{-3.7}$	$\mathbf{116.3}^{+4.1}_{-2.2}$
	$\varphi_{2}$	$53.6^{+2.4}_{-3.4}$	$94.8^{+0.6}_{-0.8}$	$90.2^{+2.0}_{-3.8}$	$\mathbf{98.4}^{+0.2}_{-0.2}$	$\mathbf{98.4}^{+0.3}_{-0.2}$	$313.4^{+14.5}_{-13.1}$	$175.1^{+9.5}_{-8.0}$	$197.3^{+19.0}_{-15.6}$	$\mathbf{112.5}^{+2.7}_{-2.4}$	$114.2^{+2.6}_{-2.4}$
	$\varphi_{3}$	$97.9^{+0.5}_{-0.5}$	$99.2^{+0.3}_{-0.7}$	$98.8^{+0.3}_{-0.3}$	$99.0^{+0.2}_{-0.2}$	$\mathbf{99.3}^{+0.2}_{-0.2}$	$161.7^{+6.3}_{-9.3}$	$128.6^{+5.1}_{-3.9}$	$143.9^{+11.5}_{-7.7}$	$\mathbf{97.2}^{+3.2}_{-2.5}$	$101.4^{+2.3}_{-2.3}$
	$\varphi_{4}$	$35.9^{+1.5}_{-2.7}$	$86.1^{+2.0}_{-1.4}$	$78.3^{+3.8}_{-4.6}$	$95.6^{+0.5}_{-0.4}$	$\mathbf{96.5}^{+0.5}_{-0.6}$	$203.2^{+16.2}_{-14.1}$	$135.3^{+11.5}_{-8.1}$	$150.9^{+21.0}_{-14.5}$	$81.2^{+3.0}_{-2.3}$	$\mathbf{80.4}^{+3.6}_{-1.9}$
	$\varphi_{5}$	$36.1^{+2.1}_{-2.6}$	$92.7^{+0.9}_{-1.2}$	$86.8^{+1.3}_{-1.9}$	$97.3^{+0.3}_{-0.3}$	$\mathbf{97.6}^{+0.4}_{-0.4}$	$288.9^{+27.0}_{-26.6}$	$126.5^{+16.6}_{-10.9}$	$150.8^{+23.2}_{-18.7}$	$75.2^{+4.0}_{-3.6}$	$\mathbf{74.7}^{+3.3}_{-1.8}$

Table 1: Evaluation results for finite -horizon LTL specifications.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 3 — Figure 3: LDBA for the formula $(\mathop{\mathsf{F}}\mathop{\mathsf{G}}\mathsf{a})\lor\mathop{\mathsf{F}}\mathsf{b}$ .

		Continuous			Discrete
	Parameter	Go1RGBZoneEnv	X2DronesEnv	FrankaCubesEnv	FalloutWorld
PPO	Number of parallel envs	$2048$
	Steps per env per update	$64$
	Epochs	$5$			$4$
	Batch size	$32768$			$8192$
	Discount factor	$0.995$			$0.99$

Appendix

Table 2: Hyperparameters for PPO and P 2 BRS. Values spanning multiple columns indicate identical values across those environments.

		Average Visits ( $\uparrow$ )
	$\psi$	Discretize	Concatenate	SIAMS	Plato	Plato + P 2 BRS
Go1RGBZoneEnv	$\psi_{1}$	$23.1^{+3.5}_{-2.2}$	$303.0^{+206.5}_{-148.6}$	$32.1^{+29.6}_{-9.1}$	$837.9^{+59.3}_{-57.3}$	$\mathbf{871.8}^{+76.6}_{-78.4}$
	$\psi_{2}$	$22.3^{+3.3}_{-1.9}$	$293.4^{+205.6}_{-145.7}$	$30.2^{+27.0}_{-8.1}$	$808.8^{+56.1}_{-59.6}$	$\mathbf{875.6}^{+81.9}_{-73.9}$
	$\psi_{3}$	$11.7^{+1.7}_{-0.9}$	$253.2^{+185.8}_{-127.6}$	$26.1^{+27.3}_{-8.2}$	$816.3^{+71.6}_{-53.8}$	$\mathbf{848.1}^{+82.1}_{-81.7}$
	$\psi_{4}$	$18.5^{+2.0}_{-1.4}$	$213.6^{+155.2}_{-109.5}$	$34.8^{+25.3}_{-7.8}$	$787.1^{+57.3}_{-65.9}$	$\mathbf{836.4}^{+73.4}_{-89.5}$
	$\psi_{5}$	$10.2^{+0.6}_{-0.7}$	$16.2^{+1.7}_{-1.0}$	$12.8^{+1.2}_{-1.7}$	$29.2^{+1.5}_{-2.1}$	$\mathbf{29.9}^{+0.8}_{-0.8}$

Appendix

Table 3: Evaluation results for infinite -horizon LTL specifications.

Figure 4: Robustness scales during training for Plato + P 2 BRS. The dotted red line represents the frozen unit scale under the ablation of no adaptive robustness scale calibration.

Panel of Figure 5 — Figure 5: Visualizations of example environment reset configurations, including predicate instances.

Table 4: Template LTL specifications used for evaluation. The notation

\mathsf{f}_{i}

represents the

i

-th instance of the predicate

\mathsf{f}

Table 5: Details regarding the curriculum for sampling reach-avoid sequences during training.

Panel of Figure 6 — Figure 6: Example trajectories for Plato with P 2 BRS, truncated to the first 4 subgoals (left to right).

Panel of Figure 7 — Figure 7: Principal component analyses of learned proposition embeddings in FrankaCubesEnv . Normalized parameters for a predicate instance $\mathsf{pos}(x,y,z)$ , $\mathsf{rot}(\phi,\theta,\zeta)$ , or $\mathsf{has}(r,g,b)$ are mapped to the RGB value $(r,g,b)\in[0,1]^{3}$ .

Panel of Figure 8 — Figure 8: Principal component analyses of learned proposition embeddings in FalloutWorld . The normalized parameter for a predicate instance $\mathsf{rad}(\eta)$ is mapped to the RGB value $(1,1-\eta,1-\eta)\in[0,1]^{3}$ such that color transitions from white to red as radiation intensity threshold increases, and normalized parameters for an instance $\mathsf{loc}(x,y)$ are mapped to the RGB value $(0,x,y)\in[0,1]^{3}$ . Note that $\mathsf{rad}(\eta)$ only yields $13$ unique data points for Discretize, since it only has $13$ bins under our discretization.

Panel of Figure 9 — Figure 9: Evaluation curves for finite -horizon specifications using LTL2Action.

Panel of Figure 10 — Figure 10: Evaluation curves for finite -horizon specifications under each P 2 BRS ablation.

Explore similar work

Jaxolotl: A Unified High-Performance Benchmark Suite for LTL-Based Multi-Task RL
Sep 29, 2026Mathias Jackermeier, Jacques Cloete, Alessandro AbateMulti-Turn Reinforcement Learning Linear Temporal Logics
Department of Computer Science, University of Oxford · Oxford Robotics Institute, University of Oxford
Live LTL Progress Tracking: Towards Task-Based Exploration
Apr 18, 2026Noel Brindise, Cedric Langbort, Melkior OrnikLinear Temporal Logics Trajectory Generation
Coordinated Science Laboratory, University of Illinois Urbana-Champaign, Urbana, USA
SpecRLBench: A Benchmark for Generalization in Specification-Guided Reinforcement Learning
Apr 27, 2026Zijian Guo, İlker Işık, H. M. Sabbir Ahmad +1Offline Reinforcement Learning Linear Temporal Logics
Boston University

Sep 29, 2026, cs.LGJ/K move · Enter open · S save

Jaxolotl: A Unified High-Performance Benchmark Suite for LTL-Based Multi-Task RL

Mathias Jackermeier, Jacques Cloete, Alessandro Abate

Department of Computer Science, University of Oxford · Oxford Robotics Institute, University of Oxford

Multi-Turn Reinforcement Learning Linear Temporal Logics

Open paper AskPDF arXiv

Training agents to follow arbitrary instructions is an important goal of multi-task reinforcement learning (RL). Linear temporal logic (LTL) provides a precise and structured formalism for specifying instructions to agents, and has been successfully adopted for training generalist multi-task policies. However, differences in implementations, task distributions, and evaluation protocols make existing methods difficult to compare, while high computational costs limit the scale and statistical reliability of experiments. We introduce Jaxolotl, a unified high-performance benchmark suite for multi-task LTL-RL to address these concerns. Jaxolotl provides a modular, end-to-end JAX implementation of six representative algorithms and four environments, together with newly curated task suites and a standardised, statistically robust evaluation protocol. By precompiling symbolic task representations into static arrays, Jaxolotl enables fully JIT-compiled training and evaluation, achieving end-to-end speedups of up to

220\times

and supporting controlled comparisons at substantially greater experimental scale. We use this framework to systematically evaluate existing approaches, revealing complementary strengths and limitations: general methods capable of non-myopic reasoning struggle as the number of propositions grows, while methods with stronger scaling rely on environment-specific assumptions and suffer from myopia.

Abstract

Explore similar work

Jaxolotl: A Unified High-Performance Benchmark Suite for LTL-Based Multi-Task RL

Live LTL Progress Tracking: Towards Task-Based Exploration

SpecRLBench: A Benchmark for Generalization in Specification-Guided Reinforcement Learning