cs.LGNov 10, 2026

Temporal Behavior Trees for Reinforcement Learning: Specifications as Rewards, Curricula, and Diagnostics

Authors: Vladimir Krsmanovic, Bernd Finkbeiner

Organizations: CISPA Helmholtz Center for Information Security · Technical University of Munich

Abstract

Reinforcement learning from formal specifications often separates task definition, reward shaping, curriculum design, and policy diagnostics into different mathematical objects. This fragmentation requires additional conversion steps and discards structure already present in the specification, such as subtask boundaries, ordering constraints, and quantitative measures. We introduce TBTCL (Temporal Behavior Tree Centric Learning), a framework that uses Temporal Behavior Trees (TBTs) as a unified interface for specification-guided reinforcement learning. A single TBT specifies the task, induces dense robustness-based rewards, defines a curriculum, and supports better understanding of learned policies. By adapting TBT syntax and semantics for specification-guided reinforcement learning, we provide a formalism that is more expressive than popular specification languages such as LTL and inherently hierarchical. We present algorithms to automatically extract a curriculum DAG and derive dense, robustness-based rewards directly from TBT semantics. Furthermore, we utilize TBT trace segmentation to build understandable competence profiles and automatically adapt the reward toward bottlenecks. We evaluate TBTCL on six specifications across discrete and continuous domains and show that Temporal Behavior Trees can serve as a specification, a reward, a curriculum, and a diagnostic.

Figures & tables

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. A Reward-Petri-Net Interpretation of Temporal Behavior Trees

    Jun 19, 2026Till Schmeil, Günther Waxenegger-Wilfing, Sebastian SchirmerReinforcement LearningReward Shaping

  2. Learning from Imperfect Demonstrations via Temporal Behavior Tree-Guided Trajectory Repair

    Apr 5, 2026Aniruddh G. Puranic, Sebastian Schirmer, John S. Baras +1Robot Imitation LearningRL for Robotics