cs.ETNov 15, 2026

Discovering Programmatic Policies from Reinforcement Learning-Based Traffic Signal Controllers

Authors: Lindong Xie, Yang Zhang, Beiyu Song, Zeren Xing, Flora D. Salim, Edward Chung

Organizations: Department of Electrical and Electronic Engineering The Hong Kong Polytechnic University, Hong Kong, China · School of Computer Science and Engineering University of New South Wales, Sydney, Australia

Abstract

Traffic signal control (TSC) is critical for mitigating urban congestion. Deep reinforcement learning (DRL) has achieved strong performance by learning adaptive policies from traffic observations, but learned neural controllers are often opaque, structurally complex, and difficult to generalize across scenarios. We propose PPL-TSC, a programmatic policy learning framework that distills compact and human-readable TSC policies from DRL teachers. PPL-TSC formulates policy learning as an evolutionary search over phase utility functions (PUFs), which score candidate signal phases and select the highest-utility phase for execution. Each generation consists of four steps: policy generation, policy review, policy calibration, and policy evaluation. In the generation step, a large language model (LLM) acts as a semantics-aware evolutionary operator to propose and refine candidate PUFs. Generated PUFs are screened through explicit checks for syntactic validity, mathematical soundness, and consistency with TSC principles, with invalid ones repaired through iterative feedback. Valid PUFs are then calibrated to improve behavioral fidelity to the DRL teacher, and evaluated using a joint score combining teacher fidelity and traffic-control performance to select candidates for the next generation. Extensive experiments across diverse traffic scenarios and representative DRL-based controllers show that PPL-TSC produces compact and human-readable policies, achieves competitive performance for most teacher families, improves transfer generalization, and substantially reduces computational cost. Code is available at https://github.com/ForrestXie9/PPL-TSC.

Figures & tables

Appendix figures & tables23 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. SignalClaw: LLM-Guided Evolutionary Synthesis of Interpretable Traffic Signal Control Skills

    Apr 7, 2026Da Lei, Feng Xiao, Lu Li +1Traffic Signal ControlAgent Skill Learning

  2. ProcessLight: Process Supervision for Large Language Model Based Traffic Signal Control

    Sep 19, 2026Huaitao Zhao, Tianlong Zhou, Weijie Wang +2Language Model-Based ControlTraffic Signal Control