Declarative Reactive Robot Behaviors from Imperative Language Models
Authors: Vincent Berenz, Tasha Kim, Bernhard Schölkopf
Organizations: Max Planck Institute for Intelligent Systems, Tübingen, Germany · University of Oxford, Oxford, United Kingdom · ELLIS Institute Tübingen, Germany · ETH Zürich, Zürich, Switzerland
Large Language Models (LLMs) are predominantly trained on imperative programming languages, and existing work on LLM-based robot programming reflects this: it focuses on goal-directed tasks expressed as plans. However, many robotic applications require not tasks but behaviors--open-ended, continuous patterns of sensory-motor engagement with no termination condition (e.g., greeting visitors, patrolling an area). Such behaviors are better served by declarative and reactive programming, on which LLMs have virtually no training. We investigate whether this gap can be bridged. Using Playful, a reactive scripting language, we show that an LLM-based coding agent can generate correct declarative and reactive programs for robotic behaviors. These programs are produced from under-specified natural language descriptions, given suitable context and a set of reusable programming guidelines. The guidelines encode design principles that redirect the agent from imperative defaults toward proper reactive patterns. Developed once by an expert, they generalize across robots and behaviors. Starting from only the generic robot operating system API, the agent jointly designs both the imperative component (Python modules for sensing, acting, and evaluating) and the declarative component (the reactive script that composes them into behavior logic). The declarative nature of reactive programming supports both incremental refinement through conversational turns and one-shot generation of complete behaviors.
Figures & tables
Fig. 1: A Playful script and its behavior tree (BT) representation. The red and violet arrows explicit show how sensorimotor loops are enforced.
Fig. 2: Experimental validation of the generated behaviors on Pepper. The greeter behavior: Pepper robot transitions from idle scanning A. to active engagement B., driven by the declarative script using the targeting keyword. The guardian behavior: Pepper robot patrols C. and intercepts a visitor with stop gesture D., governed by a script that dynamically arbitrates priority, and reuses primitives from the greeter behavior.
Fig. 3: Natural language instructions that are provided to the coding agent. Behavior 1 was developed both in a single prompt (see Section IV-B ), and incrementally, with each step issued in a separate conversational turn (see Section IV-D ).
Category
Overview
Script as source of truth
The playful script must fully describe the behavior; reading Python should not be necessary.
Targeting and generality
Nodes and evaluations are generic; scheme-type binding is expressed only in the script via targeting .
Large language model (LLM)-based program synthesis turns natural-language goals and examples into executable programs, often using tests or formal constraints.
Programming robots remains challenging for non-experts, as traditional methods require expert knowledge and even block-based interfaces often lack flexibility. Recent work has explored Large Language Models (LLMs) to automatically generate robot programs from natural language, but these systems remain limited by a lack of transparency, missing mechanisms to ensure alignment with user intent, and little support for reuse. We present APPROVE (AI-Powered Programming for Robots with Visual End-User Feedback), an LLM-based multi-modal end-user programming framework that integrates natural language input with a block-based interface and an explicit user confirmation step. Generated programs are visualized using a block-based interface in Blockly, allowing users to confirm, modify, or reject them before execution. Confirmed functions are stored in a library for reuse, gradually building a set of reliable program components. Our approach contributes a human-centered design for LLM-based robot programming that emphasizes user trust, intent alignment, and reusability.
We present Logical Robots, an interactive multi-agent simulation platform where autonomous robot behavior is specified declaratively in the logic programming language Logica. Robot behavior is defined by logical predicates that map observations from simulated radar arrays and shared memory to desired motor outputs. This approach allows low-level reactive control and high-level planning to coexist within a single programming environment, providing a coherent framework for exploring multi-agent robot behavior.
Evgeny Skvortsov, Yilin Xia, Ojaswa Garg +2
Google LLC, Kirkland, WA, USA · University of Illinois Urbana-Champaign, IL, USA · Gonzaga University, Spokane, WA, USA
Natural-language interfaces can lower the barrier to programming robots, but existing systems struggle when users request complex tasks. While large language models (LLMs) perform well with simple commands, they often struggle to generate code for multi-step tasks, decompose high-level instructions, or reuse prior solutions. We present SkillComposer, an interactive natural-language robot programming system for simulation environments that continually learns reusable program abstractions. SkillComposer uses a generate-test architecture in which an LLM iteratively generates and revises robot programs before execution. Successful programs are stored and processed by an online library-learning algorithm that compresses recurring function sequences into reusable macro skills for future tasks. We evaluate SkillComposer through ablation experiments and a user study with 12 participants to determine its effectiveness on manipulation and robot caregiving tasks. The results show that evaluator-guided generation and learned abstractions improve success rates and usability while reducing user effort in natural-language robot programming.
John Woods, Hasti Seifi
School of Computing and Augmented Intelligence, Arizona State University, Tempe, AZ