Language Model-Based Control

The field studies both how language models can control external systems and how controllers can regulate a model’s own behavior and outputs. Common designs use an LLM for high-level planning, intent interpretation, or code generation, paired with faster controllers such as MPC, reinforcement learning, PID, and deterministic tools; retrieval, world models, and formal safety shields help ground or check proposed actions. Researchers evaluate these systems in settings such as robot manipulation, autonomous driving, traffic signals, networks, HVAC, and software agents, using benchmarks including CARLA, SUMO, LIBERO, nuPlan, and OSWorld as well as some hardware tests. Results show useful planning and adaptation, but reliable deployment remains limited by latency, unsafe or incorrect actions, and gaps between simulation performance and robust real-world control.