Language Model-Based Control

Ask about this topic

Overview

The field studies both how language models can control external systems and how controllers can regulate a model’s own behavior and outputs. Common designs use an LLM for high-level planning, intent interpretation, or code generation, paired with faster controllers such as MPC, reinforcement learning, PID, and deterministic tools; retrieval, world models, and formal safety shields help ground or check proposed actions. Researchers evaluate these systems in settings such as robot manipulation, autonomous driving, traffic signals, networks, HVAC, and software agents, using benchmarks including CARLA, SUMO, LIBERO, nuPlan, and OSWorld as well as some hardware tests. Results show useful planning and adaptation, but reliable deployment remains limited by latency, unsafe or incorrect actions, and gaps between simulation performance and robust real-world control.

Latest papers

CardsList

180 papers29 papers in the last four weeks, up 71% on the four weeks before. 0.2% of all new papers.

  1. Large Language Models as Optimization Controllers: Adaptive Continuation for SIMP Topology Optimization

    Mar 26, 2026Shaoliang Yang, Jun Wang, Yunsheng WangTopology OptimizationLarge Language Model-Guided Optimization

    Department of Mechanical Engineering, Santa Clara University, Santa Clara, CA 95053, USA

  2. Toward Full Autonomous Laboratory Instrumentation Control with Large Language Models

    Mar 25, 2026Yong Xie, Kexin He, Andres Castellanos-GomezLaboratory AutomationAutonomous Experimentation

    Key Laboratory of Wide Band-Gap Semiconductor Technology, School of Advanced Materials and Nanotechnology, Xidian University, Xi’an 710071, China · 2D Foundry Research Group. Instituto de Ciencia de Materiales de Madrid (ICMM-CSIC), Madrid, E-28049, Spain.

  3. AI-driven Intent-Based Networking Approach for Self-configuration of Next Generation Networks

    Mar 24, 2026Md. Kamrul Hossain, Walid AljobyIntent-Based NetworkingLanguage Model-Based Control

    Information and Computer Science Department, King Fahd University of Petroleum and Minerals, Dhahran 31261, Saudi Arabia · IRC for Intelligent Secure Systems, King Fahd University of Petroleum and Minerals, Dhahran 31261, Saudi Arabia

  4. Describe-Then-Act: Proactive Agent Steering via Distilled Language-Action World Models

    Mar 24, 2026Massimiliano Pappa, Luca Romani, Valentino Sacco +5Efficient Inference for World Action ModelsAI Agent Safety

    Sapienza University of Rome, Italy · Inria, Univ. Grenoble Alpes, CNRS, LJK

  5. PowerLens: Taming LLM Agents for Safe and Personalized Mobile Power Management

    Mar 20, 2026Xingyu Feng, Chang Sun, Yuzhu Wang +5Language Model-Based ControlUser Preference Modeling

    China University of Geosciences (Beijing) · Shenzhen University · Hong Kong University of Science and Technology +1

  6. Robotic Agentic Platform for Intelligent Electric Vehicle Disassembly

    Mar 19, 2026Zachary Allen, Max Conway, Lyle Antieau +2Robotic ManipulationLanguage Model-Based Control

    Department of Computer Science, University of Colorado, Boulder.

  7. WASD: Locating Critical Neurons as Sufficient Conditions for Explaining and Controlling LLM Behavior

    Mar 19, 2026Haonan Yu, Junhao Liu, Zhenyu Yan +2LLM InterpretabilityLanguage Model-Based Control

    Key Lab of High Confidence Software Technologies (Peking University), Ministry of Education School of Computer Science, Peking University, Beijing, China

  8. Proactive Rejection and Grounded Execution: A Dual-Stage Intent Analysis Paradigm for Safe and Efficient AIoT Smart Homes

    Mar 17, 2026Xinxin Jin, Zhengwei Ni, Zhengguo Sheng +1Language Model-Based ControlAI Agent Reliability

    School of Information and Electronic Engineering (Sussex Artificial Intelligence Institute), Zhejiang Gongshang University, Hangzhou 310018, China. · Sussex Artificial Intelligent Institute, Zhejiang Gongshang University, Hangzhou 310018, China · Department of Engineering and Design, University of Sussex, BN1 9RH, Brighton, U.K. +3

  9. Reflection-Driven Self-Improving Language Model Agents for Adaptive RAN Slicing

    Mar 11, 2026Yuanhao Li, Haozhe Wang, Geyong Min +2Self-Improving AgentsLanguage Model-Based Control

    Department of Computer Science, Faculty of Environment, Science and Economy, University of Exeter, Exeter, EX4 4QF, UK. · British Telecom, UK.

  10. LLMs for Text-Based Exploration and Navigation Under Partial Observability

    Mar 10, 2026Stephan Sandfuchs, Maximilian Melchert, Jörg FrochteLanguage Model-Based ControlRobot Navigation

    AKIS – Interdisciplinary Institute for Applied AI and Data Science Ruhr, Bochum University of Applied Sciences, Bochum, Germany

  11. Beyond Test-Time Memory: State-Space Optimal Control for LLM Reasoning

    Mar 10, 2026Peihao Wang, Shan Yang, Xijun Wang +8LLM ReasoningLanguage Model-Based Control

    Amazon · The University of Texas at Austin · Georgia Institute of Technology +2

  12. Large Language Model-Assisted Superconducting Qubit Experiments

    Mar 9, 2026Shiheng Li, Jacob M. Miller, Phoebe J. Lee +14Autonomous ExperimentationLanguage Model-Based Control

    Department of Physics, University of Chicago, Chicago, IL 60637, USA · Pritzker School of Molecular Engineering, University of Chicago, Chicago, IL 60637, USA · Pico group, QTF Centre of Excellence, Department of Applied Physics, Aalto University School of Science, P.O. Box 13500, 00076 Aalto, Finland +2

  13. RACAS: Controlling Diverse Robots With a Single Agentic System

    Mar 5, 2026Dylan R. Ashley, Jan Przepióra, Yimeng Chen +5Agentic AI for RoboticsRobotic Control

    Center of Excellence in Generative AI, King Abdullah University of Science and Technology (KAUST), Saudi Arabia. · Dalle Molle Institute for Artificial Intelligence Research (IDSIA), Switzerland. · Università della Svizzera italiana (USI), Switzerland. +3

  14. From Language to Action: Can LLM-Based Agents Be Used for Embodied Robot Cognition?

    Mar 3, 2026Shinas Shaji, Fabian Huppertz, Alex Mitrevski +1Large Language Model-Based Robot PlanningLanguage Model-Based Control

    Institute of AI and Autonomous Systems (A2S), Hochschule Bonn-Rhein-Sieg, Sankt Augustin, Germany · Fraunhofer Institute for Intelligent Analysis and Information Systems, Sankt Augustin, Germany

  15. DriveCode: Domain Specific Numerical Encoding for LLM-Based Autonomous Driving

    Mar 1, 2026Zhiye Wang, Yanbo Jiang, Rui Zhou +5Autonomous DrivingNumerical Reasoning in Language Models

    School of Information Science and Engineering, Lanzhou University, Lanzhou, China · State Key Laboratory of Intelligent Green Vehicle and Mobility, Tsinghua University, Beijing, China · The School of Vehicle and Mobility, Tsinghua University, Beijing, China +2

  16. Wireless Power Control Based on Large Language Models

    Feb 28, 2026Jiacheng Wang, Yucheng Sheng, Le Liang +2Wireless Resource AllocationLanguage Model-Based Control

    School of Information Science and Engineering, Southeast University, Nanjing 210096, China · Department of Electrical and Computer Engineering, University of California, Santa Cruz, CA 95064, USA

  17. SAGE-LLM: Towards Safe and Generalizable LLM Controller with Fuzzy-CBF Verification and Graph-Structured Knowledge Retrieval for UAV Decision

    Feb 27, 2026Wenzhe Zhao, Yang Zhao, Ganchao Liu +4Language Model-Based ControlLLM Agent Safety

    School of Artificial Intelligence, OPtics and ElectroNics (iOPEN), Northwestern Polytechnical University, Xi’an 710072, China · Institute of Artificial Intelligence (TeleAI), China Telecom, China

  18. Dynamic Personality Adaptation in Large Language Models via State Machines

    Feb 25, 2026Leon Pielage, Ole Hätscher, Mitja Back +2Medical EducationPersonality Modeling in Language Models

    Institute for Geoinformatics, University of Münster, 48149 Münster, Germany · Faculty of Mathematics and Computer Science, University of Münster, 48149 Münster, Germany · Department of Psychology, University of Münster, 48149 Münster, Germany +1

  19. Two-Stage Active Distribution Network Voltage Control via LLM-RL Collaboration: A Hybrid Knowledge-Data-Driven Approach

    Feb 25, 2026Xu Yang, Chenhui Lin, Xiang Ma +4Power SystemsRL Control

    Sichuan Energy Internet Research Institute, Tsinghua University, Chengdu 610213, China. · State Grid Zhejiang Electric Power Co., Ltd., Jinhua Power Supply Company, Jinhua 321017, China.

  20. Large Language Model-Assisted UAV Operations and Communications: A Multifaceted Survey and Tutorial

    Feb 23, 2026Yousef Emami, Hao Zhou, Radha Reddy +5Language Model-Based ControlUAV Navigation

  21. Agentic Wireless Communication for 6G: Intent-Aware and Continuously Evolving Physical-Layer Intelligence

    Feb 19, 2026Zhaoyang Li, Xingzhi Jin, Junyu Pan +2Wireless Communications6G Networks

    College of Information Science and Electronic Engineering, Zhejiang University, Hangzhou, China.

  22. Toward Autonomous O-RAN: A Multi-Scale Agentic AI Framework for Real-Time Network Control and Management

    Feb 15, 2026Hojjat Navidan, Mohammad Cheraghinia, Jaron Fontaine +5Multi-Agent OrchestrationLanguage Model-Based Control

    IDLab, Department of Information Technology at Ghent University - imec, Ghent, Belgium · Department of Electrical and Computer Engineering, Princeton University, USA

  23. Demonstration-Free Robotic Control via LLM Agents

    Jan 28, 2026Brian Y. Tsui, Alan Y. Fang, Tiffany J. HwuRobotic ControlLanguage Model-Based Control

    All authors are independent researchers.

  24. One Prompt, Many Sounds: Modeling Listener Variability in LLM-Based Equalization

    Jan 14, 2026Ioannis Stylianou, Jon Francombe, Pablo Martinez-Nuevo +2Language Model-Based Control

    Bang & Olufsen A/S, Struer, Denmark · Department of Electronic Systems, Aalborg University, Aalborg, Denmark · Pioneer Centre for AI, Copenhagen, Denmark

  25. Agentic Physical AI toward a Domain-Specific Foundation Model for Energy Systems: A Case Study on Nuclear Reactor Control

    Dec 29, 2025Yoon Pyo Lee, Samrendra Roy, Kazuma Kobayashi +5Language Model-Based ControlSafety-Critical Control

    The Grainger College of Engineering, Nuclear, Plasma & Radiological Engineering, University of Illinois Urbana-Champaign, Urbana, IL, USA · Department of Nuclear Engineering, Hanyang University, Republic of Korea · University of Texas - El Paso, University Ave, El Paso, USA +4

  26. LAMP: Language-Assisted Motion Planning for Controllable Video Generation

    Dec 3, 2025Muhammed Burak Kizil, Enes Sanli, Niloy J. Mitra +3Camera-Controlled Video GenerationTrajectory Generation

    Koç University · University College London · Adobe Research +1

  27. GraphPilot: Grounded Scene Graph Conditioning for Language-Based Autonomous Driving

    Nov 14, 2025Fabian Schmidt, Markus Enzweiler, Abhinav ValadaVision-Language ModelsVLMs for Autonomous Driving

    Esslingen University of Applied Sciences · University of Freiburg

  28. LLM-Flock: LLM-Driven Decentralized Multi-Robot Flocking via Influence-Based Consensus with Collision Avoidance

    May 10, 2025Peihan Li, Lifeng ZhouMulti-Robot Formation ControlSwarm Robotics

    Drexel University, United States

  29. Exploring Context-aware and LLM-driven Locomotion for Immersive Virtual Reality

    Apr 24, 2025Suleyman Ozdel, Kadir Burak Buldu, Enkelejda Kasneci +1Human-Computer InteractionLanguage Model-Based Control

    Human-Centered Technologies for Learning, Technical University of Munich, Germany · Munich Center for Machine Learning, Munich, Germany

  30. Evolutionary Guided Decoding: Iterative Value Refinement for LLMs

    Mar 4, 2025Zhenhua Liu, Lijun Li, Ruizhe Chen +5LLM AlignmentLanguage Model Decoding

    Shanghai Artificial Intelligence Laboratory · Soochow University · Zhejiang University +1