Instruction Following

Ask about this topic

Overview

Instruction-following research asks whether language models and agents can carry out what users request, including exact formats, multiple constraints, changing requirements, and rules that override other instructions. Researchers test these abilities with automatically checked benchmarks and realistic settings such as long conversations, tool use, coding, speech, and embodied tasks. To improve performance, they use supervised fine-tuning, preference learning and reinforcement learning with verifiable or rubric-based rewards, as well as prompt compilation, model steering, and inference-time checks. Models can handle many simple instructions, but reliability often falls when constraints interact or accumulate, and improving compliance without harming reasoning, task quality, or general capabilities remains a central challenge.

Latest papers

CardsList

135 papers22 papers in the last four weeks, up 57% on the four weeks before. 0.1% of all new papers.

  1. Instruction Capacity: Behavior and Degradation of Language Models under Controlled Load

    Oct 26, 2026Hennadii KutomanovLLM EvaluationInstruction Following

    Independent researcher

  2. StateWise: Teaching Language Models to Track and Obey Evolving Constraints in Multi-Turn Dialogue

    Oct 24, 2026Samarth Varshney, Ritu YadavInstruction FollowingMulti-Turn Dialogue Management

  3. BARC-Bench: Benign Asks, Risky Consequences

    Oct 23, 2026Kangkang Li, Kelvin Luu, Lily Ma +3AI Agent Safety BenchmarksLLM Agent Safety

    Scale AI

  4. DIAL: Steerable Language Models with Distilled Instruction-Adherence Levels

    Oct 22, 2026Thanh Q. Tran, John Russell Himawan, Arun Verma +4Language Model SteeringInstruction Following

    School of Computing, National University of Singapore, Singapore · Singapore-MIT Alliance for Research and Technology Centre, Singapore · CSAIL, Massachusetts Institute of Technology, USA +1

  5. When LLMs Forget What They Believe: Characterizing State Integrity Collapse in Long-Horizon Dialogues

    Oct 22, 2026Yancheng Zhu, Yifan YuanLLM ReliabilityBehavioral Consistency of LLM Agents

    School of Artificial Intelligence, Shenzhen University, Shenzhen, China

  6. LIFT: Scoped Constraints for Evaluating and Training Instruction Following

    Oct 20, 2026Eran Goldstein, Yuval Belfer, Yonatan Belinkov +1Instruction FollowingGenerative AI Evaluation

    AI21 Labs

  7. From Abstraction to Action: DSLs for Natural Language Abstraction Interpretation and Execution

    Oct 20, 2026Amit Lisha, Reut TsarfatyInstruction FollowingStructured Reasoning in Language Models

    Bar-Ilan University, Israel

  8. Cache2LoRA: Understanding and Improving Context Internalization

    Oct 16, 2026Yi Fang, Gaurav Srivastava, Zhenyu Bi +2LLM Context CompressionKnowledge Augmentation for Language Models

    Department of Computer Science, Virginia Tech, Blacksburg, VA, USA · Department of Biomedical Data Science, Stanford University, Stanford, CA, USA

  9. EduWild: How Teacher-Configured Chatbots Fail in Real-World Use and How to Fix Them

    Oct 16, 2026Ziqi Liu, Ruiwei Xiao, Yuxuan Li +1Educational Dialogue SystemsAI in Education

    Carnegie Mellon University

  10. ReFormBench: Benchmarking the Stability of Response-Conditioned Format Correction in Large Language Models

    Oct 14, 2026Tianxiang Gao, Yahan Li, Chenlu Guo +3LLM EvaluationInstruction Following

    School of Artificial Intelligence, Jilin University · College of Software, Jilin University · Xingchen AGI Lab, China Telecom Artificial Intelligence Technology (Beijing) Co., Ltd +4

  11. Diagnosing and Correcting Reward Hacking in Verifiable Instruction Following

    Oct 14, 2026Sangyeon Yoon, Wooseok Seo, Hamish Ivison +5Reward HackingInstruction Following

    Yonsei University · FuriosaAI · University of Washington

  12. AM-Bench: Understanding Agents under Task Underspecification across Models and Harnesses

    Oct 13, 2026Kaiyu Huang, Yiting Li, Angyang Gu +3LLM Agent EvaluationAI Agent Benchmarks

    Tongji University · The Chinese University of Hong Kong, Shenzhen

  13. The Fragility and Dynamics of LLM Pruning-Recovery Steps

    Oct 11, 2026Luis Frentzen Salim, Komang Rinartha, Chia-Chun Wu +3LLM PruningLLM Compression

    Institute of Information Science, Academia Sinica · National Taiwan University of Science and Technology · AI Research Center, Hon Hai Research Institute

  14. Do Prompt Optimizers Break Requirements They Are Not Scored On? A Preregistered Study

    Oct 9, 2026Sangdo Han, Moonsoo KimPrompt OptimizationInstruction Following

    CEO Business School

  15. SteerablePlex: Can We Steer Full-Duplex Models?

    Oct 8, 2026Haolong Zheng, Maike Züfle, Dominik Macháček +6Full-Duplex Spoken Dialogue SystemsLanguage Model Steering

    University of Illinois Urbana-Champaign · Karlsruhe Institute of Technology · Charles University +3

  16. Thinking Inertia: LLMs Keep Thinking When Told Not To

    Oct 8, 2026Dianqiao Lei, Kevin Qinghong Lin, Pan Lu +2LLM EvaluationInstruction Following

    TSINGHUA UNIVERSITY · UNIVERSITY OF OXFORD · STANFORD UNIVERSITY

  17. You Changed Your Mind, The Model Didn't: Demystifying Intent in Multi-Turn Dialogue

    Oct 5, 2026Junle Chen, Wei Chen, Zhengjun Huang +5Intent ClassificationInstruction Following

    HKUST · Tencent

  18. When Verifiable Counts Depend on Wording: Auditing Wording Robustness in Instruction Following

    Oct 4, 2026Qishi Zhan, Seoyeon Jang, Zihan Dong +3LLM EvaluationInstruction Following

    Marquette University · University of California, San Diego · Georgia Tech +3

  19. Beyond Leaderboards: Tokenomics of Agentic Small Language Model Ensembles

    Oct 1, 2026Alexei N. Skurikhin, Emily M. Taylor, Nathan A. DeBardelebenLLM Inference EfficiencyLanguage Model Generation Evaluation

    Los Alamos National Laboratory Los Alamos, New Mexico, USA

  20. Emergent Unfaithfulness: How Alignment Training Causes Language Models to Silently Override Task Faithfulness

    Sep 30, 2026Pardis Sadat Zahraei, Janvijay Singh, Gokhan Tur +1LLM AlignmentInstruction Following

    University of Illinois Urbana-Champaign

  21. Coverage Before Control: Route-Instruction Grounding and Steering for Controllable Retrosynthesis

    Sep 30, 2026Xuemin Chen, Xiaozhuang Song, Xinjian Zhao +2Instruction FollowingInstruction Tuning

    The Chinese University of Hong Kong, Shenzhen

  22. A2Z GameSpec-Bench: How Faithfully Can Coding Agents Generate Games from Game Design Specifications?

    Sep 30, 2026Seonho Lee, Wonryeol Jeong, Alberto Cereser +4AI Coding AgentsAI Agent Benchmarks

    KRAFTON · KAIST · Korea National University of Arts

  23. Learned Reporting Preferences in RLVR Can Conflict with the Current Request

    Sep 29, 2026Yupeng Chang, Wenxuan Zhang, Yuan WuInstruction FollowingReinforcement Learning with Verifiable Rewards

    School of Artificial Intelligence, Jilin University · Singapore University of Technology and Design · Key Laboratory of Symbolic Computation and Knowledge Engineering, Jilin University

  24. Reading Too Much into Context: Passive Exposure Can Steer LLM Decisions

    Sep 27, 2026Yuxiang Zheng, Lin Tian, Marian-Andrei RizoiuLLM Decision-MakingLLM Reliability

    Behavioral Data Science Lab University of Technology Sydney Sydney, Australia

  25. ScopeIF: Improving Scope-Aware Precise Instruction-Following in Large Language Models via Graded Reward Modeling

    Sep 26, 2026Bosi Wen, Yilin Niu, Xiaoying Ning +3Instruction FollowingInstruction Tuning

    The Conversational Artificial Intelligence (CoAI) Group, Tsinghua University · Zhipu AI

  26. Counterfactual Constraint-Conditioned On-Policy Distillation for Multi-Constraint Instruction Following

    Sep 23, 2026Yanzhao Zheng, Yuanqiang Yu, Tianze Xu +6Reward ShapingInstruction Following

    Alibaba Group, Hangzhou, China

  27. IBBench-Light: A Paired Evaluation of Task-Conditioned Responses to External Directives

    Sep 12, 2026Kainan Zhou, Zhaoyi Li, Janet Sung +2LLM EvaluationBenchmark Design

    Google LLC Mountain View, CA, USA · Intuit Inc. Mountain View, CA, USA · Fortinet, Inc. Sunnyvale, CA, USA

  28. ContractEval: Query-Conditioned Execution Matching for Procedural Instruction Conformance

    Sep 8, 2026Praphul Singh, Shanu Kumar, Akshat Agarwal +1LLM Agent EvaluationInstruction Following

    Oracle Health AI · MBZUAI

  29. LANTERN: Language Model Assessment on Noisy and Transformed Tasks for Understanding Error and Robustness Nuances

    Sep 7, 2026Vamsi Krishna Kodavali, Rituraj SinghLLM EvaluationInstruction Following

    Samsung

  30. In-Place Instruction Following in Diffusion Language Models

    Sep 7, 2026Zheng Nie, Zherui Li, Jiaming Zhang +3Instruction FollowingDiffusion Language Models

    National University of Singapore · Nanyang Technological University · Peking University