Instruction Following

Follow this topic Ask about this topic

Momentum

11 papers in the last four weeks, up 10% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 83

All topics

You Changed Your Mind, The Model Didn't: Demystifying Intent in Multi-Turn Dialogue
Oct 5, 2026Junle Chen, Wei Chen, Zhengjun Huang +5Intent Classification Instruction Following
HKUST · Tencent
When Verifiable Counts Depend on Wording: Auditing Wording Robustness in Instruction Following
Oct 4, 2026Qishi Zhan, Seoyeon Jang, Zihan Dong +3LLM Evaluation Instruction Following
Marquette University · University of California, San Diego · Georgia Tech +3
Beyond Leaderboards: Tokenomics of Agentic Small Language Model Ensembles
Oct 1, 2026Alexei N. Skurikhin, Emily M. Taylor, Nathan A. DeBardelebenLLM Inference Efficiency Language Model Generation Evaluation
Los Alamos National Laboratory Los Alamos, New Mexico, USA
Emergent Unfaithfulness: How Alignment Training Causes Language Models to Silently Override Task Faithfulness
Sep 30, 2026Pardis Sadat Zahraei, Janvijay Singh, Gokhan Tur +1LLM Alignment Instruction Following
University of Illinois Urbana-Champaign
Coverage Before Control: Route-Instruction Grounding and Steering for Controllable Retrosynthesis
Sep 30, 2026Xuemin Chen, Xiaozhuang Song, Xinjian Zhao +2Instruction Following Instruction Tuning
The Chinese University of Hong Kong, Shenzhen
A2Z GameSpec-Bench: How Faithfully Can Coding Agents Generate Games from Game Design Specifications?
Sep 30, 2026Seonho Lee, Wonryeol Jeong, Alberto Cereser +4AI Coding Agents AI Agent Benchmarks
KRAFTON · KAIST · Korea National University of Arts
Counterfactual Constraint-Conditioned On-Policy Distillation for Multi-Constraint Instruction Following
Sep 23, 2026Yanzhao Zheng, Yuanqiang Yu, Tianze Xu +6Reward Shaping Instruction Following
Alibaba Group, Hangzhou, China
IBBench-Light: A Paired Evaluation of Task-Conditioned Responses to External Directives
Sep 12, 2026Kainan Zhou, Zhaoyi Li, Janet Sung +2LLM Evaluation Benchmark Design
Google LLC Mountain View, CA, USA
ContractEval: Query-Conditioned Execution Matching for Procedural Instruction Conformance
Sep 10, 2026Praphul Singh, Shanu Kumar, Akshat Agarwal +1LLM Agent Evaluation Instruction Following
Oracle Health AI · MBZUAI
LANTERN: Language Model Assessment on Noisy and Transformed Tasks for Understanding Error and Robustness Nuances
Sep 7, 2026Vamsi Krishna Kodavali, Rituraj SinghLLM Evaluation Instruction Following
Samsung
In-Place Instruction Following in Diffusion Language Models
Sep 7, 2026Zheng Nie, Zherui Li, Jiaming Zhang +3Instruction Following Diffusion Language Models
National University of Singapore · Nanyang Technological University · Peking University
ElderBench: Benchmarking Autonomous Mobile Agents for Older Adults
Sep 7, 2026Weide Zhan, Qumu Shaqu, Yuanqing Liu +6Mobile GUI Automation Computer-Use Agent Benchmarks
Fudan University Shanghai, China
Instruction Duplication as an Inference-Time Control Primitive
Sep 3, 2026Victor LavrenkoInstruction Following Inference-Time Language Model Alignment
PeaceTech VC, Israel
DuplexSpeechBench-IFEval: Evaluating Implicit Instruction Following in Full-Duplex Voice Agents
Sep 3, 2026Puneet Mathur, Dinesh ManochaAI Agent Benchmarks Instruction Following
University of Maryland College Park, USA
From Production Traffic to Post-Training: Building a Self-Hosted LLM That Covers the Corporate Request Mix
Sep 1, 2026Olga Tsymboi, Dmitrii Stoianov, Ramil Latypov +11LLM Serving Instruction Following
T-Tech
Compile, Don't Memorize: A Context Compilation Architecture (CCA) for In-Context Learning
Sep 1, 2026Jinhu Qi, Minda Hu, Wentao Zhang +4Instruction Following In-Context Learning
The Chinese University of Hong Kong · Macao Polytechnic University · Xi’an Jiaotong University +1
Dead text or binding clause? Measuring and restoring constraint influence in black-box LLM dialogues
Aug 12, 2026Haoyuan ZhuInstruction Following LLM Reliability
Department of Electronic and Electrical Engineering University of Sheffield
Large Language Models Can Follow Instructions, But Not Many at Once: Phase Transitions in Compositional Constraint Satisfaction
Aug 12, 2026Mariya I. VasilevaLLM Evaluation Instruction Following
Work done while at Meta Superintelligence Labs.
Harness-IF: Evaluating Instruction Following Across Instruction Surfaces in Coding Agents
Aug 12, 2026Zining Huang, Haoran Que, Hong Zeng +8AI Coding Agents LLM Agent Evaluation
ByteDance Seed · Tsinghua University · Peking University
EnterpriseRAG: Benchmarking LLM Instruction Adherence and Robustness under Non-Ideal Enterprise Retrieval
Aug 12, 2026Huiqi Miao, Xinbao Sun, Bo Wang +6Retrieval-Augmented Generation Instruction Following
Jiutian Research, China Mobile, Beijing, China
Why Does CLAUDE.md Keep Growing? Catastrophic Remembering in Agentic Coding
Aug 11, 2026Kushal ChakrabartiLLM Prompting Agentic Code Generation
South Park Commons
UNSPECIFIC: General Constraint Synthesis for Breaking Copy-and-Paste Shortcut in LLM Instruction Following
Aug 10, 2026Jeet Sharma, Balpreet Kaur, Jeremiah Hong +2Instruction Following Constraint Satisfaction
University of Massachusetts, Amherst, USA
LLM within MCP Matters: Measuring Inefficient Resource Utilization Driven by LLMs
Aug 9, 2026Minhan Cho, Soyoung Park, Kihyeon Jeong +3LLM Inference Efficiency Instruction Following
Remember & Company AI Lab · Sungkyunkwan University · National Assembly Research Service +2
Your Prompt Is Not the Only Prompt: How Much Do LLMs Weight Structured-Output Schema Descriptions?
Aug 8, 2026Sin-Ying LinPrompt Sensitivity Structured Prompting
Independent Researcher
Lost in Compaction: Evaluating Side-Constraint Loss under Context Compaction
Jul 31, 2026Zhiqi Wang, Yichi Zhang, Dongwon Lee +1Persistent Memory for Language Models Instruction Following
The Pennsylvania State University
Instruction Stacking Collapse: A Benchmark and the Capability-Dependent Value of Prompt Compilation
Jul 31, 2026Atul Anand, Sourav ChattarajLLM Evaluation LLM Prompting
IFHierBench: Hierarchical Instruction Following for Large Language Models
Jul 30, 2026Yuetian Mao, Chunyang ChenLLM Evaluation Language Model Generation Evaluation
Technical University of Munich
HANDBOOK.md: A Benchmark for Long-Context Agentic Instruction Following
Jul 28, 2026Liudas Panavas, Sebastian Minus, Bradley Monton +4Computer-Use Agent Benchmarks AI Agent Reliability
Surge AI*
SyRuP: Enhancing System-Prompt Following via Reward-Guided Prediction in LLM Decoding
Jul 27, 2026Seoyeon Kim, Minjae Kang, Jaehyung KimLanguage Model Decoding Language Model Steering
Yonsei University
Task Competence Is Not Instruction Following: Evaluating Instruction-Conflicting Behavior in Small Language Models
Jul 21, 2026Mahdiyeh Farajidizaji, Vatsal RainaLLM Evaluation Instruction Following
Khajeh Nasir Toosi University of Technology · Apta AI, Spark AI Research
Prompt Design at Scale: How Format, Instruction Count, and Context Length Shape Instruction Adherence and Hallucination in Large Language Models
Jul 21, 2026Netanel EliavLLM Evaluation Language Model Generation Evaluation
Machine Human Intelligence Lab (MHIL)
Compile, Then Page: Executable SOP Programs and a Capability-Gated Runtime for Procedural LLM Agents
Jul 13, 2026Chenglin Yu, Li Yin, Ying Yu +4Instruction Following Runtime Enforcement for AI Agents
Department of Industrial and Systems Engineering, The Hong Kong Polytechnic University · Department of Data and Systems Engineering, The University of Hong Kong · College of Economics and Management, Zhejiang Normal University +3
Unlocking Speech-Text Compositional Powers: Instruction-Following Speech Language Models without Instruction Tuning
Jul 2, 2026Congrui Du, Yang Zhang, Kaizhi Qian +1Language Model Pretraining Instruction Following
University of California, Santa Barbara, USA · MIT-IBM Computing Research Lab, IBM Research, USA
Seed2.0 Model Card: Towards Intelligence Frontier for Real-World Complexity
Jun 30, 2026Bytedance SeedLLM Evaluation Instruction Following
Bytedance Seed
CDR-Bench: Evaluating Faithful Execution of Compositional, Order-Sensitive Data Refinement Recipes
Jun 30, 2026Yuchen Huang, Xiang Li, Zhenqing Ling +5LLM Evaluation Instruction Following
1HKUST · 2NUS · 3Tongyi Lab , Alibaba Group
Indi-RomCoM: Code-Mixed Benchmark for Evaluating LLMs on Romanized Indic-English Instructions
Jun 29, 2026Avisha Das, Mihir Parmar, Mohana Ramnath +1Multilingual Language Model Evaluation Instruction Following
Shiv Nadar University Chennai · Google Cloud AI Research · IIT Madras
IHDec: Divergence-Steered Contrastive Decoding for Securing Multi-Turn Instruction Hierarchies
Jun 29, 2026Nicole Geumheon Liu, Haeun Jang, Yonghyun Jun +1Instruction Following Prompt Injection Defense
Chung-Ang University, Seoul, Korea
Are Humans Evolved Instruction Followers? An Underlying Inductive Bias Enables Rapid Instructed Task Learning
Jun 29, 2026Anjishnu KumarInstruction Following Inductive Bias
Amazon Alexa AI Seattle, USA
Masked Language Flow Models
Jun 26, 2026Iskander Azangulov, Kianoosh Ashouritaklimi, Leo Zhang +2Masked Diffusion Models Instruction Following
Department of Statistics, University of Oxford
FBK's Long-form SpeechLLMs for IWSLT 2026 Instruction Following
Jun 25, 2026Zhihang Xie, Marco Gaido, Sara Papi +2Audio-Language Model Evaluation Speech Processing
Fondazione Bruno Kessler · University of Trento
A Framework for Evaluating Agentic Skills at Scale
Jun 16, 2026Maksim Shaposhnikov, Nicolas Fortuin, Simon Stipcich +3LLM Agent Evaluation Instruction Following
Tessl London, United Kingdom
Beyond English: Uncovering the Multilingual Gap in Vision-Language-Action Models
Jun 14, 2026Hanyang Chen, Hongliang Li, Jiarui Cao +6Cross-Lingual Representation Alignment Instruction Following
Beijing Jiaotong University
Soft-Prompt Tuning for Fair and Efficient LLM Benchmark Evaluation
Jun 10, 2026Selen Erkan, Bastian Boll, Kristian Kersting +2LLM Evaluation Instruction Following
Aleph Alpha Research Lab1141 · TU Darmstadt Hessian.AI Lab1141
When Built-in Thinking Helps and Hurts: Constraint-Level Error Shifts in Instruction Following
Jun 8, 2026Sai Adith Senthil KumarLLM Evaluation Instruction Following
George Mason University
ComplexConstraints and Beyond: Expert Rubrics for RLVR
Jun 8, 2026Sushant Mehta, Liudas Panavas, Suhaas Garre +1LLM-as-a-Judge Rubric-Based RL
Surge AI
Where Instruction Hierarchy Breaks: Diagnosing and Repairing Failures in Reasoning Language Models
Jun 5, 2026Sanjay Kariyappa, G. Edward SuhLLM Auditing Instruction Following
NVIDIA
RECAP: Regression Evaluation for Continual Adaptation of Prompts
Jun 4, 2026Harsh Deshpande, Kushal Chawla, Sangwoo Cho +2Continual Learning for LLMs LLM Evaluation
Capital One
MDP-GRPO: Stabilized Group Relative Policy Optimization for Multi-Constraint Instruction Following
Jun 4, 2026Mohammad Mahdi Salmani-Zarchi, Zahra Rahimi, Heshaam Faili +1Group Relative Policy Optimization Instruction Following
Department of Electrical and Computer Engineering, College of Engineering, University of Tehran, Tehran, Iran · Department of Statistics, Mathematics and Computer Science, Allameh Tabataba’i University, Tehran, Iran
Multilingual Long-Form Speech Instruction Following: KIT's Submission to IWSLT 2026
Jun 3, 2026Enes Yavuz Ugan, Maike Züfle, Yuka Ko +5MBR Decoding Instruction Following
1Karlsruhe Institute of Technology · 2Carnegie Mellon University
Bridging Auxiliary Constraints to Resolve Instruction Following in Large Reasoning Models
Jun 2, 2026Zhengyi Zhao, Shubo Zhang, Huimin Wang +7LLM Prompting Instruction Following
The Chinese University of Hong Kong · Tencent Jarvis Lab · Westlake University +2
AnyAudio-Judge: A Dynamic Rubric-Based Benchmark and Evaluator for Audio Instruction Following
Jun 2, 2026Haitao Li, Tian Tan, Yuguang Yang +2Audio-Language Model Evaluation LLM-as-a-Judge
Zhejiang University · Shanghai Innovation Institute · Shanghai Jiao Tong University +1
Label-Free Reinforcement Learning via Cross-Model Entropy
May 27, 2026Matt Gorbett, Hossein ShiraziReinforcement Learning Instruction Following
Independent Researcher · San Diego State University
Soft-SVeRL: Self-Verified Reinforcement Learning with Soft Rewards
May 27, 2026Saurabh Dash, Pierre Clavier, John Dang +4Reinforcement Learning Instruction Following
Cohere Labs · Cohere · Adaption Labs +1
The Missing Piece in Pre-trained Model Evaluation: Reward-Guided Decoding Unlocks Task-Oriented Behavior Without Parameter Updates
May 27, 2026Shaobo Wang, Guo Chen, Ziyue Wang +5LLM Evaluation Language Model Decoding
Shanghai Jiao Tong University · Qwen Team, Alibaba Group · The Chinese University of Hong Kong, Shenzhen
Disentangling Language Roles in Multilingual LLM Task Execution
May 26, 2026Qishi Zhan, Minxuan Hu, Seoyeon Jang +7Multilingual Language Models Multilingual Language Model Evaluation
Marquette · Cornell · UC San Diego +7
MAIGO: Mitigating Lost-in-Conversation with History-Cleaned On-Policy Self-Distillation
May 26, 2026Haoyu Zheng, Yun Zhu, Shu Yuan +5Instruction Following Data Contamination in Language Models
1Zhejiang University · 2Shanghai AI Laboratory · 3Huazhong University of Science and Technology +2
ContextGuard: Structured Self-Auditing for Context Learning in Language Models
May 26, 2026Hongbo Jin, Chi Wang, Haoran Tang +5LLM Auditing Instruction Following
Peking University · SCUT · Tsinghua University
Do as I Say, Not as I Do: Instruction-Induction Conflict in LLMs
May 19, 2026Carolina Camassa, Derek ShillerLLM Evaluation Language Model Self-Assessment
Future Impact Group · Rethink Priorities
Dimension-Level Intent Fidelity Evaluation for Large Language Models: Evidence from Structured Prompt Ablation
May 14, 2026GAng PengMultilingual Language Model Evaluation LLM Evaluation
Huizhou Lateni AI Technology Co., Ltd., Huizhou, China · Huizhou University, Huizhou, China
When Attention Closes: How LLMs Lose the Thread in Multi-Turn Interaction
May 13, 2026Vardhan Dongre, Joseph Hsieh, Viet Dac Lai +3Self-Attention LLM Interpretability
University of Illinois Urbana-Champaign · Adobe Research

Oct 5, 2026, cs.AIJ/K move · Enter open · S save

You Changed Your Mind, The Model Didn't: Demystifying Intent in Multi-Turn Dialogue

Junle Chen, Wei Chen, Zhengjun Huang, Zhoujin Tian+4

HKUST · Tencent

Intent Classification Instruction Following Multi-Turn Dialogue Evaluation Language Model Robustness

Open paper AskPDF arXiv

When a large language model handles a multi-turn task and a user proposes a change but ultimately rejects it, the model should continue as if nothing changed. We find a surprising failure: merely mentioning a rejected change can derail task execution, even when the user's final intent remains unchanged. To systematically study language model behavior under evolving user intent, we introduce Intent-Eval, a controlled benchmark spanning tool actions, code, databases, and mathematics. Across diverse tasks, models are vulnerable to both rejected proposals and superseded requirements, consistent with mentioned-as-in-effect confusion: conversational content is treated as active requirements even after it has been rejected or replaced. Accuracy degradation can deepen or persist as interaction continues, highlighting the need to distinguish what has been mentioned from what remains in effect. Building on this insight, we propose Intent-OPSD, a decision-conditioned on-policy self-distillation framework with Teacher and Student initialized from the same model. The frozen Teacher provides active-intent supervision from the complete task matching the user's decision, training the Student on the full dialogue to follow active requirements reflecting user intent.

Momentum

Related topics

Latest papers 83