Instruction Following

Follow this topic Ask about this topic

Momentum

11 papers in the last four weeks, down 8% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 85

All topics

Dimension-Level Intent Fidelity Evaluation for Large Language Models: Evidence from Structured Prompt Ablation
May 14, 2026GAng PengMultilingual Language Model Evaluation LLM Evaluation
Huizhou Lateni AI Technology Co., Ltd., Huizhou, China · Huizhou University, Huizhou, China
When Attention Closes: How LLMs Lose the Thread in Multi-Turn Interaction
May 13, 2026Vardhan Dongre, Joseph Hsieh, Viet Dac Lai +3Self-Attention LLM Interpretability
University of Illinois Urbana-Champaign · Adobe Research
Premover: Fast Vision-Language-Action Control via Early Execution During Instruction Delivery
May 12, 2026Joonha Park, Jiseung Jeong, Taesik GongVisuomotor Policy Learning AI Control
UNIST, Ulsan, Republic of Korea · The Catholic University of Korea, Bucheon, Republic of Korea
Instructions Shape Production of Language, not Processing
May 11, 2026Andreas Waldis, Leshem Choshen, Yufang Hou +1LLM Interpretability Instruction Following
Department of Linguistics, University of Tübingen · IBM Research, MIT, and MIT-IBM Watson AI Lab · MIT, and MIT-IBM Watson AI Lab +2
Instruction Adherence in Coding Agent Configuration Files: A Factorial Study of Four File-Structure Variables
May 11, 2026Damon McMillanCoding Agents LLM Agent Evaluation
HxAI Australia
CALYREX: Cross-Attention LaYeR EXtended Transformers for System Prompt Anchoring
May 10, 2026Li LixingSelf-Attention Cross-Attention
Cornell University Ithaca, NY 14853
SEIF: Self-Evolving Reinforcement Learning for Instruction Following
May 8, 2026Qingyu Ren, Qianyu He, Jiajie Zhu +7RL for Language Models Reinforcement Learning
Shanghai Key Laboratory of Data Science, College of Computer Science and Artificial Intelligence, Fudan University · School of Data Science, Fudan University · Ant Group
Think-with-Rubrics: From External Evaluator to Internal Reasoning Guidance
May 8, 2026Jiachen Yu, Zhihao Xu, Junjie Wang +1Instruction Following Rubric-Based Evaluation
1Tsinghua University · 2Renmin University of China
SEQUOR: A Multi-Turn Benchmark for Realistic Constraint Following
May 7, 2026Beatriz Canaverde, Duarte M. Alves, José Pombal +2Instruction Following Generative AI Evaluation
Instituto de Telecomunicac¸˜oes · Instituto Superior T´ecnico, Universidade de Lisboa · Sword Health +2
Decomposing the Basic Abilities of Large Language Models: Mitigating Cross-Task Interference in Multi-Task Instruct-Tuning
May 7, 2026Bing Wang, Ximing Li, Changchun Li +3LLM Fine-Tuning Gradient Interference
College of Computer Science and Technology, Jilin University · Key Laboratory of Symbolic Computation and Knowledge Engineering, Ministry of Education, Jilin University · RIKEN Center for Advanced Intelligence Project +1
MCJudgeBench: A Benchmark for Constraint-Level Judge Evaluation in Multi-Constraint Instruction Following
May 5, 2026Jaeyun Lee, Junyoung Koh, Zeynel Tok +2LLM Evaluation LLM-as-a-Judge
University of Oxford · Yonsei University
When LLMs Stop Following Steps: A Diagnostic Study of Procedural Execution in Language Models
May 1, 2026Sailesh Panda, Pritam Kadasi, Abhishek Upperwal +1LLM Evaluation Instruction Following
Indian Institute of Technology Gandhinagar · Soket AI
Models Recall What They Violate: Constraint Adherence in Multi-Turn LLM Ideation
Apr 30, 2026Garvin KruthofLLM Evaluation LLM Alignment
Technical University of Munich
From Coarse to Fine: Benchmarking and Reward Modeling for Writing-Centric Generation Tasks
Apr 30, 2026Qingyu Ren, Tianjun Pan, Xingzhou Chen +1Reward Modeling RL for Language Models
Shanghai Key Laboratory of Data Science, College of Computer Science and Artificial Intelligence, Fudan University · 2Shanghai Artificial Intelligence Laboratory
Compliance versus Sensibility: On the Reasoning Controllability in Large Language Models
Apr 29, 2026Xingwei Tan, Marco Valentino, Mahmud Elahi Akhter +3LLM Interpretability Instruction Following
School of Computer Science, University of Sheffield · School of EECS, Queen Mary University of London · The Alan Turing Institute
Beyond the Attention Stability Boundary: Agentic Self-Synthesizing Reasoning Protocols
Apr 27, 2026Dahlia Shehata, Ming LiInstruction Following Attention Mechanisms
University of Waterloo · Canada
Omission Constraints Decay While Commission Constraints Persist in Long-Context LLM Agents
Apr 22, 2026Yeran GamageAI Agent Monitoring LLM Security
University of South Florida · Independent AI Security Researcher
Measuring Distribution Shift in User Prompts and Its Effects on LLM Performance
Apr 19, 2026Parker Seegmiller, Sarah Masud PreumLLM Evaluation Instruction Following
Department of Computer Science Dartmouth College Hanover, NH, USA
TinyJudge: Unverifiable Constraint Alignment via Lightweight Specialist Ensembles
Apr 19, 2026Yirong Zeng, Yufei Liu, Xiao Ding +9Reward Modeling LLM Alignment
Harbin Institute of Technology SCIR Lab · Peking University · Huawei Technologies Co., Ltd
Mini-BEHAVIOR-Gran: Revealing U-Shaped Effects of Instruction Granularity on Language-Guided Embodied Agents
Apr 18, 2026Sukai Huang, Chenyuan Zhang, Fucai Ke +4AI Agent Evaluation AI Agent Benchmarks
Faculty of Information Technology, Monash University
Reasoning Up the Instruction Ladder for Controllable Language Models
Oct 30, 2025Zishuo Zheng, Vidhisha Balachandran, Chan Young Park +2LLM Safety Alignment Instruction Following
Department of Computer Science and Engineering, The Ohio State University · Microsoft Research · Allen Institute for AI
COCORELI: Enforcing Execution Preconditions for Reliable Collaborative Instruction Following
Aug 29, 2025Swarnadeep Bhar, Omar Naim, Eleni Metheniti +4Instruction Following LLM Agent Planning
IRIT · ANTI · ENS Paris-Saclay, France +1
RECAST: Expanding the Boundaries of LLMs' Complex Instruction Following with Multi-Constraint Data
May 25, 2025Zhengkang Guo, Wenhao Liu, Mingchen Xie +13LLM Fine-Tuning Synthetic Data Generation
College of Computer Science and Artificial Intelligence, Fudan University · Xiaohongshu Inc.
Measuring Pragmatic Influence in Large Language Model Instructions
Date pendingYilin Geng, Omri Abend, Eduard Hovy +1LLM Evaluation LLM Prompting
University of Melbourne · Hebrew University of Jerusalem
How LLMs Follow Instructions: Skillful Coordination, Not a Universal Mechanism
Date pendingElisabetta Rocchetti, Alfio FerraraLLM Interpretability Instruction Following
Department of Computer Science, Università degli Studi di Milano

May 14, 2026, cs.CLJ/K move · Enter open · S save

Dimension-Level Intent Fidelity Evaluation for Large Language Models: Evidence from Structured Prompt Ablation

GAng Peng

Huizhou Lateni AI Technology Co., Ltd., Huizhou, China · Huizhou University, Huizhou, China

Multilingual Language Model Evaluation LLM Evaluation Language Model Generation Evaluation Instruction Following

Open paper AskPDF arXiv

Holistic evaluation scores capture overall output quality but do not distinguish whether a model reproduced the structural form of a user's request from whether it preserved the user's specific intent. We propose a dimension-level intent fidelity evaluation framework, applied here through a structured prompt ablation study across 2,880 outputs spanning three languages, three task domains, and six LLMs, that separately measures structural recovery and intent fidelity for each semantic dimension. This framework reveals a systematic structural-fidelity split: among Chinese-language outputs with complete paired scores, 25.7% received perfect holistic alignment scores (GA=5) while exhibiting measurable dimensional intent deficits; among English-language outputs, this proportion rose to 58.6%. Human evaluation confirmed that these split-zone outputs represent genuine quality deficits and that dimensional fidelity scores track human judgements more reliably than holistic scores do. A public-private decomposition of 2,520 ablation cells characterises when models successfully compensate for missing intent and when they fail, while proxy annotation distinguishes prior inferability from default recoverability. A weight-perturbation experiment shows that moderate misalignment is typically absorbed, whereas severe dimensional inversion is consistently harmful. These findings demonstrate that dimension-level intent fidelity evaluation is a necessary complement to holistic assessment when evaluating LLM outputs for user-specific tasks.

Momentum

Related topics

Latest papers 85