Instruction Hierarchy for Language Models

Follow this topic Ask about this topic

Latest papers

12 papers1 paper in the last four weeks, with none the four weeks before. 0.0% of all new papers.

Improving Instruction Hierarchy by Learning from Goal-Goal Conflicts
Oct 26, 2026Sizhe Chen, Sanjay Kariyappa, Liwei Jiang +4Instruction Hierarchy for Language Models LLM Safety Alignment
NVIDIA · UC Berkeley
Position, Role or Label? A Controlled Study of Instruction Conflicts in LLM Agents
Oct 23, 2026Anish Laddha, Nitesh Pradhan, Gaurav SrivastavaInstruction Hierarchy for Language Models LLM Agents
Department of Computer Science and Engineering, LNMIIT, India · Department of Computer Science, Virginia Tech, USA
How Language Models Choose Sides: Internal Representations of Instruction Hierarchy
Aug 16, 2026Enrique Balp-Straffon, Chih-Hao Hsu, Rushiraj Gadhvi +3Instruction Hierarchy for Language Models Causal Interventions in Language Models
Amazon · National Taiwan University · Plaksha University, Mohali, India +3
IFHierBench: Hierarchical Instruction Following for Large Language Models
Jul 30, 2026Yuetian Mao, Chunyang ChenLLM Evaluation Language Model Generation Evaluation
Technical University of Munich
Steering Instruction Hierarchies at Inference Time
Jul 28, 2026Siqi Zeng, Sewoong Lee, Han Zhao +1Language Model Steering LLM Safety Alignment
Siebel School of Computing and Data Science University of Illinois, Urbana-Champaign Urbana, IL 61801, USA
Language Shapes Instruction Hierarchy Compliance in Multilingual LLMs
Jul 26, 2026Jiwon Moon, Yerin Hwang, Kyomin JungMultilingual Language Model Evaluation Instruction Hierarchy for Language Models
IPAI, Seoul National University · Dept. of ECE, Seoul National University
IHDec: Divergence-Steered Contrastive Decoding for Securing Multi-Turn Instruction Hierarchies
Jun 29, 2026Nicole Geumheon Liu, Haeun Jang, Yonghyun Jun +1Instruction Following Prompt Injection Defense
Chung-Ang University, Seoul, Korea
Training LLMs to Enforce Multi-Level Instruction Hierarchies via Gravity-Weighted Direct Preference Optimization
Jun 9, 2026Lena S. Bolliger, Lena A. JägerDirect Preference Optimization LLM Safety Alignment
Department of Computational Linguistics, University of Zurich, Switzerland
Where Instruction Hierarchy Breaks: Diagnosing and Repairing Failures in Reasoning Language Models
Jun 5, 2026Sanjay Kariyappa, G. Edward SuhLLM Auditing Instruction Following
NVIDIA
M-CARE: Standardized Clinical Case Reporting for AI Model Behavioral Disorders, with a 20-Case Atlas and Experimental Validation
Mar 27, 2026Jihoon JeongInstruction Hierarchy for Language Models AI Agent Evaluation
Department of Electrical Engineering and Computer Science, Daegu Gyeongbuk Institute of Science and Technology (DGIST) · ModuLabs
IH-Challenge: A Training Dataset to Improve Instruction Hierarchy on Frontier LLMs
Mar 11, 2026Chuan Guo, Juan Felipe Ceron Uribe, Sicheng Zhu +10Instruction Hierarchy for Language Models Adversarial Robustness of Language Models
Michael Pokorny · OpenAI
Reasoning Up the Instruction Ladder for Controllable Language Models
Oct 30, 2025Zishuo Zheng, Vidhisha Balachandran, Chan Young Park +2LLM Safety Alignment Instruction Hierarchy for Language Models
Department of Computer Science and Engineering, The Ohio State University · Microsoft Research · Allen Institute for AI

Oct 26, 2026J/K move · Enter open · S save

Improving Instruction Hierarchy by Learning from Goal-Goal Conflicts

Sizhe Chen, Sanjay Kariyappa, Liwei Jiang, Chong Xiang+3

NVIDIA · UC Berkeley

Instruction Hierarchy for Language Models LLM Safety Alignment Instruction Tuning

Open paper AskPDF arXiv

The recent incident in which OpenAI agents autonomously compromised Hugging Face's infrastructure illustrates the importance of instruction hierarchy (IH): an LLM should follow lower-priority instructions, such as user requests, only when they do not conflict with higher-priority instructions, such as system policies. IH can be effectively improved by fine-tuning LLMs on simulated examples of instruction conflicts, whose diversity is key to teaching LLMs to properly handle conflicts at test time. However, our decomposition of a task into goals and constraints reveals a conflict type -- goal-goal conflicts -- that is absent from existing IH training sets from OpenAI and NVIDIA. We show the importance of this conflict type: fine-tuning Qwen3.5-9B and Gemma4-12B-it on examples of this conflict type significantly improves IH on a wide range of real-world tasks unseen in training; adding these examples to existing IH training sets for fine-tuning also gives better IH adherence. The IH gains come with trivial mean utility changes on no-conflict tasks across eight benchmarks. Our work advocates including this new conflict type in IH training, even if these examples do not reflect typical LLM use.