cs.CLApr 10, 2026

Many-Tier Instruction Hierarchy in LLM Agents

Authors: Jingyu Zhang, Tianjian Li, William Jurayj, Hongyuan Zhan, Benjamin Van Durme, Daniel Khashabi

Organizations: Johns Hopkins University

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Large language model agents receive instructions from many sources-system messages, user prompts, tool outputs, other agents, and more-each carrying different levels of trust and authority. When these instructions conflict, agents must reliably follow the highest-privilege instruction to remain safe and effective. The dominant paradigm, instruction hierarchy (IH), assumes a fixed, small set of privilege levels (typically fewer than five) defined by rigid role labels (e.g., system > user). This is inadequate for real-world agentic settings, where conflicts can arise across far more sources and contexts. In this work, we propose Many-Tier Instruction Hierarchy (ManyIH), a paradigm for resolving instruction conflicts among instructions with arbitrarily many privilege levels. We introduce ManyIH-Bench, the first benchmark for ManyIH. ManyIH-Bench requires models to navigate up to 12 levels of conflicting instructions with varying privileges, comprising 853 agentic tasks (427 coding and 426 instruction-following). ManyIH-Bench composes constraints developed by LLMs and verified by humans to create realistic and difficult test cases spanning 46 real-world agents. Our experiments show that even the current frontier models perform poorly (~40% accuracy) when instruction conflict scales. This work underscores the urgent need for methods that explicitly target fine-grained, scalable instruction conflict resolution in agentic settings.

Figures & tables

Figure 1: Overview of Many-Tier Instruction Hierarchy compared with existing IH. ManyIH enables arbitrary levels of privilege to be defined at inference based on instruction sources through a dedicated Privilege Prompt Interface (§ 3 ), tackling the fixed-tier bottleneck (§ 2 ) of existing instruction hierarchy.
Figure 2: Left : overall accuracy on ManyIH-Bench . Right : accuracy by subset. Both frontier and open-source models struggle with ManyIH. Error bars show bootstrap 95% CIs.
Figure 3: Effect of scaling IH tiers and test-time reasoning. Left : Accuracy across IH tiers on the coding subset. Mid : Accuracy on ManyIH-Bench coding subset vs. reasoning effort. Right : Distribution of CoT length across models. Model performance consistently degrades as the number of IH tiers increases, and increasing reasoning effort alone is unlikely to resolve ManyIH.
 Ordinal →\to ScalarScalar Perturbation
ModelΔ\Delta AccΔ\Delta Acc testΔ\Delta Acc styleΔ\Delta AccΔ\Delta Acc testΔ\Delta Acc styleOverall flipTest flipStyle flip
GPT-5.4−- 8.4−- 0.7−- 8.7++ 4.7++ 1.2++ 4.716.43.016.9
Opus 4.6−- 8.0−- 0.9−- 8.7++ 3.3++ 0.5++ 3.08.00.98.7
Sonnet 4.6++ 2.3++ 0.0++ 3.7−- 1.4−- 0.2−- 1.212.61.613.3
Qwen3.5-122B++ 5.9++ 19.9−- 2.1++ 4.0−- 0.7++ 4.517.18.219.0
Qwen3.5-9B−- 0.9++ 3.7−- 2.3++ 1.2−- 0.5++ 0.08.712.210.8
Table 1: Effect of interface variants on coding subset (%). Left: switching from ordinal to scalar PPI. Right: perturbing zz -values in the scalar PPI by δ∼Uniform(−3,+3)\delta\sim\mathrm{Uniform}(-3,+3) while preserving the strict privilege ordering. Δ\Delta : perturbed minus original. Flip rate: fraction of samples whose binary pass/fail outcome changed. Models exhibit non-trivial sensitivity to privilege interface change and privilege value perturbations.
ModelOrdinalScalarΔ\Delta
Gemini 3.1 Pro42.746.8++ 4.1
GPT-5.439.434.1−- 5.3
Qwen3.5-397B34.133.3−- 0.8
Opus 4.633.228.8−- 4.3
Kimi K2.532.732.7++ 0.0
Grok 4.2031.730.4−- 1.3
Table 2: Overall accuracy on ManyIH-Bench (%) under the ordinal and scalar PPI, sorted by ordinal accuracy. Δ\Delta : scalar minus ordinal. Per-model sensitivity is non-trivial, but the induced ranking is nearly identical across the two formats.
Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

ModelAccAcc testAcc style
GPT-5.460.989.767.9
Gemini 3.1 Pro59.091.365.1
Grok 4.2054.186.263.0
GPT-5.553.991.658.8
GPT-5.6-sol52.591.357.9
Opus 4.651.392.556.7
Appendix
Table 3: Performance breakdown on the coding subset. Overall accuracy is bottlenecked by style compliance.
StatisticCodingIF
General
Samples427426
SourceMBPP ( Austin et al., 2021 )AgentIF ( Qi et al., 2025 )
Max privilege levels per sample127
Instruction & Conflict Structure
Instructions per sample12 (fixed)19.4 (avg)
Appendix
Table 4: Key statistics of the ManyIH-Bench benchmark. The coding subset has fixed difficulty parameters by design, while the instruction-following subset has naturally varying complexity drawn from real agentic scenarios. Input length is measured in whitespace-delimited words across all messages in the model input.
Statistic6 tiers8 tiers12 tiers
Samples427427427
Instructions per sample6812
Privilege levels per sample6812
Style groups per sample444
Winning styles per sample666
Pairwise conflicts per sample02–4 (avg 3.5)7–11 (avg 9.8)
Appendix
Table 5: Statistics of the three coding subset configurations used in the IH tier scaling experiment (§ 6.2 ). Style groups and winning styles are held constant across variants; only the number of instructions, privilege levels, and conflicts increase. Input length is measured in whitespace-delimited words.
CategoryStyle IDInstruction
indentindent_2Indent using exactly 2 spaces
indent_4Indent using exactly 4 spaces
indent_tabIndent using tabs
indent_spacesIndent using spaces (not tabs)
quotesquotes_singleUse single quotes for strings
quotes_doubleUse double quotes for strings
Appendix
Table 6: All 12 style groups and 41 style constraints in the coding subset. Constraints within the same group may conflict (e.g., indent_2 vs. indent_4 ); constraints across different groups are always compatible.
More in this topic

LLM Agents

Large language model agents use language models to plan, use tools, and act across multi-step tasks and environments.

See all 705 papers

Explore similar work

CardsList
Oct 26, 2026

Improving Instruction Hierarchy by Learning from Goal-Goal Conflicts

The recent incident in which OpenAI agents autonomously compromised Hugging Face's infrastructure illustrates the importance of instruction hierarchy (IH): an LLM should follow lower-priority instructions, such as user requests, only when they do not conflict with higher-priority instructions, such as system policies. IH can be effectively improved by fine-tuning LLMs on simulated examples of instruction conflicts, whose diversity is key to teaching LLMs to properly handle conflicts at test time. However, our decomposition of a task into goals and constraints reveals a conflict type -- goal-goal conflicts -- that is absent from existing IH training sets from OpenAI and NVIDIA. We show the importance of this conflict type: fine-tuning Qwen3.5-9B and Gemma4-12B-it on examples of this conflict type significantly improves IH on a wide range of real-world tasks unseen in training; adding these examples to existing IH training sets for fine-tuning also gives better IH adherence. The IH gains come with trivial mean utility changes on no-conflict tasks across eight benchmarks. Our work advocates including this new conflict type in IH training, even if these examples do not reflect typical LLM use.

Sizhe Chen, Sanjay Kariyappa, Liwei Jiang +4

NVIDIA · UC Berkeley

LLM Safety AlignmentInstruction Tuning
AskPDF
Apr 10, 2026

Hierarchical Alignment: Enforcing Hierarchical Instruction-Following in LLMs through Logical Consistency

Large language models increasingly operate under multiple instructions from heterogeneous sources with different authority levels, including system policies, user requests, tool outputs, and retrieved context. While prior work on instruction hierarchy highlights the importance of respecting instruction priorities, it mainly focuses on adversarial attacks and overlooks the benign but common instruction conflicts that arise in real-world applications. In such settings, models must not only avoid security violations but also preserve task utility and behavioral consistency when instructions partially or implicitly conflict. We propose Neuro-Symbolic Hierarchical Alignment (NSHA) for hierarchical instruction-following by explicitly modeling and enforcing instruction priorities. At inference time, we introduce solver-guided reasoning that formulates instruction resolution as a constraint satisfaction problem, enabling the model to derive a maximally consistent set of applicable instructions under hierarchical constraints. At training time, NSHA distills solver-based decisions into model parameters using automatically constructed supervision. We evaluate our approach on rule following, task execution, tool use, and safety, covering both single-turn and multi-turn interactions, and show that NSHA significantly improves performance under such conflicts while maintaining competitive utility in reference settings.

Shu Yang, Zihao Zhou, Di Wang +1

King Abdullah University of Science and Technology (KAUST) · University of Liverpool · University of Edinburgh

LLM AlignmentInference-Time Language Model AlignmentNeuro-Symbolic Reasoning
AskPDF
Oct 23, 2026

Position, Role or Label? A Controlled Study of Instruction Conflicts in LLM Agents

LLM agents take instructions from the operator, repository configuration, user, tool results and saved memory, which often conflict. Instruction-hierarchy benchmarks ask whether the higher-ranked source wins, but a source sets position, role and label at once. First, we build WhoWins, 1,080 code-scored scenarios in five task types, each with up to five mutually exclusive instructions, and seven controlled experiments that vary only how instructions are delivered. Second, we study 14 open-weight models (0.6B to 122B, thinking off and on) over 1,887,356 scored replies, checking main findings on held-out scenarios. Inside one message the first of two conflicting lines wins 72.42 to 94.66%, and across messages it keeps 22.95 to 62.65%. A policy sent first in the system (or developer) message loses most conflicts with later lines in every Qwen model but wins most on gpt-oss when it acts. In tool and file-write tasks it wins less often than the same text sent first as a labelled user message, in every model-mode. Third, we test whether pairs forecast larger conflicts; forecasts improve in every model-mode once they know where the task line is. Finally, by our keyword patterns at most 2.92% of decided pair replies flag or hedge a conflict.

Anish Laddha, Nitesh Pradhan, Gaurav Srivastava

Department of Computer Science and Engineering, LNMIIT, India · Department of Computer Science, Virginia Tech, USA

LLM AgentsLLM Agent EvaluationAI Agent Benchmarks
AskPDF