Following complex instructions with multiple explicit constraints remains a fundamental challenge for large language models (LLMs). Existing alignment methods, such as DPO, optimize holistic reward signals that often underemphasize strict satisfaction of individual constraints, particularly under out-of-distribution or multi-constraint settings. In this paper, we propose STAIF, a stage-wise optimization framework that decouples the alignment of subjective (soft) constraints from the optimization of objectively verifiable (hard) constraints. Stage 1 applies preference optimization with multiple negative samples to sharpen sensitivity to soft constraints, while Stage 2 applies Reinforcement Learning with Verifiable Rewards (RLVR) to enforce strict compliance with hard constraints. To support this method, we construct STAINSTRUCT, a high-quality bilingual (English, Chinese) dataset of approximately 31,000 complex multi-constraint instructions. Extensive analyses validate the design of STAIF and show state-of-the-art performance on representative benchmarks against strong baselines, as well as genuine generalization.
Figure 2: Overview of the data construction pipeline for StaInstruct .
Figure 3: Mapping between task categories and constraint categories based on real-world user data.
Figure 4: Distribution of the number of constraints per instruction in StaInstruct .
Figure 5: Constraint type distribution in StaInstruct .
Model
IFEval
IFBench
CFBench
P-S
P-L
I-S
I-L
P-S
P-L
I-S
I-L
CSR
ISR
PSR
GPT-4.1
87.8
91.2
91.1
93.2
38.7
43.8
42.1
47.5
83.0
61.0
70.0
DeepSeek-v3.2
86.4
90.5
90.8
93.5
39.7
43.5
42.3
46.8
87.0
65.0
72.0
Gemini 2.5
89.8
91.1
92.9
94.0
39.7
50.3
43.0
52.5
83.0
60.0
70.0
Doubao 1.8
83.4
87.0
88.5
90.0
29.3
36.4
32.8
39.7
83.0
60.0
68.0
QwQ-32B
83.5
87.7
88.8
91.8
36.3
40.4
39.7
44.2
87.0
61.0
70.0
Table 1: Main results on IFEval, IFBench, and CFBench. For IFEval and IFBench we report accuracy (%) at the prompt (P) and instruction (I) levels under strict (S) and loose (L) matching: P-S , P-L , I-S , I-L . For CFBench, we report CSR , ISR , and PSR (%).
Model
Method
IFEval
IFBench
CFBench
P-S
P-L
I-S
I-L
P-S
P-L
I-S
I-L
CSR
ISR
PSR
Qwen2.5-7B
StaIF
87.0
88.7
91.0
92.3
44.2
48.3
46.0
49.9
82.0
51.0
61.0
w/o stage1
80.2
82.8
86.3
88.1
37.8
41.2
39.7
43.9
80.0
45.0
58.0
w/o stage2
79.4
82.0
85.5
87.6
38.4
42.5
40.3
45.4
81.0
46.0
58.0
Tulu3-8B
StaIF
90.4
91.1
92.9
93.5
50.3
52.0
53.1
55.2
73.0
35.0
45.0
w/o stage1
85.1
87.0
89.3
90.2
42.2
45.6
43.3
47.8
70.0
33.0
41.0
Table 2: Ablation study on the two training stages of StaIF , evaluated on Qwen2.5-7B and Tulu3-8B.
Method
IFEval
IFBench
P-S
P-L
I-S
I-L
P-S
P-L
I-S
I-L
DPO (joint)
79.1
82.0
84.6
86.9
36.9
39.5
39.7
43.7
RLVR (joint)
81.6
83.3
86.9
89.4
41.3
43.2
44.1
45.1
DPO+RLVR (joint)
84.5
86.8
88.4
90.7
41.1
45.1
44.7
48.3
StaIF
87.0
88.7
91.0
92.3
44.2
48.3
46.0
49.9
Table 3: Impact of decoupling soft and hard constraints, evaluated on Qwen2.5-7B.
Model
Method
Negative
IFEval
IFBench
CFBench
Sample
P-S
P-L
I-S
I-L
P-S
P-L
I-S
I-L
CSR
ISR
PSR
Qwen2.5-7B
Stage 1 only
Single
76.3
80.4
83.5
86.5
34.0
38.1
36.1
40.6
80.0
43.0
56.0
Multiple
79.4
82.0
85.5
87.6
38.4
42.5
40.3
45.4
81.0
46.0
58.0
StaIF
Single
85.0
86.3
89.7
90.6
42.2
46.5
44.9
47.5
80.0
46.0
56.0
Multiple
87.0
88.7
91.0
92.3
44.2
48.3
46.0
49.9
82.0
51.0
61.0
Tulu3-8B
Stage 1 only
Single
76.5
79.4
83.5
86.1
31.6
34.6
33.7
37.0
69.0
30.0
40.0
Table 4: Comparison of single vs. multiple negative samples in StaIF , evaluated on Qwen2.5-7B and Tulu3-8B.
Model
MT-Bench
GSM8K
MMLU-Pro
BBH
Qwen2.5-7B-Instruct
7.4
91.6
55.9
72.3
+ StaIF
7.5
91.4
56.0
74.4
Tulu3-8B-SFT
6.3
78.7
35.0
67.8
+ StaIF
7.1
81.3
35.4
71.2
Table 5: Semantic quality evaluation before and after applying StaIF . MT-Bench is reported on its standard 10-point scale; GSM8K, MMLU-Pro, and BBH are reported as accuracy percentages.
Figure 6: Performance by constraint type on IFEval and IFBench (prompt-level, loose accuracy) for Tulu3, Tulu3+Stage 1, and Tulu3+ StaIF .
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Category
Sub-category
Description
Semantic Understanding
Semantic Analysis
Analyze the logical structure, semantic relationships, and core meaning of text.
Sentiment Classification
Identify the emotional tendency or intensity of text.
Element Extraction
Extract key information units (e.g., people, events, relationships) from unstructured text.
Table 8: Comparison with recent instruction-following alignment methods. Dashes indicate metrics not reported by the corresponding method. * are cited from the corresponding reference for each method.
Precise instruction-following is a fundamental ability of large language models (LLMs), requiring their outputs to strictly satisfy objective constraints in input instructions. In complex application scenarios, these constraints often possess diverse scopes that govern specific response segments rather than the entire output. However, existing optimization methods often neglect constraint scope during data construction and rely on binary per-constraint rewards, yielding limited data diversity and sparse supervision for complex constraints. To this end, we propose ScopeIF, a novel training framework for scope-aware precise instruction-following. We first introduce a unified schema that factorizes objective constraints into three decoupled dimensions: Scope, Target, and Range. Grounded in this schema, we construct ScopeInstruct, a large-scale instruction dataset with diverse scope-aware constraints, and combine tool-grounded verification with graded reward modeling to quantify the violation degree of each constraint, providing dense supervision for policy optimization. Extensive experiments demonstrate that ScopeIF consistently outperforms existing methods, particularly on complex scope-aware constraints, while preserving general capabilities. Notably, it enables optimized Qwen3-4B and 8B models to rival or surpass strong frontier models such as Gemini-2.5-Pro and DeepSeek-V3.2, establishing an effective paradigm for advancing scope-aware instruction-following. Our code and data are available at https://github.com/thu-coai/ScopeIF.
Bosi Wen, Yilin Niu, Xiaoying Ning +3
The Conversational Artificial Intelligence (CoAI) Group, Tsinghua University · Zhipu AI
Common instruction following (IF) benchmarks, such as IFEval, evaluate the ability of language models (LMs) to follow simple constraints that can easily be verified programmatically. These same constraints, in turn, are used as reward signals for reinforcement learning. This design may lead to reward hacking of constraints, pushing models away from the main task. Recent attempts to address this resulted in benchmarks with richer constraints, but required expert labor, remained closed-access, and cannot be easily extended. We present Lift-Bench, an IF benchmark that asks a model to draft a document given a topic and a structured outline. Constraints are scoped to certain document sections, providing finer control over constraint density. Lift-Bench includes both verifiable and judged constraints, as well as a training counterpart, Lift-Train. Importantly, Lift-Bench provides both main task and instruction-following constraints, mitigating shallow reward hacking. On experiments with multiple models, we find that frontier models struggle with the harder subsets of our benchmark and analyze model performance along multiple axes. We also find that training on Lift-Train is much less susceptible to reward hacking than training on the common IFTrain. Lift-Bench and Lift-Train are publicly available, as well as the data generating pipeline.
Instruction-following ability is critical for deploying large language models in real-world applications, where downstream components depend on the output satisfying specific constraints. Modern deployments increasingly handle the full task in a single LLM call, with one prompt specifying a layered output whose overall artifact, structural sections, and nested fields must each satisfy concrete constraints. Existing instruction-following benchmarks treat the constraint set as a flat list applied uniformly to the response, so they cannot scope a check to a particular section of the output. We introduce IFHierBench, a hierarchical instruction-following benchmark of 600 prompts stratified across four constraint-tree depths and 35 distinct constraints, each prompt paired with a deterministic checker that verifies satisfaction at every scope. Evaluating seven leading proprietary and open-weight models, we find that even the strongest model only marginally exceeds 50% prompt-level accuracy and that accuracy degrades sharply as constraint depth grows. Reliably following nested constraints remains a substantial gap for current LLMs, motivating future training methods that consider constraint adherence at finer granularity to achieve better instruction-following ability.