STAIF: A Stage-wise Optimization for Complex Instruction Following

Jian Hong; Chen Cheng; Quan Liu; Yuhao Chen; Enhong Chen

cs.AIJun 26, 2026

STAIF: A Stage-wise Optimization for Complex Instruction Following

Authors: Jian Hong, Chen Cheng, Quan Liu, Yuhao Chen, Enhong Chen

Organizations: University of Science and Technology of China · iFLYTEK Research Group

Abstract

Following complex instructions with multiple explicit constraints remains a fundamental challenge for large language models (LLMs). Existing alignment methods, such as DPO, optimize holistic reward signals that often underemphasize strict satisfaction of individual constraints, particularly under out-of-distribution or multi-constraint settings. In this paper, we propose STAIF, a stage-wise optimization framework that decouples the alignment of subjective (soft) constraints from the optimization of objectively verifiable (hard) constraints. Stage 1 applies preference optimization with multiple negative samples to sharpen sensitivity to soft constraints, while Stage 2 applies Reinforcement Learning with Verifiable Rewards (RLVR) to enforce strict compliance with hard constraints. To support this method, we construct STAINSTRUCT, a high-quality bilingual (English, Chinese) dataset of approximately 31,000 complex multi-constraint instructions. Extensive analyses validate the design of STAIF and show state-of-the-art performance on representative benchmarks against strong baselines, as well as genuine generalization.

Figures & tables

Figure 1: Illustration of StaIF : Stage 1 soft-constraint alignment; Stage 2 hard-constraint verification.

Figure 2: Overview of the data construction pipeline for StaInstruct .

Figure 3: Mapping between task categories and constraint categories based on real-world user data.

Figure 4: Distribution of the number of constraints per instruction in StaInstruct .

Figure 5: Constraint type distribution in StaInstruct .

Model	IFEval				IFBench				CFBench
Model	P-S	P-L	I-S	I-L	P-S	P-L	I-S	I-L	CSR	ISR	PSR
GPT-4.1	87.8	91.2	91.1	93.2	38.7	43.8	42.1	47.5	83.0	61.0	70.0
DeepSeek-v3.2	86.4	90.5	90.8	93.5	39.7	43.5	42.3	46.8	87.0	65.0	72.0
Gemini 2.5	89.8	91.1	92.9	94.0	39.7	50.3	43.0	52.5	83.0	60.0	70.0
Doubao 1.8	83.4	87.0	88.5	90.0	29.3	36.4	32.8	39.7	83.0	60.0	68.0
QwQ-32B	83.5	87.7	88.8	91.8	36.3	40.4	39.7	44.2	87.0	61.0	70.0

Table 1: Main results on IFEval, IFBench, and CFBench. For IFEval and IFBench we report accuracy (%) at the prompt (P) and instruction (I) levels under strict (S) and loose (L) matching: P-S , P-L , I-S , I-L . For CFBench, we report CSR , ISR , and PSR (%).

Model	Method	IFEval				IFBench				CFBench
Model	Method	P-S	P-L	I-S	I-L	P-S	P-L	I-S	I-L	CSR	ISR	PSR
Qwen2.5-7B	StaIF	87.0	88.7	91.0	92.3	44.2	48.3	46.0	49.9	82.0	51.0	61.0
	w/o stage1	80.2	82.8	86.3	88.1	37.8	41.2	39.7	43.9	80.0	45.0	58.0
	w/o stage2	79.4	82.0	85.5	87.6	38.4	42.5	40.3	45.4	81.0	46.0	58.0
Tulu3-8B	StaIF	90.4	91.1	92.9	93.5	50.3	52.0	53.1	55.2	73.0	35.0	45.0
	w/o stage1	85.1	87.0	89.3	90.2	42.2	45.6	43.3	47.8	70.0	33.0	41.0

Table 2: Ablation study on the two training stages of StaIF , evaluated on Qwen2.5-7B and Tulu3-8B.

Method	IFEval				IFBench
Method	P-S	P-L	I-S	I-L	P-S	P-L	I-S	I-L
DPO (joint)	79.1	82.0	84.6	86.9	36.9	39.5	39.7	43.7
RLVR (joint)	81.6	83.3	86.9	89.4	41.3	43.2	44.1	45.1
DPO+RLVR (joint)	84.5	86.8	88.4	90.7	41.1	45.1	44.7	48.3
StaIF	87.0	88.7	91.0	92.3	44.2	48.3	46.0	49.9

Table 3: Impact of decoupling soft and hard constraints, evaluated on Qwen2.5-7B.

Model	Method	Negative	IFEval				IFBench				CFBench
Model	Method	Sample	P-S	P-L	I-S	I-L	P-S	P-L	I-S	I-L	CSR	ISR	PSR
Qwen2.5-7B	Stage 1 only	Single	76.3	80.4	83.5	86.5	34.0	38.1	36.1	40.6	80.0	43.0	56.0
	Stage 1 only	Multiple	79.4	82.0	85.5	87.6	38.4	42.5	40.3	45.4	81.0	46.0	58.0
	StaIF	Single	85.0	86.3	89.7	90.6	42.2	46.5	44.9	47.5	80.0	46.0	56.0
	StaIF	Multiple	87.0	88.7	91.0	92.3	44.2	48.3	46.0	49.9	82.0	51.0	61.0
Tulu3-8B	Stage 1 only	Single	76.5	79.4	83.5	86.1	31.6	34.6	33.7	37.0	69.0	30.0	40.0

Table 4: Comparison of single vs. multiple negative samples in StaIF , evaluated on Qwen2.5-7B and Tulu3-8B.

Model	MT-Bench	GSM8K	MMLU-Pro	BBH
Qwen2.5-7B-Instruct	7.4	91.6	55.9	72.3
+ StaIF	7.5	91.4	56.0	74.4
Tulu3-8B-SFT	6.3	78.7	35.0	67.8
+ StaIF	7.1	81.3	35.4	71.2

Table 5: Semantic quality evaluation before and after applying StaIF . MT-Bench is reported on its standard 10-point scale; GSM8K, MMLU-Pro, and BBH are reported as accuracy percentages.

Figure 6: Performance by constraint type on IFEval and IFBench (prompt-level, loose accuracy) for Tulu3, Tulu3+Stage 1, and Tulu3+ StaIF .

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Category	Sub-category	Description
Semantic Understanding	Semantic Analysis	Analyze the logical structure, semantic relationships, and core meaning of text.
	Sentiment Classification	Identify the emotional tendency or intensity of text.
	Element Extraction	Extract key information units (e.g., people, events, relationships) from unstructured text.
	Element Recognition	Identify extracted entities (e.g., “Apple” $\rightarrow$ “Organization”, “Beijing” $\rightarrow$ “Location”).
	Natural Language Inference	Determine the logical relationship among contents (Entailment/Contradiction/Neutral).
	Text Processing	Preprocessing (cleaning, segmentation, normalization) or postprocessing (formatting, filtering) of text data.

Appendix

Table 6: Task Taxonomy.

Category	Sub-category	Description
Content	Theme	The generated content should focus on a specific topic or field.
	Exclusion	Clearly specify the information or content that should not be included in the generated content.
	Inclusion	Clearly specify the particular information or content that must be included in the generated content.
	Value	The generated content should not contain information that violates values, such as safety, false information, discrimination, or bias.
	Numerical	Limit the length and number of words, sentences, and paragraphs in the generated content, or use numerical precision constraints to ensure accuracy.
	Target Audience	The generated content should target a specific audience, which affects terminology, detail level, and complexity.

Appendix

Table 7: Constraint Taxonomy.

	$\displaystyle\mathcal{L}_{\text{StaIF}}(\pi_{\theta};\pi_{\text{ref}})$	(9)
$\displaystyle=$	$\displaystyle-\mathbb{E}_{(x,\mathbf{y})\sim\mathcal{D}}\left[\log\frac{\exp\left(\beta\log\frac{\pi_{\theta}(y^{+}\mid x)}{\pi_{\text{ref}}(y^{+}\mid x)}\right)}{\exp\left(\beta\log\frac{\pi_{\theta}(y^{+}\mid x)}{\pi_{\text{ref}}(y^{+}\mid x)}\right)+\sum_{i}\exp\left(\beta\log\frac{\pi_{\theta}(y^{-}_{i}\mid x)}{\pi_{\text{ref}}(y^{-}_{i}\mid x)}\right)}\right]$
$\displaystyle=$	$\displaystyle-\mathbb{E}_{(x,\mathbf{y})\sim\mathcal{D}}\left[\log S_{\mathbf{y}}\left(\beta\log\frac{\pi_{\theta}(y^{+}\mid x)}{\pi_{\text{ref}}(y^{+}\mid x)}\right)\right]$

Appendix

Table 14

Stage 1
Initial learning rate	$1\times 10^{-5}$
Number of training epochs	3
$\beta$	0.1
Maximum sequence length	4,096
Per-GPU batch size	1

Appendix

Table 15

Stage 2
Initial learning rate	$1\times 10^{-6}$
Maximum sequence length	4,096
Number of training epochs	2
Number of rollouts per iteration	16
KL divergence loss coefficient	$1\times 10^{-3}$

Appendix

Table 16

Qwen2.5-7B-Instruct
Model	IFEval				CFBench
Model	P-S	P-L	I-S	I-L	CSR	ISR	PSR
VeRL	–	83.9*	–	–	76.0*	40.0*	52.0*
RLCF	72.6*	77.3*	80.3*	84.1*	–	–	–
StaIF	87.0	88.7	91.0	92.3	82.0	51.0	61.0
LLaMA3-8B-Instruct

Appendix

Table 8: Comparison with recent instruction-following alignment methods. Dashes indicate metrics not reported by the corresponding method. * are cited from the corresponding reference for each method.

LLM Alignment

Large language model alignment studies methods for making models follow human intentions, values, and safety constraints across contexts.

See all 611 papers

ScopeIF: Improving Scope-Aware Precise Instruction-Following in Large Language Models via Graded Reward Modeling
Sep 26, 2026Bosi Wen, Yilin Niu, Xiaoying Ning +3Instruction Following Instruction Tuning
The Conversational Artificial Intelligence (CoAI) Group, Tsinghua University · Zhipu AI
LIFT: Scoped Constraints for Evaluating and Training Instruction Following
Oct 20, 2026Eran Goldstein, Yuval Belfer, Yonatan Belinkov +1Instruction Following Generative AI Evaluation
AI21 Labs
IFHierBench: Hierarchical Instruction Following for Large Language Models
Jul 30, 2026Yuetian Mao, Chunyang ChenLLM Evaluation Language Model Generation Evaluation
Technical University of Munich

Sep 26, 2026J/K move · Enter open · S save

ScopeIF: Improving Scope-Aware Precise Instruction-Following in Large Language Models via Graded Reward Modeling

Bosi Wen, Yilin Niu, Xiaoying Ning, Ying Zhang+2

The Conversational Artificial Intelligence (CoAI) Group, Tsinghua University · Zhipu AI

Instruction Following Instruction Tuning RL for Language Models Reward Modeling

Open paper AskPDF arXiv

Precise instruction-following is a fundamental ability of large language models (LLMs), requiring their outputs to strictly satisfy objective constraints in input instructions. In complex application scenarios, these constraints often possess diverse scopes that govern specific response segments rather than the entire output. However, existing optimization methods often neglect constraint scope during data construction and rely on binary per-constraint rewards, yielding limited data diversity and sparse supervision for complex constraints. To this end, we propose ScopeIF, a novel training framework for scope-aware precise instruction-following. We first introduce a unified schema that factorizes objective constraints into three decoupled dimensions: Scope, Target, and Range. Grounded in this schema, we construct ScopeInstruct, a large-scale instruction dataset with diverse scope-aware constraints, and combine tool-grounded verification with graded reward modeling to quantify the violation degree of each constraint, providing dense supervision for policy optimization. Extensive experiments demonstrate that ScopeIF consistently outperforms existing methods, particularly on complex scope-aware constraints, while preserving general capabilities. Notably, it enables optimized Qwen3-4B and 8B models to rival or surpass strong frontier models such as Gemini-2.5-Pro and DeepSeek-V3.2, establishing an effective paradigm for advancing scope-aware instruction-following. Our code and data are available at https://github.com/thu-coai/ScopeIF.

Abstract

LLM Alignment

Explore similar work

ScopeIF: Improving Scope-Aware Precise Instruction-Following in Large Language Models via Graded Reward Modeling

LIFT: Scoped Constraints for Evaluating and Training Instruction Following

IFHierBench: Hierarchical Instruction Following for Large Language Models