Enhancing the ability of large language models (LLMs) to follow complex instructions is critical for their deployment in real-world applications. However, existing evaluation methods often oversimplify instruction complexity as a mere additive combination of atomic constraints, failing to adequately capture the high-dimensional complexity arising from the intricate interplay of content and format, logical workflow control, and real-world applications. This leads to a significant gap between current evaluation practices and practical demands. To bridge this gap, we introduce CCR-Bench, a novel benchmark designed to assess LLMs' adherence to complex instructions. CCR-Bench is characterized by: (1) deep entanglement of content and formatting requirements in task specifications; (2) instructions that involve intricate task decomposition, conditional reasoning, and procedural planning; and (3) evaluation samples derived entirely from real-world industrial scenarios. Extensive experiments on CCR-Bench demonstrate that even state-of-the-art models exhibit substantial performance deficiencies, clearly quantifying the gap between current LLM capabilities and the demands of realworld instruction understanding. We believe that CCR-Bench offers a more rigorous and realistic evaluation framework, advancing the development of LLMs toward the next generation of models capable of understanding and executing complex tasks in industrial applications.
Figures & tables
Figure 1: Framework for Complex Instructions Generation.
Figure 2: Distribution of Constraints System.
Scenario Name
Number
Real-World Scenario
Data Workflow
5
Flight Ticket Booking
10
Real Estate
10
Customer Service
10
Online Game
10
Printer Assistant
10
Table 1: Properties of the Logical Workflow Control Data.
Model
HSR
SSR
Non-thinking
GPT-4.1
0.119
0.702
DeepSeek-V3-0324
0.097
0.704
Qwen3-32B
0.089
0.651
Qwen2.5-72B-Instruct
0.091
0.631
Thinking
Gemini-2.5-Pro
0.064
0.758
OpenAI-o3-mini
0.166
0.755
Table 2: Results in Complex Content-Format Constraints.
Model
Content Elements
Formatting and Structuring
Logic and Reasoning
Contextual Application
Non-thinking
GPT-4.1
0.913
0.579
0.846
0.841
DeepSeek-V3-0324
0.908
0.595
0.818
0.835
Qwen3-32B
0.874
0.575
0.654
0.771
Qwen2.5-72B-Instruct
0.800
0.571
0.630
0.753
Thinking
Gemini-2.5-Pro
0.966
0.623
0.944
0.882
OpenAI-o3-mini
0.913
0.673
0.827
0.882
Table 3: SSR Metrics Across Different Constraint Categories.
Model
TSR
TCR
Non-thinking
GPT-4.1
0.529
0.800
DeepSeek-V3-0324
0.300
0.562
Qwen3-32B
0.157
0.438
Qwen2.5-72B-Instruct
0.329
0.631
Thinking
Gemini-2.5-Pro
0.700
0.844
OpenAI-o3-mini
0.514
0.768
Table 4: Results in Logical Workflow Control.
Figure 3: Impact of Tool-Use Chain Length on TSR.
Figure 4: Impact of Nested Workflows on TSR.
Figure 5: Impact of Implicit Tool Invocation on TSR.
Model
HSR
SSR
Non-thinking
GPT-4.1
0.135
0.640
DeepSeek-V3-0324
0.102
0.647
Qwen3-32B
0.158
0.689
Qwen2.5-72B-Instruct
0.080
0.558
Thinking
Gemini-2.5-Pro
0.415
0.817
OpenAI-o3-mini
0.242
0.652
Table 5: Results in Industrial Applications.
Appendix figures & tables21 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: The Prompt for Extracting Basic Instructions and Constraints from Instructions.
Figure 7: The Prompt for Assessing the Quality of Basic Instructions.
Figure 8: The Prompt for Identifying Task Domains of Basic Instructions.
Figure 9: The Prompt for Identifying Task Types of Basic Instructions.
Figure 10: The Prompt for Identifying Language Styles of Basic Instructions.
Figure 11: The Prompt for Assessing the Difficulty Levels of Basic Instructions.
Figure 12: Distribution of Diversity and Difficulty for Basic Instructions.
Figure 13: The Clustering Results of Constraints from Instructions.
Figure 14: The Prompt for Generating Complex Content-Format Constraints Data.
Figure 15: The Prompt for Assessing Instruction Adherence.
Figure 16: The Prompt for Model Acting as a User Agent.
Figure 17: The Prompt for Model Inference.
Figure 18: The Prompt for Extracting Constraints from Industrial Application Data.
Constraint Type
Constraint Details
GPT-4.1
DeepSeek-V3-0324
Qwen3-32B
Qwen2.5-72B-Instruct
Contextual Application
Scenario Simulation
0.841
0.835
0.771
0.753
Content Elements
Expressive Techniques
0.771
0.857
0.814
1.000
Content Source
0.931
0.992
0.908
0.831
Specific Content
0.936
0.885
0.873
0.746
Formatting and Structuring
Word Count Limitation
0.177
0.113
0.186
0.160
Content Structure
0.776
0.902
0.825
0.865
Appendix
Table 6: SSR Metrics on Complex Content-Format Constraints Data in Non-thinking Mode.
Constraint Type
Constraint Details
Gemini-2.5-Pro
OpenAI-o3-mini
DeepSeek-R1-0528
QwQ-32B
Qwen3-32B
Contextual Application
Scenario Simulation
0.882
0.882
0.918
0.829
0.765
Content Elements
Expressive Techniques
1.000
0.986
1.000
1.000
1.000
Content Source
0.939
1.000
1.000
0.992
1.000
Specific Content
0.970
0.864
0.982
0.952
0.903
Formatting and Structuring
Word Count Limitation
0.058
0.303
0.215
0.181
0.160
Content Structure
0.937
0.912
0.925
0.922
0.827
Appendix
Table 7: SSR Metrics on Complex Content-Format Constraints Data in Thinking Mode.
Reasoning Mode
Model
Maze
Tree Drawing
World Cup
TSR
TCR
TSR
TCR
TSR
TCR
Non-thinking
GPT-4.1
0.600
0.633
0.800
0.989
0.000
0.641
DeepSeek-V3-0324
0.000
0.085
0.200
0.591
0.000
0.476
Qwen3-32B
0.000
0.085
0.000
0.201
0.000
0.280
Qwen2.5-72B-Instruct
0.000
0.183
0.000
0.696
0.000
0.807
Thinking
Gemini-2.5-Pro
1.000
1.000
1.000
1.000
0.600
1.000
Appendix
Table 8: Model Performance on Abstract Scenario Data.
Scenario
GPT-4.1
DeepSeek- V3-0324
Qwen3-32B
Qwen2.5- 72B-Instruct
TSR
TCR
TSR
TCR
TSR
TCR
TSR
TCR
Customer Service
0.500
0.892
0.100
0.301
0.200
0.656
0.100
0.333
Data Flow
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
Flight Ticket Booking
0.900
0.980
0.600
0.813
0.100
0.578
0.600
0.800
Maze
0.600
0.633
0.000
0.085
0.000
0.085
0.000
0.183
Online Game
0.400
0.790
0.500
0.699
0.100
0.388
0.300
0.660
Appendix
Table 9: Model Performance Across Various Scenario in Non-thinking Mode.
Scenario
Gemini- 2.5-Pro
OpenAI- o3-mini
DeepSeek- R1-0528
QwQ-32B
Qwen3-32B
TSR
TCR
TSR
TCR
TSR
TCR
TSR
TCR
TSR
TCR
Customer Service
0.700
0.863
0.400
0.686
0.300
0.504
0.300
0.507
0.400
0.633
Data Flow
1.000
1.000
0.800
0.900
1.000
1.000
0.800
0.960
0.800
0.960
Flight Ticket Booking
0.700
0.790
0.700
0.930
0.500
0.800
0.700
0.858
0.600
0.830
Maze
1.000
1.000
1.000
1.000
0.400
0.441
0.200
0.631
0.400
0.585
Online Game
0.500
0.667
0.400
0.650
0.300
0.568
0.400
0.674
0.300
0.590
Appendix
Table 10: Model Performance Across Various Scenario in Thinking Mode.
Constraint
GPT-4.1
DeepSeek-V3-0324
Qwen3-32B
Qwen2.5-72B-Instruct
Content Filling Requirement
0.400
0.900
1.000
0.667
Chief Complaint Filling Requirement
0.986
0.943
0.729
0.686
Personal History Filling Requirement
0.714
0.429
0.714
0.714
Family History Filling Requirement
1.000
0.714
0.986
1.000
Marital and Childbearing History Filling Requirement
0.857
0.571
0.857
0.857
Past Medical History Filling Requirement
0.557
0.343
0.314
0.414
Appendix
Table 11: SSR Metrics on Industrial Applications Data in Non-thinking Mode.
Constraint
Gemini-2.5-Pro
OpenAI-o3-mini
DeepSeek-R1-0528
QwQ-32B
Qwen3-32B
Content Filling Requirement
0.600
0.500
0.533
0.433
0.900
Chief Complaint Filling Requirement
1.000
0.971
1.000
0.914
0.814
Personal History Filling Requirement
0.643
0.143
0.571
0.429
0.143
Family History Filling Requirement
1.000
0.286
0.714
0.700
0.286
Marital and Childbearing History Filling Requirement
0.943
0.157
0.571
0.571
0.143
Past Medical History Filling Requirement
0.686
0.314
0.214
0.143
0.043
Appendix
Table 12: SSR Metrics on Industrial Applications Data in Thinking Mode.
Figure 19: Responses of Five Models on the Medical Example.