Think about how human handles complex reading tasks: marking key points, inferring their relationships, and structuring information to guide understanding and responses. Likewise, can a large language model benefit from text structure to enhance text-processing performance? To explore it, in this work, we first introduce Structure of Thought (SoT), a prompting technique that explicitly guides models to construct intermediate text structures, consistently boosting performance across eight tasks and three model families. Building upon this insight, we present T2S-Bench, the first benchmark designed to evaluate and improve text-to-structure capabilities of models. T2S-Bench includes 1.8K samples across 6 scientific domains and 32 structural types, rigorously constructed to ensure accuracy, fairness, and quality. Evaluation on 45 mainstream models reveals substantial improvement potential: the average accuracy on the multi-hop reasoning task is only 52.1%, and even the most advanced model achieves 58.1% node accuracy in end-to-end extraction. Furthermore, on Qwen2.5-7B-Instruct, SoT alone yields an average +5.7% improvement across eight diverse text-processing tasks, and fine-tuning on T2S-Bench further increases this gain to +8.6%. These results highlight the value of explicit text structuring and the complementary contributions of SoT and T2S-Bench. Dataset and eval code have been released at https://t2s-bench.github.io/T2S-Bench-Page/.
Figures & tables
Figure 1: Performance of SoT and Importance of Text Structuring. We evaluated three models on eight distinct text-processing tasks using three prompting strategies: direct answering, Chain-of-Thought (CoT), and Structure of Thought (SoT). The horizontal axis shows the model’s performance with direct answering, while the vertical axis indicates the performance change relative to direct answering. Our evaluations follow standards from lm-eval and Longbench tasks. SoT consistently boosts performance across different tasks and models.
Figure 2: Construction Process of T2S-Bench, including Sample Collection, Muti-hop Reasoning and End-to-End Dataset Construction.
Domain
Specific Area
CS Algorithm / AI
(CS1) Model Architecture (CS2) Training / Inference Pipeline (CS3) RAG / Agent System Component Diagram
CS System
(CS4) End to end Pipeline (CS5) System Component Architecture (CS6) Serving / Inference System
Table 1: Major Science Domain Taxonomy (Compact). Abbreviations: CS Computer Science, LS Life Sciences, SS Social Sciences, ES Environmental Sciences, EM Economics & Management Sciences, PS Physical Sciences.
1. Fault Localization
Single
(FL2) First directly affected downstream node, (FL4) Key bottleneck / Dominator node, (FL6) Branch isolation: who will not be affected, (FL7) Fault amplification point in a feedback loop
Multiple
(FL1) Upstream root cause set, (FL3) Minimum cut set, (FL8) Multi fault explainability
Table 2: Task Taxonomy of T2SBench. Organized by Single choice , Multiple choice , and Mixed (Single+Multiple choice) .
Figure 3: (Left) Sample Distributions of different Dataset. (Right) Overview of T2S-Bench Sample Distributions.
Multi-choice QA Overall
Boundary Testing
Counterfactual Reasoning
Fault Localization
Functional Mapping
Structure Score
Model
EM
F1
EM
F1
EM
F1
EM
F1
EM
F1
Node
Link
Gemini-2.5-Pro
81.40
91.56
80.00
91.78
90.65
93.46
74.51
90.62
89.86
91.01
58.09
84.32
Gemini-2.5-Flash
72.20
83.67
77.50
87.33
78.50
84.33
64.22
82.76
76.81
78.94
46.90
75.10
Gemini-2.0-flash
61.20
79.63
72.50
87.03
75.70
86.76
41.67
70.99
76.81
81.26
42.71
66.42
Gemini-2.0-flash-lite
53.40
72.41
65.00
81.08
79.44
83.74
29.90
63.35
62.32
66.57
39.22
69.18
GPT-5.2
71.80
84.32
77.50
90.47
84.11
89.69
62.75
82.24
69.57
71.45
50.57
77.76
Table 4: Overall performance on T2S-Bench. The leftmost two columns show overall performance (EM and F1 scores) on T2S-Bench-MR, multiple-choice dataset requiring multi-hop reasoning. The central columns represent accuracy for each question type within T2S-Bench-MR. The rightmost two columns show performance on the T2S-Bench-E2E dataset, separately evaluating node extraction (measured by average semantic similarity between predicted and reference nodes) and link extraction (measured by F1 score of correctly identified link pairs). The best-performing model within each model family on T2S-Bench-MR is highlighted with pink shading. Additionally, the top three performances in each metric are highlighted in red (first place), green (second place), and blue (third place).
T2S-Bench
LongBench
Scrolls
Model
MC (EM)
MC (F1)
HotpotQA
2WikiMQA
Qasper
GovReport
QMSum
ContractNLI
Quality
Summscreen
Qwen2.5-7B-Instruct
28.8
59.4
60.0
46.9
43.5
32.0
23.9
55.8
37.8
17.3
+ CoT
36.6
62.1
62.6
59.0
41.9
25.7
19.7
52.1
38.4
15.8
+ SoT
40.6
68.4
65.8
63.2
48.0
37.3
27.9
58.4
41.4
21.3
+ T2S-Train
46.1
73.5
68.2
65.3
51.2
41.2
30.9
60.3
42.8
24.5
LLaMA3.1-8B-Instruct
24.2
54.2
59.1
53.7
44.9
34.3
25.2
31.5
39.8
26.8
Table 5: Performance improvements after fine-tuning on the T2S-Train-1.2k. We fine-tuned Qwen2.5-7B-Instruct and Llama3.1-8B-Instruct models for 100 epochs using GRPO on T2S-Train-1.2k (detailed training settings provided in Appendix F.1 ). Evaluations were conducted on various text-processing tasks within Longbench and SCROLLS using lm-eval.
Figure 4: F1 scores across different topics on T2S-Bench-MR. We selected one representative model from each model family; The first fig shows their average F1 scores across various domains. The remains fig illustrate individual model performances per domain, with the vertical axis indicating deviations from the average performance. The dark dashed rectangle represents the average performance (set to zero). Scores outside this rectangle indicate above-average performance, while scores inside indicate below-average performance..
Figure 5: Link F1 scores on MR-Bench-E2E across texts corresponding to reference graphs with varying node counts.
Figure 6: Correlation of model performance between T2S-Bench-MR and Longbench Pro Dataset.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 7: Structure diagram examples across main domains
Overall
CS
Eco
Environ
Life
Phy
Social
Gemini-2.5-Pro
81.40
91.56
83.48
93.12
78.95
91.66
75.76
91.67
80.65
88.57
88.52
92.33
80.90
91.96
Gemini-2.5-Flash
72.20
83.67
73.91
81.18
65.79
82.04
71.21
85.53
65.59
79.09
81.97
88.81
76.40
88.14
Gemini-2.0-flash
61.20
79.63
61.74
77.07
61.84
80.95
60.61
80.81
59.14
79.32
67.21
76.12
58.43
83.66
Gemini-2.0-flash-lite
53.40
72.41
53.04
72.01
50.00
69.04
56.06
74.44
53.76
73.50
60.66
68.47
49.44
75.88
GPT-5.2
71.80
84.32
73.91
82.55
65.79
86.44
66.67
79.85
73.12
87.10
77.05
80.77
73.03
87.65
GPT-5.1
67.80
83.46
73.91
83.83
65.79
83.65
68.18
82.22
62.37
82.32
73.77
80.42
62.92
86.99
Appendix
Table 6: Performance on Multi choice QA. Shaded columns report EM, unshaded columns report F1.