The Plain Writing Act in the United States requires government documents to be accessible in clear and simple language that the general public can easily understand, yet existing summarization systems struggle to address diverse linguistic and cognitive barriers among general readers. We present NRLB (No Reader Left Behind), a multi-agent framework for plain language summarization that simulates three representative reader groups: elementary school student readers, non-native readers, and readers with attention deficits. NRLB combines template-based planning with iterative, reader-oriented refinement, enabling systematic detection and resolution of difficult terms, missing contexts, and confusing sentences. Evaluations across multiple datasets demonstrate consistent improvements in readability while preserving factual accuracy. Human evaluation further validates NRLB's impact, with annotator preference rates ranging from 55% to 76%, highlighting NRLB's potential to produce plain language summaries that are both faithful to the source and broadly accessible to the general public.
Figures & tables
Figure 1: Examples of revision strategies based on the U.S. Plain Writing Act, including shortening sentences, using concrete language, and defining technical terms.
Figure 2: Overview of NRLB. Module 1 produces an initial draft via a Planner and Domain Expert Agent, while Module 2 iteratively refines it through a feedback loop with Reader, Checklist, Domain Expert, and Editor Agents.
PLOS
GovReport
BillSum
BigPatent
GPT-4o
Llama-3.1
Qwen3
GPT-4o
Llama-3.1
Qwen3
GPT-4o
Llama-3.1
Qwen3
GPT-4o
Llama-3.1
Qwen3
Initial Summary
46.85
47.27
41.62
37.93
46.93
46.19
46.70
45.81
41.08
41.87
44.94
34.21
ROUGE-1
Round 1
45.12
44.43
37.55
36.35
45.33
41.41
43.28
42.66
37.08
37.74
40.09
30.55
Round 2
42.77
41.64
36.03
34.73
44.19
40.04
40.88
39.70
34.62
35.40
36.87
29.18
Initial Summary
86.91
86.51
85.78
86.14
86.30
85.59
87.07
86.83
85.61
86.52
86.68
85.55
BERTScore
Round 1
86.61
86.00
85.33
85.89
85.84
85.04
86.43
86.20
85.03
85.82
85.73
84.89
Table 1: Performance of three models (GPT-4o, Llama-3.1-8B-Instruct, Qwen3-8B) on four benchmarks across initial summary and two refinement rounds; best scores in bold. ↓ indicates that lower is better in readability metrics.
Method
PLOS
GovReport
BillSum
BigPatent
FKGL ↓
DCRS ↓
CLI ↓
LENS
FKGL ↓
DCRS ↓
CLI ↓
LENS
FKGL ↓
DCRS ↓
CLI ↓
LENS
FKGL ↓
DCRS ↓
CLI ↓
LENS
All
15.40
12.06
13.19
67.64
16.09
12.01
14.29
59.08
15.00
11.68
13.09
62.63
14.68
11.51
12.93
64.40
w/o ele
15.73
12.19
13.45
66.18
16.56
12.05
14.68
59.94
14.48
11.35
12.52
62.26
15.23
11.67
12.79
61.90
w/o non
15.20
12.01
13.10
66.36
16.19
12.01
14.41
60.12
14.52
11.47
12.60
62.10
16.83
11.91
13.31
59.82
w/o att
15.40
12.06
13.21
67.31
16.34
12.06
14.51
59.27
14.49
11.42
12.58
63.01
15.57
11.74
13.00
62.34
w/o ele,non
16.01
12.28
13.68
64.75
17.03
12.05
14.88
58.26
14.61
11.39
12.58
62.77
16.64
11.90
13.31
58.71
Table 2: Ablation study on the effect of Reader Agent combinations across four datasets (200 samples each). “ele” = Elementary School Student Reader, “non” = Non-Native Reader, “att” = Attention-Deficit Reader.
Dataset
Setting
Initial
Round 1
Round 2
PLOS
w/ Editor
49.6662
49.1664 ( − 0.50 )
52.3063 ( +3.14 )
w/o Editor
49.9958
48.8337 ( − 1.16 )
49.8839 ( +1.05 )
GovReport
w/ Editor
43.6340
45.7042 ( +2.07 )
49.9039 ( +4.20 )
w/o Editor
43.6733
43.0163 ( − 0.66 )
43.6692 ( +0.65 )
BillSum
w/ Editor
33.4814
34.5100 ( +1.03 )
35.0046 ( +0.49 )
w/o Editor
33.2260
33.4913 ( +0.27 )
33.2915 ( − 0.20 )
Table 3: SummaC scores across datasets and revision rounds (Initial denotes the initial summary) with and without the Editor (Editor Agent). Positive changes are shown in bold.
Figure 3: Impact of checklist length on summary quality; K = 3 achieves the best trade-off between relevance and readability.
Relevance
Readability
Factuality
Dataset
Method
ROUGE-1
BERTScore
FKGL ↓
DCRS ↓
CLI ↓
LENS
SummaC
PLOS (500 samples)
AgentSimp-Synchronous
44.59
86.23
22.20
13.85
16.45
60.08
45.44
AgentSimp-Pipeline
44.92
86.31
22.33
13.84
16.37
60.79
46.07
NRLB (Ours)
41.64
85.43
15.21
12.06
13.20
66.68
50.86
GovReport (500 samples)
AgentSimp-Synchronous
43.62
85.55
27.41
14.44
17.14
50.24
45.14
AgentSimp-Pipeline
43.39
85.56
26.31
14.28
17.08
50.62
45.63
Table 4: Automatic evaluation of NRLB and AgentSimp variants on four benchmarks (500 samples each). Best scores are shown in bold.
Group
Dataset
NRLB
AgentSimp
Pref. (%)
Coh.
Simpl.
Faith.
Coh.
Simpl.
Faith.
Non-Native
PLOS
4.10
3.85
3.85
3.77
2.61
3.78
75.0
GovR.
4.27
3.73
4.12
3.72
3.35
3.62
65.0
BillSum
4.30
3.88
3.97
3.37
2.97
3.40
71.7
Patent
4.27
4.18
4.12
3.40
2.35
3.32
76.7
Average
4.23
3.91
4.00
3.56
2.82
3.53
72.1
Table 5: Human evaluation results (5-point Likert scale) for NRLB vs. AgentSimp across two groups (Non-Native and Elementary School Student). “GovR.” denotes GovReport; bold indicates the higher score.
Figure 4: Example case study shows how reader feedback drives lexical, contextual, and structural simplifications in NRLB.
Appendix figures & tables19 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: Prompt for the Planner Agent used for document type classification and expert assignment.
Figure 6: Prompt for the Editor Agent used for revision.
Figure 7: Role-based prompting templates for four document types.
Figure 8: Prompt for the Domain Expert Agent used for initial summary generation.
Figure 9: Revision prompt for Domain Expert Agents used for readability refinement.
Figure 10: Prompt instructions for Reader Agents used for readability assessment.
Dataset
# of agents
Combination
ROUGE-1
BERTScore
FKGL ↓
DCRS ↓
CLI ↓
LENS
SummaC
PLOS
1
w/ ele
41.71
85.46
15.27
12.03
13.09
66.84
55.48
w/ non
42.11
85.51
15.74
12.24
13.47
64.84
51.48
w/ att
42.24
85.45
16.01
12.28
13.68
64.75
51.18
2
w/ ele, non
41.98
85.50
15.40
12.06
13.21
67.31
50.35
w/ ele, att
41.77
85.39
15.20
12.01
13.10
66.36
52.74
w/ non, att
42.18
85.48
15.73
12.19
13.45
66.18
50.71
Appendix
Table 6: Results across datasets for different reader agent configurations. Bold values indicate the best result within each dataset. Agent abbreviations are as follows: ele = Elementary School Student, non = Non-Native, att = Attention-Deficit, sen = Senior, lea = Learning Difficulties.
Type
Dataset
Elementary
Non-Native
Attention-Deficit
Intersection (%)
unknown terms
PLOS
5,115
4,626
3,874
2,393 (17.58%)
GovReport
3,472
3,002
1,618
568 (7.02%)
BillSum
3,088
2,677
2,067
943 (12.04%)
BigPatent
3,139
2,857
2,334
1,321 (15.86%)
missing contexts
PLOS
87
6
3
1 (1.04%)
GovReport
82
7
10
2 (2.02%)
Appendix
Table 7: Feedback distribution and overlap across Reader Agents. Intersection (%) denotes the proportion of instances jointly identified by all three agents. Agent types are abbreviated as follows: Elementary = Elementary School Student Reader, Non-Native = Non-Native Reader, Attention-Deficit = Attention-Deficit Reader.
Dataset
Document Type
Domain
# Samples
PLOS
Paper
Biomedical
27.5K
GovReport
Report
Policy
19.5K
BillSum
Document
Legal
22K
BigPatent
Document
Patent
1.3M
Appendix
Table 8: Statistics of datasets
Dataset
Method
ROUGE-1
BERTScore
FKGL ↓
DCRS ↓
CLI ↓
LENS
SummaC
PLOS
Direct
37.52
85.38
12.59
11.61
12.89
78.05
58.67
NRLB-Round 1
44.43
86.00
16.55
12.67
14.54
67.21
49.17
NRLB-Round 2
41.64
85.43
15.21
12.06
13.20
66.68
50.86
GovReport
Direct
38.58
85.02
14.93
12.36
14.97
60.61
67.51
NRLB-Round 1
45.33
85.84
17.51
12.51
15.09
59.20
45.70
NRLB-Round 2
44.19
85.48
15.98
11.96
14.23
60.60
47.76
Appendix
Table 9: Comparison between the direct baseline and NRLB across datasets. ↓ indicates that lower values are better for readability-related metrics (FKGL, DCRS, CLI). Best scores per dataset are shown in bold.
PLOS
GovReport
BillSum
BigPatent
GPT-4o
Llama-3.1
Qwen3
GPT-4o
Llama-3.1
Qwen3
GPT-4o
Llama-3.1
Qwen3
GPT-4o
Llama-3.1
Qwen3
ROUGE-1
Initial Summary
46.85
47.27
41.62
37.93
46.93
46.19
46.70
45.81
41.08
41.87
44.94
34.21
Round 1
45.12
44.43
37.55
36.35
45.33
41.41
43.28
42.66
37.08
37.74
40.09
30.55
Round 2
42.77
41.64
36.03
34.73
44.19
40.04
40.88
39.70
34.62
35.40
36.87
29.18
Round 3
41.10
39.21
34.91
33.62
43.12
39.21
39.10
37.75
33.95
33.85
34.70
27.97
ROUGE-2
Initial Summary
13.83
14.80
9.50
14.93
20.07
14.87
20.80
22.89
15.68
13.28
17.71
8.38
Appendix
Table 10: Performance of three models (GPT-4o, Llama-3.1-8B-Instruct, Qwen3-8B) on four benchmarks across the initial summary and three refinement rounds. Best scores are shown in bold.
Metric
Stage
PLOS
GovReport
BillSum
BigPatent
Latency (Sec)
Round 0
17.21
16.59
9.82
11.35
Round 1
32.16
34.05
22.91
21.89
Round 2
47.39
52.80
35.86
31.89
Round 3
63.31
72.78
48.72
42.35
API Calls
Round 0
3.00
3.00
3.00
3.00
Round 1
7.97
7.96
7.96
7.97
Appendix
Table 11: Cost, latency, and API call analysis of NRLB across datasets and refinement rounds.
# K
Metric
PLOS
GovReport
BillSum
BigPatent
1
ROUGE-1
42.94
44.90
42.79
39.91
FKGL ↓
16.95
17.92
15.66
16.96
DCRS ↓
12.58
12.53
11.87
12.21
CLI ↓
14.39
15.43
13.49
14.08
LENS
65.28
59.29
62.46
61.93
SummaC
49.65
45.06
33.99
57.29
Appendix
Table 12: Evaluation results across datasets for different checklist sizes (K = 1, 2, 3, 6, 9).
Figure 11: Overview of the web-based human evaluation platform. The interface presents project background, evaluation goals, and scoring criteria.
Figure 12: Example evaluation screen for a medical document. Evaluators compare two summaries, assign scores on three criteria, and select a final preference.
Figure 13: NRLB revision process for a biomedical paper. Reader agents identify readability barriers, the Domain Expert Agent proposes targeted edits, and the Editor Agent integrates them into a simplified summary. Colors, underlines, and highlights indicate unknown terms, missing contexts, and confusing sentences, respectively.
Figure 14: NRLB revision process for a policy document. Reader agents identify readability barriers, the Domain Expert Agent proposes targeted edits, and the Editor Agent integrates them into a simplified summary. Colors, underlines, and highlights indicate unknown terms, missing contexts, and confusing sentences, respectively.
Figure 15: NRLB revision process for a legislative bill. Reader agents identify readability barriers, the Domain Expert Agent proposes targeted edits, and the Editor Agent integrates them into a simplified summary. Colors, underlines, and highlights indicate unknown terms, missing contexts, and confusing sentences, respectively.
Figure 16: NRLB revision process for a patent document. Reader agents identify readability barriers, the Domain Expert Agent proposes targeted edits, and the Editor Agent integrates them into a simplified summary. Colors, underlines, and highlights indicate unknown terms, missing contexts, and confusing sentences, respectively.
Plain Language Summaries (PLS) aim to make research accessible to lay readers, but they are typically written in a one-size-fits-all style that ignores differences in readers' information needs and comprehension. In health contexts, this limitation is particularly important because misunderstanding scientific information can affect real-world decisions. Large language models (LLMs) offer new opportunities for personalizing PLS, but it remains unclear whether personalization helps, which strategies are most effective, and how to balance personalization with safety. We introduce ReLay, a dataset of 300 participant--PLS pairs from 50 lay participants in both static (expert-written) and interactive (LLM-personalized) settings. ReLay includes user characteristics, health information needs, information-seeking behavior, comprehension outcomes, interaction logs, and quality ratings. We use ReLay to evaluate five LLMs across two personalization methods. Personalization improves comprehension and perceived quality, but it also raises the risk of reinforcing user biases and introducing hallucinations, revealing a trade-off between personalization and safety. These findings highlight the need for personalization methods that are both effective and trustworthy for diverse lay audiences.
Joey Chan, Yikun Han, Jingyuan Chen +8
University of Illinois Urbana-Champaign · University of Virginia · University of Colorado Anschutz +4
Interdisciplinary research is accelerating, yet scientific papers remain difficult to understand outside their home fields. We study large language model (LLM)-based simplification of scientific texts and present a human-in-the-loop workflow that transforms expert summaries into more accessible versions for non-specialists. Using SciSummNet as the source corpus, we first generate baseline simplifications with GPT-4o-mini. In Phase 1, readers from STEM fields outside computer science identify difficult sentences and phrases and compare the original and GPT-simplified summaries in terms of comprehensibility, naturalness, and simplicity. In Phase 2, computer science experts use this feedback to create expert-edited reference simplifications. We release the resulting corpus together with human judgments and automatic evaluation results. The Phase 1 judgments show a clear preference for the GPT-generated summaries in terms of comprehensibility and simplicity, while qualitative analysis of the Phase 2 edits highlights the importance of preserving domain-specific terminology and the strength of scientific claims. The resulting resource supports the training and benchmarking of simplification systems for cross-disciplinary scientific communication.
Kyuri Im, Michael Färber
ScaDS.AI, Technische Universität Dresden, Dresden, Germany
We introduce PlainMedScale, a topic-aligned medical corpus spanning four levels of comprehensibility in German and English, drawn from MSD (professional and consumer), Gesund.Bund, Apotheken Umschau Einfache Sprache, and the NHS. The four tiers correspond to distinct communicative functions --- reference, explanation, decision support, and access --- and move beyond the binary expert--lay contrast of prior corpora. In two pilot studies enabled by the alignments, we show that many readability metrics established on two registers fail to generalize across the full gradient, and that a SOTA open-weight LLM prompted for Plain Language still partially preserves the difficulty of its input. Code (https://github.com/GS-Uni-Heidelberg/PlainMedScale) and data (https://doi.org/10.5281/zenodo.21728290) are made available.
Bruno Brocai, Ilaria Papagno, Mayumi Ohta
Heidelberg University German Linguistics · Heidelberg University Translation Studies · Heidelberg University SFB 1671