No Reader Left Behind: Multi-Agent Summaries Everyone Can Understand

Jimin Jung; MyoungJin Kim; Jaehyung Seo; Heuiseok Lim

cs.CLApr 12, 2026

No Reader Left Behind: Multi-Agent Summaries Everyone Can Understand

Authors: Jimin Jung, MyoungJin Kim, Jaehyung Seo, Heuiseok Lim

Organizations: Department of Computer Science and Engineering, Korea University · Department of Computer Science and Engineering, Konkuk University

Abstract

The Plain Writing Act in the United States requires government documents to be accessible in clear and simple language that the general public can easily understand, yet existing summarization systems struggle to address diverse linguistic and cognitive barriers among general readers. We present NRLB (No Reader Left Behind), a multi-agent framework for plain language summarization that simulates three representative reader groups: elementary school student readers, non-native readers, and readers with attention deficits. NRLB combines template-based planning with iterative, reader-oriented refinement, enabling systematic detection and resolution of difficult terms, missing contexts, and confusing sentences. Evaluations across multiple datasets demonstrate consistent improvements in readability while preserving factual accuracy. Human evaluation further validates NRLB's impact, with annotator preference rates ranging from 55% to 76%, highlighting NRLB's potential to produce plain language summaries that are both faithful to the source and broadly accessible to the general public.

Figures & tables

Figure 1: Examples of revision strategies based on the U.S. Plain Writing Act, including shortening sentences, using concrete language, and defining technical terms.

Figure 2: Overview of NRLB. Module 1 produces an initial draft via a Planner and Domain Expert Agent, while Module 2 iteratively refines it through a feedback loop with Reader, Checklist, Domain Expert, and Editor Agents.

		PLOS			GovReport			BillSum			BigPatent
		GPT-4o	Llama-3.1	Qwen3	GPT-4o	Llama-3.1	Qwen3	GPT-4o	Llama-3.1	Qwen3	GPT-4o	Llama-3.1	Qwen3
	Initial Summary	46.85	47.27	41.62	37.93	46.93	46.19	46.70	45.81	41.08	41.87	44.94	34.21
ROUGE-1	Round 1	45.12	44.43	37.55	36.35	45.33	41.41	43.28	42.66	37.08	37.74	40.09	30.55
	Round 2	42.77	41.64	36.03	34.73	44.19	40.04	40.88	39.70	34.62	35.40	36.87	29.18
	Initial Summary	86.91	86.51	85.78	86.14	86.30	85.59	87.07	86.83	85.61	86.52	86.68	85.55
BERTScore	Round 1	86.61	86.00	85.33	85.89	85.84	85.04	86.43	86.20	85.03	85.82	85.73	84.89

Table 1: Performance of three models (GPT-4o, Llama-3.1-8B-Instruct, Qwen3-8B) on four benchmarks across initial summary and two refinement rounds; best scores in bold. ↓ indicates that lower is better in readability metrics.

Method	PLOS				GovReport				BillSum				BigPatent
Method	FKGL $\downarrow$	DCRS $\downarrow$	CLI $\downarrow$	LENS	FKGL $\downarrow$	DCRS $\downarrow$	CLI $\downarrow$	LENS	FKGL $\downarrow$	DCRS $\downarrow$	CLI $\downarrow$	LENS	FKGL $\downarrow$	DCRS $\downarrow$	CLI $\downarrow$	LENS
All	15.40	12.06	13.19	67.64	16.09	12.01	14.29	59.08	15.00	11.68	13.09	62.63	14.68	11.51	12.93	64.40
w/o ele	15.73	12.19	13.45	66.18	16.56	12.05	14.68	59.94	14.48	11.35	12.52	62.26	15.23	11.67	12.79	61.90
w/o non	15.20	12.01	13.10	66.36	16.19	12.01	14.41	60.12	14.52	11.47	12.60	62.10	16.83	11.91	13.31	59.82
w/o att	15.40	12.06	13.21	67.31	16.34	12.06	14.51	59.27	14.49	11.42	12.58	63.01	15.57	11.74	13.00	62.34
w/o ele,non	16.01	12.28	13.68	64.75	17.03	12.05	14.88	58.26	14.61	11.39	12.58	62.77	16.64	11.90	13.31	58.71

Table 2: Ablation study on the effect of Reader Agent combinations across four datasets (200 samples each). “ele” = Elementary School Student Reader, “non” = Non-Native Reader, “att” = Attention-Deficit Reader.

Dataset	Setting	Initial	Round 1	Round 2
PLOS	w/ Editor	49.6662	49.1664 ( $-$ 0.50 )	52.3063 ( +3.14 )
PLOS	w/o Editor	49.9958	48.8337 ( $-$ 1.16 )	49.8839 ( +1.05 )
GovReport	w/ Editor	43.6340	45.7042 ( +2.07 )	49.9039 ( +4.20 )
GovReport	w/o Editor	43.6733	43.0163 ( $-$ 0.66 )	43.6692 ( +0.65 )
BillSum	w/ Editor	33.4814	34.5100 ( +1.03 )	35.0046 ( +0.49 )
BillSum	w/o Editor	33.2260	33.4913 ( +0.27 )	33.2915 ( $-$ 0.20 )

Table 3: SummaC scores across datasets and revision rounds (Initial denotes the initial summary) with and without the Editor (Editor Agent). Positive changes are shown in bold.

Figure 3: Impact of checklist length on summary quality; K = 3 achieves the best trade-off between relevance and readability.

		Relevance		Readability				Factuality
Dataset	Method	ROUGE-1	BERTScore	FKGL $\downarrow$	DCRS $\downarrow$	CLI $\downarrow$	LENS	SummaC
PLOS (500 samples)	AgentSimp-Synchronous	44.59	86.23	22.20	13.85	16.45	60.08	45.44
	AgentSimp-Pipeline	44.92	86.31	22.33	13.84	16.37	60.79	46.07
	NRLB (Ours)	41.64	85.43	15.21	12.06	13.20	66.68	50.86
GovReport (500 samples)	AgentSimp-Synchronous	43.62	85.55	27.41	14.44	17.14	50.24	45.14
	AgentSimp-Pipeline	43.39	85.56	26.31	14.28	17.08	50.62	45.63

Table 4: Automatic evaluation of NRLB and AgentSimp variants on four benchmarks (500 samples each). Best scores are shown in bold.

Group	Dataset	NRLB			AgentSimp			Pref. (%)
Group	Dataset	Coh.	Simpl.	Faith.	Coh.	Simpl.	Faith.	Pref. (%)
Non-Native	PLOS	4.10	3.85	3.85	3.77	2.61	3.78	75.0
	GovR.	4.27	3.73	4.12	3.72	3.35	3.62	65.0
	BillSum	4.30	3.88	3.97	3.37	2.97	3.40	71.7
	Patent	4.27	4.18	4.12	3.40	2.35	3.32	76.7
	Average	4.23	3.91	4.00	3.56	2.82	3.53	72.1

Table 5: Human evaluation results (5-point Likert scale) for NRLB vs. AgentSimp across two groups (Non-Native and Elementary School Student). “GovR.” denotes GovReport; bold indicates the higher score.

Figure 4: Example case study shows how reader feedback drives lexical, contextual, and structural simplifications in NRLB.

Appendix figures & tables19 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Prompt for the Planner Agent used for document type classification and expert assignment.

Figure 6: Prompt for the Editor Agent used for revision.

Figure 7: Role-based prompting templates for four document types.

Figure 8: Prompt for the Domain Expert Agent used for initial summary generation.

Figure 9: Revision prompt for Domain Expert Agents used for readability refinement.

Figure 10: Prompt instructions for Reader Agents used for readability assessment.

Dataset	# of agents	Combination	ROUGE-1	BERTScore	FKGL $\downarrow$	DCRS $\downarrow$	CLI $\downarrow$	LENS	SummaC
PLOS	1	w/ ele	41.71	85.46	15.27	12.03	13.09	66.84	55.48
		w/ non	42.11	85.51	15.74	12.24	13.47	64.84	51.48
		w/ att	42.24	85.45	16.01	12.28	13.68	64.75	51.18
	2	w/ ele, non	41.98	85.50	15.40	12.06	13.21	67.31	50.35
		w/ ele, att	41.77	85.39	15.20	12.01	13.10	66.36	52.74
		w/ non, att	42.18	85.48	15.73	12.19	13.45	66.18	50.71

Appendix

Table 6: Results across datasets for different reader agent configurations. Bold values indicate the best result within each dataset. Agent abbreviations are as follows: ele = Elementary School Student, non = Non-Native, att = Attention-Deficit, sen = Senior, lea = Learning Difficulties.

Type	Dataset	Elementary	Non-Native	Attention-Deficit	Intersection (%)
unknown terms	PLOS	5,115	4,626	3,874	2,393 (17.58%)
	GovReport	3,472	3,002	1,618	568 (7.02%)
	BillSum	3,088	2,677	2,067	943 (12.04%)
	BigPatent	3,139	2,857	2,334	1,321 (15.86%)
missing contexts	PLOS	87	6	3	1 (1.04%)
	GovReport	82	7	10	2 (2.02%)

Appendix

Table 7: Feedback distribution and overlap across Reader Agents. Intersection (%) denotes the proportion of instances jointly identified by all three agents. Agent types are abbreviated as follows: Elementary = Elementary School Student Reader, Non-Native = Non-Native Reader, Attention-Deficit = Attention-Deficit Reader.

Dataset	Document Type	Domain	# Samples
PLOS	Paper	Biomedical	27.5K
GovReport	Report	Policy	19.5K
BillSum	Document	Legal	22K
BigPatent	Document	Patent	1.3M

Appendix

Table 8: Statistics of datasets

Dataset	Method	ROUGE-1	BERTScore	FKGL $\downarrow$	DCRS $\downarrow$	CLI $\downarrow$	LENS	SummaC
PLOS	Direct	37.52	85.38	12.59	11.61	12.89	78.05	58.67
	NRLB-Round 1	44.43	86.00	16.55	12.67	14.54	67.21	49.17
	NRLB-Round 2	41.64	85.43	15.21	12.06	13.20	66.68	50.86
GovReport	Direct	38.58	85.02	14.93	12.36	14.97	60.61	67.51
	NRLB-Round 1	45.33	85.84	17.51	12.51	15.09	59.20	45.70
	NRLB-Round 2	44.19	85.48	15.98	11.96	14.23	60.60	47.76

Appendix

Table 9: Comparison between the direct baseline and NRLB across datasets.

\downarrow

indicates that lower values are better for readability-related metrics (FKGL, DCRS, CLI). Best scores per dataset are shown in bold.

		PLOS			GovReport			BillSum			BigPatent
		GPT-4o	Llama-3.1	Qwen3	GPT-4o	Llama-3.1	Qwen3	GPT-4o	Llama-3.1	Qwen3	GPT-4o	Llama-3.1	Qwen3
ROUGE-1	Initial Summary	46.85	47.27	41.62	37.93	46.93	46.19	46.70	45.81	41.08	41.87	44.94	34.21
	Round 1	45.12	44.43	37.55	36.35	45.33	41.41	43.28	42.66	37.08	37.74	40.09	30.55
	Round 2	42.77	41.64	36.03	34.73	44.19	40.04	40.88	39.70	34.62	35.40	36.87	29.18
	Round 3	41.10	39.21	34.91	33.62	43.12	39.21	39.10	37.75	33.95	33.85	34.70	27.97
ROUGE-2	Initial Summary	13.83	14.80	9.50	14.93	20.07	14.87	20.80	22.89	15.68	13.28	17.71	8.38

Appendix

Table 10: Performance of three models (GPT-4o, Llama-3.1-8B-Instruct, Qwen3-8B) on four benchmarks across the initial summary and three refinement rounds. Best scores are shown in bold.

Metric	Stage	PLOS	GovReport	BillSum	BigPatent
Latency (Sec)	Round 0	17.21	16.59	9.82	11.35
	Round 1	32.16	34.05	22.91	21.89
	Round 2	47.39	52.80	35.86	31.89
	Round 3	63.31	72.78	48.72	42.35
API Calls	Round 0	3.00	3.00	3.00	3.00
	Round 1	7.97	7.96	7.96	7.97

Appendix

Table 11: Cost, latency, and API call analysis of NRLB across datasets and refinement rounds.

# K	Metric	PLOS	GovReport	BillSum	BigPatent
1	ROUGE-1	42.94	44.90	42.79	39.91
	FKGL $\downarrow$	16.95	17.92	15.66	16.96
	DCRS $\downarrow$	12.58	12.53	11.87	12.21
	CLI $\downarrow$	14.39	15.43	13.49	14.08
	LENS	65.28	59.29	62.46	61.93
	SummaC	49.65	45.06	33.99	57.29

Appendix

Table 12: Evaluation results across datasets for different checklist sizes (K = 1, 2, 3, 6, 9).

Figure 11: Overview of the web-based human evaluation platform. The interface presents project background, evaluation goals, and scoring criteria.

Figure 12: Example evaluation screen for a medical document. Evaluators compare two summaries, assign scores on three criteria, and select a final preference.

Figure 13: NRLB revision process for a biomedical paper. Reader agents identify readability barriers, the Domain Expert Agent proposes targeted edits, and the Editor Agent integrates them into a simplified summary. Colors, underlines, and highlights indicate unknown terms, missing contexts, and confusing sentences, respectively.

Figure 14: NRLB revision process for a policy document. Reader agents identify readability barriers, the Domain Expert Agent proposes targeted edits, and the Editor Agent integrates them into a simplified summary. Colors, underlines, and highlights indicate unknown terms, missing contexts, and confusing sentences, respectively.

Figure 15: NRLB revision process for a legislative bill. Reader agents identify readability barriers, the Domain Expert Agent proposes targeted edits, and the Editor Agent integrates them into a simplified summary. Colors, underlines, and highlights indicate unknown terms, missing contexts, and confusing sentences, respectively.

Figure 16: NRLB revision process for a patent document. Reader agents identify readability barriers, the Domain Expert Agent proposes targeted edits, and the Editor Agent integrates them into a simplified summary. Colors, underlines, and highlights indicate unknown terms, missing contexts, and confusing sentences, respectively.

Text Simplification

See all 18 papers

ReLay: Personalized LLM-Generated Plain-Language Summaries for Better Understanding, but at What Cost?
May 1, 2026Joey Chan, Yikun Han, Jingyuan Chen +8LLM Personalization Language Model Safety Evaluation
University of Illinois Urbana-Champaign · University of Virginia · University of Colorado Anschutz +4
A Human-in-the-Loop Corpus for LLM-Based Simplification of Scientific Summaries
Jul 28, 2026Kyuri Im, Michael FärberLLM-Assisted Annotation Scientific Communication
ScaDS.AI, Technische Universität Dresden, Dresden, Germany
PlainMedScale: A Corpus of Multi-Level Simplified Medical Texts in German and English
Aug 2, 2026Bruno Brocai, Ilaria Papagno, Mayumi OhtaClinical NLP
Heidelberg University German Linguistics · Heidelberg University Translation Studies · Heidelberg University SFB 1671

May 1, 2026J/K move · Enter open · S save

ReLay: Personalized LLM-Generated Plain-Language Summaries for Better Understanding, but at What Cost?

Joey Chan, Yikun Han, Jingyuan Chen, Samuel Fang+7

University of Illinois Urbana-Champaign · University of Virginia · University of Colorado Anschutz+4

LLM Personalization Language Model Safety Evaluation Hallucination in Language Models Text Summarization

Open paper AskPDF arXiv

Plain Language Summaries (PLS) aim to make research accessible to lay readers, but they are typically written in a one-size-fits-all style that ignores differences in readers' information needs and comprehension. In health contexts, this limitation is particularly important because misunderstanding scientific information can affect real-world decisions. Large language models (LLMs) offer new opportunities for personalizing PLS, but it remains unclear whether personalization helps, which strategies are most effective, and how to balance personalization with safety. We introduce ReLay, a dataset of 300 participant--PLS pairs from 50 lay participants in both static (expert-written) and interactive (LLM-personalized) settings. ReLay includes user characteristics, health information needs, information-seeking behavior, comprehension outcomes, interaction logs, and quality ratings. We use ReLay to evaluate five LLMs across two personalization methods. Personalization improves comprehension and perceived quality, but it also raises the risk of reinforcing user biases and introducing hallucinations, revealing a trade-off between personalization and safety. These findings highlight the need for personalization methods that are both effective and trustworthy for diverse lay audiences.

Abstract

Text Simplification

Explore similar work

ReLay: Personalized LLM-Generated Plain-Language Summaries for Better Understanding, but at What Cost?

A Human-in-the-Loop Corpus for LLM-Based Simplification of Scientific Summaries

PlainMedScale: A Corpus of Multi-Level Simplified Medical Texts in German and English