cs.CLDec 6, 2025
SaveEdit Policy-Aligned Text Simplification
Organizations: The University of Osaka · Institute of Science Tokyo · NTT Inc.
Abstract
Text simplification aims to modify documents and sentences to make them easier to read and understand while preserving their meanings. Different applications require distinct simplification policies, e.g., lexical-oriented simplification and overall rewriting while trading off detail for simplicity. However, achieving such policy-driven control remains an open challenge. In this work, we introduce a simple yet powerful approach that leverages Large Language Model-as-a-Judge (LLM-Judge) to automatically construct policy-aligned training data, removing the need for costly human annotation or parallel corpora. Our method enables the construction of simplification models that adapt to diverse simplification policies at both the sentence and document levels. Remarkably, small-scale open-source LLMs match or surpass GPT-5, as verified by both automatic metrics and edit analysis. The consistent improvements across model families and sizes demonstrate the robustness of our approach.
Figures & tables
| Source | Johan was also the original bassist of the Swedish power metal band HammerFall, but quit before the band ever released a studio album. |
|---|---|
| Sent-Overall | Johan was also the original bassist for HammerFall, a Swedish power metal band. He left before they released their first studio album. |
| Sent-Lexical | Johan was also the original bassist of the Swedish power metal band HammerFall, but he left before they released any studio albums . |
Table 1: Sentence simplifications by our model under two edit policies (Phi-3-mini-3.8B ( Abdin et al., 2024 ) ). We highlight the main simplification edits in each part of the sentence using different colors. Green: Paraphrasing Red: Delete Blue: Structural Change
Figure 1: Document simplifications by our model under the Doc-Lexical policy (Llama3.1-8B ( Grattafiori et al., 2024 ) ). Following Laban et al. (2023) , we visualize the edits applied to the source document.
Figure 2: Overview of our sentence-level simplification; the same framework can be applied to the document level.
| Model | Method | Turk-SARI | ASSET-LENS |
|---|---|---|---|
| GPT-5 | zs | ||
| fs | |||
| Phi3.8B | Vanilla-zs | ||
| Vanilla-fs | |||
| Vanilla-iter1 | |||
| Vanilla-iter2 |
Table 2: Sentence-level results ( bold : the best under each model group, underline : the second best.)
Table 3: Document-level results ( bold : the best under each model group, underline : the second best.)
Table 4: Precision, recall, and F1 scores of edit patterns (Lex: lexical edits, Ela: elaboration edits). Scores in parentheses are exact-matching.
Appendix figures & tables28 assets
Supplementary material from the paper’s appendix.
Appendix
| Model | Method | Turk-SARI | ASSET-LENS |
|---|---|---|---|
| Phi3.8B | PO | ||
| w/o evidence | |||
| Qwen7B | PO | ||
| w/o evidence | |||
| Llama8B | PO | ||
| w/o evidence |
Table 5: Ablation study on the effect of evidence-guided judge design on simplification performance.
| Source: Most of the state lies in an area known as Tornado Alley characterized by frequent interaction between cold and warm air masses producing severe weather. | ||||
| ID | Candidate | Aspect | w/ evidence-guided judge | w/o evidence-guided judge |
| 0 | Most of the state is in Tornado Alley, where cold and warm air masses often interact, causing severe weather. | Lexical | Replaces “lies in an area known as… characterized by” with “is in… where… interact” (active voice simplification) high reward. “Producing” “causing” (direct synonym) high reward. Retains key terms (“Tornado Alley,” “air masses,” “severe weather”) high reward. | Uses “is in Tornado Alley, where cold and warm air masses often interact, causing severe weather.” The word “often” is a good substitute for “frequent.” “Causing” is correct. Lexically accurate. |
| Structural | Flattens the source’s complex participial clause (“characterized by… producing”) into a relative clause (“where… interact, causing”) high reward. | Structure is similar, uses “where” clause. Good. | ||
| 1 | Tornado Alley is located in the state where cold and warm air masses frequently meet and cause severe weather. | Lexical | Reverses subject-relationship (“Tornado Alley is located in the state” vs. original focus on “most of the state lies in Tornado Alley”) moderate penalty. Replaces “interaction” with “meet” (less precise) moderate penalty. | Changes the structure a bit. Says Tornado Alley is located in the state, which might be a slight misrepresentation. Also, “meet” is less precise than “interact.” Maybe not as good lexically. |
| Structural | Inverts subject-object relationship, creating a less intuitive focus (Tornado Alley as subject) high penalty. Adds “located in” (redundant phrasing) moderate penalty. | Changes the subject to Tornado Alley, which might not follow the original structure as closely. Structural change. | ||
| 2 | Most of the state is in Tornado Alley, where cold and warm air masses frequently interact, causing severe weather. | Lexical | Identical to 0 except “often” “frequently” (synonym with no simplification) neutral. | Similar to 0 but uses “frequently interact” instead of “often interact.” “Frequently” is closer to the original’s “frequent.” So maybe better than 0. |
Table 6: Comparison of lexical and structural judge analyses with and without evidence-guided judge design. The analyses are taken from the model output before the final judgments. Final judgments (best/worst) for w/ evidence-guided judge were Lexical: 0/1, Structural: 0/1, Overall: 0/1, while for w/o evidence-guided judge were Lexical: 2/1, Structural: 0/1, Overall: 2/1.
Table 7: Comparison of preference optimization and SFT on our preference data.
Figure 3: Impact of training sample size in sentence simplification. Colored lines show models trained on our preference data; colored triangles show models trained on human-written parallel data. Overlap in triangles is due to the nearly identical LENS scores from Qwen14B and Llama8B.
Figure 4: Impact of training sample size in document simplification.
| Model | Method | L-SARI | O-LENS |
|---|---|---|---|
| Phi3.8B | Vanilla-zs | ||
| PO (Ours) | |||
| Qwen7B | Vanilla-zs | ||
| PO (Ours) | |||
| Llama8B | Vanilla-zs | ||
| PO (Ours) |
Table 8: Out-of-domain evaluation results on SimPA. We report results for two policy types: Sent-Lexical (L) and Sent-Overall (O).
| Model | Turk-SARI | ASSET-LENS |
|---|---|---|
| Gemma3-27B | ||
| OSS-20B | ||
| Aya-Expanse-32B | ||
| Qwen3-32B |
Table 9: Comparison of larger open-source LLMs under the same few-shot prompting setting.
Table 10: Comparison between Gemma3-27B and Qwen3-32B as the judge.
| Model | Sentence-level | Document-level | ||||
|---|---|---|---|---|---|---|
| w/o Parse | w/ Parse | Doc-Lexical | Doc-Elaboration | |||
| Sent-Lexical | Sent-Overall | Sent-Lexical | Sent-Overall | |||
| Qwen2.5-7B | / | / | / | / | / | / |
| Llama3.1-8B | / | / | / | / | / | / |
| Phi4-14B | / | / | / | / | / | / |
| Qwen3-32B | / | / | / | / | / | / |
Table 11: Preference distribution under the Qwen3-32B judge. Each value is reported as preferred/dispreferred ratio. For sentence-level criteria, we also report ablation results without parse-tree information.
| Model | Sentence |
|---|---|
| Source 1 | One side of the armed conflicts is composed mainly of the Sudanese military and the Janjaweed, a Sudanese militia group recruited mostly from the Afro-Arab Abbala tribes of the northern Rizeigat region in Sudan. |
| Phi3B-Lexical | One side of the armed conflicts is mainly made up of the Sudanese military and the Janjaweed, a Sudanese militia group mostly recruited from the Afro-Arab Abbala tribes in the northern Rizeigat region of Sudan. |
| Phi3B-Overall | The armed conflicts involve the Sudanese military and the Janjaweed, a Sudanese militia group mostly made up of the Afro-Arab Abbala tribes from the northern Rizeigat region in Sudan. |
| Qwen7B-Lexical | One side of the armed conflicts is mainly made up of the Sudanese military and the Janjaweed, a Sudanese militia group mostly from the Afro-Arab Abbala tribes in the northern Rizeigat region of Sudan. |
| Qwen7B-Overall | One side of the armed conflicts includes the Sudanese military and the Janjaweed, a militia group from the northern Rizeigat region. |
| Llama8B-Lexical | One side of the conflicts is made up mainly of the Sudanese military and the Janjaweed, a militia group from the Afro-Arab Abbala tribes in northern Sudan. |
Table 12: Examples of Sent-Lexical and Sent-Overall simplifications by our models.
| Model | Document |
|---|---|
| Source 1 | Dame Sally Davies, the Chief Medical Officer for England, has likened the problem of antibiotic resistance to the risks presented by international terrorism. While this might sound like an exaggeration, the threat was actually, if anything, understated. Each year, the global number of deaths to which bacterial resistance contributes far outstrips those caused by terrorist attacks. While it is difficult to track the global impact of antibiotic resistance across all bacterial species, the World Health Organization estimates that for tuberculosis alone multi-drug resistance accounts for more than 150,000 deaths each year. Antibiotic resistance is no longer an abstract risk: this is now a war. In the past hundred years, our expectations of life and survival have changed beyond all recognition… |
| Qwen7B-Lexical | Dame Sally Davies, the Chief Medical Officer for England, has compared the threat of antibiotic resistance to international terrorism. This comparison is not an exaggeration; the problem is even more serious than it might seem. Each year, the number of deaths caused by antibiotic-resistant bacteria is much higher than those caused by terrorist attacks. While it’s hard to track the global impact of antibiotic resistance, the World Health Organization estimates that multi-drug resistant tuberculosis alone causes more than 150,000 deaths annually. Antibiotic resistance is no longer just a potential risk; it’s a real and urgent threat. In the past century, our expectations of life and survival have changed dramatically… |
| Llama8B-Lexical | Dame Sally Davies, the Chief Medical Officer for England, has compared the problem of antibiotic resistance to the risks of international terrorism. While this might seem like an exaggeration, the threat is actually more serious. Each year, the number of deaths caused by bacterial resistance is much higher than those caused by terrorist attacks. The World Health Organization estimates that for tuberculosis alone, multi-drug resistance causes more than 150,000 deaths each year. Antibiotic resistance is no longer just a risk; it’s a real war. In the past hundred years, our expectations of life and survival have changed dramatically… |
| Source 2 | Standing at the edge of space above the deserts of New Mexico, Felix Baumgartner paused slightly. It was a small step away from the capsule, but a 24-mile drop back down to Earth. Our guardian angel will take care of you, said mission control, and the man known as Fearless Felix jumped. Ten heart-stopping minutes later the Austrian landed back on Earth, after reaching speeds of up to 725mph, and breaking three world records, including becoming the worlds first supersonic skydiver by breaking the sound barrier at Mach 1.24. We love you Felix, cheered the control room as his mother, Ava Baumgartner, wept. Baumgartner, who claimed the records for the highest altitude manned balloon flight and the highest altitude skydive, raised his arms in a victory salute to thank his team… |
| Qwen7B-Lexical | Felix Baumgartner stood at the edge of space above the deserts of New Mexico. He was about to jump from a capsule 24 miles above the Earth. Mission control said, "Our guardian angel will take care of you." Baumgartner jumped. Ten minutes later, he landed safely on Earth, having reached speeds of up to 725 mph and breaking three world records, including becoming the first supersonic skydiver by breaking the sound barrier at Mach 1.24. The control room cheered, and Baumgartner’s mother, Ava Baumgartner, cried. He raised his arms in a victory salute to thank his team… |
| Llama8B-Lexical | Felix Baumgartner, known as Fearless Felix, stood at the edge of space above the deserts of New Mexico. He was about to jump from a capsule 24 miles above the Earth. Mission control told him, "Our guardian angel will take care of you." Felix jumped, and 10 minutes later, he landed back on Earth, reaching speeds of up to 725mph. He broke three world records, including becoming the first person to break the sound barrier in free-fall. The control room cheered, and his mother, Ava Baumgartner, cried. Felix raised his arms in a victory salute to thank his team… |
Table 13: Examples of Doc-Lexical simplifications by our models. Documents are truncated for readability.
| Content | ||
| Source | Unveiling a car with a top speed of 25mph, two seats, and no pedals or steering wheel might not make much of an impression at a motor show. But Google, in the US, sent a minor earthquake through the car and taxi industries as it unveiled the latest version of its driverless car. The electrically powered vehicle, which Google has begun testing around its headquarters in Mountain View, California, dispenses with all the normal controls, including foot pedals. Instead, it has a smartphone app that summons it and tells it the destination, and a single STOP button mounted between the two front-facing seats in case the occupants need to override the computer. [truncated] | |
| Simplified | A car with a maximum speed of 25 miles per hour, two seats, and no pedals or steering wheel does not sound very interesting. But Google, in the US, shocked the car and taxi industries when it unveiled the latest version of its driverless car. Google has begun testing the electric car at its headquarters in Mountain View, California. The car does not have the normal controls, such as foot pedals. Instead, it has a smartphone app that calls it and tells it the destination, and a STOP button between the two seats in case the passengers need to override the computer. [truncated] | |
| Edit Category | Lexical Generic | Location 1 |
| “source”: “Unveiling a car with a top … at a motor show.” | ||
| “simplified”: “A car with a maximum … very interesting.” | ||
| “edit”: “A” “Unveiling a”; “maximum” “top”; … [truncated] | ||
Table 14: Anchor example for Doc-Lexical policy used in the prompt for LLM-Judge. This example is a held-out sample from the OSE dataset. For readability, we truncate each field because the documents and edit annotations are very long.
Figure 5: Prompts used for sentence-level simplification generation (from Wu and Arase (2026) ).
Figure 6: Prompts used for document-level simplification generation. The instructions are designed to reflect the reference editing behavior observed in the OSE and Newsela corpora.
Figure 7: Sentence-level evaluation guidelines provided to the judge LLM
Figure 8: Example 1 for sentence-level LLM-Judge
Figure 9: Example 2 for sentence-level LLM-Judge
Figure 10: Example 3 for sentence-level LLM-Judge
Figure 11: Document-level evaluation guidelines provided to the judge LLM
Figure 12: Prompts used for parsing
Figure 13: System messages for sentence alignment
Figure 14: One-shot example for sentence alignment
Figure 15: Guidelines in sentence-level iterative feedback. Contents are from Figure 7 , 8 , 9 , and 10 .
Figure 16: Three-shot examples of Sent-Lexical used for LLM-Judge in the iterative feedback setting.
Figure 17: 3-shot examples of Sent-Overall used for LLM-Judge in the iterative feedback setting
Figure 18: Prompts used for self-correction in the iterative feedback setting at the sentence level
Figure 19: Guidelines in document-level iterative feedback.
Figure 20: Prompts used for self-correction in the iterative feedback setting at the document level