Taming CATS: Controllable Automatic Text Simplification through Instruction Fine-Tuning with Control Tokens
Authors: Hanna Hubarava, Yingqiang Gao
Organizations: Department of Computational Linguistics, University of Zurich, Switzerland · Department of Clinical Research, University of Bern, Switzerland
Controllable Automatic Text Simplification (CATS) produces user-tailored outputs, yet controllability is often treated as a decoding problem and evaluated with metrics that are not reflective to the measure of control. We observe that controllability in ATS is significantly constrained by data and evaluation. To this end, we introduce a domain-agnostic CATS framework based on instruction fine-tuning with discrete control tokens, steering open-source models to target readability levels and compression rates. Across three model families with different model sizes (Llama, Mistral, Qwen; 1-14B) and four domains (medicine, public administration, news, encyclopedic text), we find that smaller models (1-3B) can be competitive, but reliable controllability strongly depends on whether the training data encodes sufficient variation in the target attribute. Readability control (FKGL, ARI, Dale-Chall) is learned consistently, whereas compression control underperforms due to limited signal variability in the existing corpora. We further show that standard simplification and similarity metrics are insufficient for measuring control, motivating error-based measures for target-output alignment. Finally, our sampling and stratification experiments demonstrate that naive splits can introduce distributional mismatch that undermines both training and evaluation.
Figures & tables
Figure 1: Mean control-attribute values across models and datasets. Fine-tuned models tend to generate text that is simpler than the source but more complex than the target. Sentence-aligned datasets ( Med-EASi, SimPA, WikiLarge ) show very little compression signal (around 1.0).
Figure 2: Both SimPA and Newsela show larger distribution shifts by readability than by length transformations. Text-aligned Newsela shows greater distribution shift than the sentence-aligned SimPA.
Figure 4: WikiLarge . Global sampling with stratification by readability and length, measured in terms of KS, EMD and JSD. Stratification by N chars shows smallest divergence across all metrics.
SARI ↑
LENS ↑
MAE ↓
1B
8B
1B
8B
1B
8B
Med-EASi FKGL
43.68
36.20
59.03
68.95
3.37
4.89
42.64 -1.04
43.99 +7.77
57.33 -1.70
59.83 -9.12
3.96 +0.51
2.91 ?
Med-EASi char
32.38
36.76
21.23
62.89
1.23
0.37
Table 2: IFT of 1B and 8B Llama models improves SARI and MAE over the non-fine-tuned baseline, with stronger gains for larger models and compression-based controls, whereas LENS does not consistently improve and in some cases decreases after fine-tuning.
Table 3: Scaling experiment with a broader parameter size range. Rows in green represent results on MED-EASi , whereas rows in blue represent results on SimPA . Top value per row and metric is in bold, second-best value is underscored. (a) - (c) FKGL, (d) - (f) char compression.
Figure 6: Dataset: SimPA . Control attribute: FKGL. Scaling does not always boost performance. However, we observe strong positive correlation between SARI and COMET, and a strong negative correlation between SARI/COMET and error-based metrics across most datasets and control attributes.
Data
Ctrl.
SARI ↑
LENS ↑
MAE ↓
Llama
Mistral
Qwen
Llama
Mistral
Qwen
Llama
Mistral
Qwen
1B
8B
3B
7B
1.7B
8B
1B
8B
3B
7B
1.7B
8B
1B
8B
3B
7B
1.7B
8B
Med-EASi
base
ARI
42.47
43.47
44.78
50.90
50.32
50.55
56.89
60.17
50.20
59.28
58.74
59.33
4.39
3.35
8.19
3.30
3.63
2.86
base
DC
42.04
44.03
43.85
52.18
50.29
50.49
55.21
59.64
50.89
57.32
57.14
57.62
1.54
1.27
1.66
1.29
1.39
1.16
base
char
42.43
45.25
44.86
52.03
52.09
51.91
55.74
59.83
51.25
57.42
60.54
59.94
0.39
0.21
0.70
0.28
0.18
0.19
Table 4: Main experiment results show strong performance of the fine-tuned Qwen models closely followed by the larger Mistral. “DC” stands for Dale-Chall. “Mono” refers to the monotonically filtered subsets of the respective dataset; “base” refers to its full version. “Word” and “char” denote word- and character-level compression ratio, respectively. Higher is better for SARI and LENS; lower is better for MAE.
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 7: Model performance on Newsela .
Figure 8: Model performance on Med-EASi .
Figure 9: Model performance on SimPA .
Figure 10: Model performance on WikiLarge .
Figure 11: Dataset: SimPA
Figure 16: Pearson correlation heatmaps analyzing metric correlation, aggregated across all models. Dataset: Newsela .
Figure 17: Pearson correlation heatmaps analyzing metric correlation, aggregated across all models. Dataset: Med-EASi .
Figure 18: Pearson correlation heatmaps analyzing metric correlation, aggregated across all models. Dataset: SimPA .
Figure 19: Pearson correlation heatmaps analyzing metric correlation, aggregated across all models. Dataset: WikiLarge .
DC
Text
Source
9.5
“In public saunas it is more common that men and women go to the sauna separately, although people of both sexes may sometimes bathe together, for example in student clubs. Still, saunas are not associated with sex and sexuality”.
Reference
5.7
“Some saunas are only for men , or only for women”.
Llama-3.2-1B-Instruct
11.1
“The sauna is not associated with sex and sexuality”.
Llama-3.1-8B-Instruct
7.3
“The sauna is a place where people go to relax”.
Ministral-3b-instruct
9.3
“In public saunas it is more common that men and women go to the sauna separately, although people of both sexes may occasionally bathe together”.
Mistral-7B-Instruct-v0.1
9.8
“In public saunas it is more common that men and women go to the sauna separately, although people of both sexes may sometimes bathe together, for example in student clubs”.
Appendix
Table 5: Dataset: WikiLarge . Control Attribute: Dale-Chall. Instance ID: wikilarge_ori_252807. In this example, we see how different models focus on different information in the source text, resulting in very different simplifications. Further, reference simplification is much shorter than the source text and omits essential information. This is an example of a poor reference text, with lower adequacy and fluency than the automatically generated simplifications by various models.
ARI
Text
Source
20.1
“Symptoms include a sense of fullness, heaviness, aching, cramps, pain, tiredness, and paresthesias in the legs; these symptoms worsen with standing or walking and are relieved by rest and elevation”.
Reference
15.2
“The swelling may not cause any symptoms, but some people feel fullness, heaviness, aching, cramps, pain, tiredness, and tingling in the legs”.
Llama-3.2-1B-Instruct
11.8
“The symptoms include a sense of fullness, heaviness, aching, cramps, pain, tiredness, and numbness in the legs. These symptoms worsen with standing or walking and are relieved by rest and elevation”.
Llama-3.1-8B-Instruct
19.6
“The symptoms include a sense of fullness, heaviness, aching, cramps, pain, tiredness, and numbness in the legs; these symptoms worsen with standing or walking and are relieved by rest and elevation”.
Ministral-3b-instruct
10.7
“Symptoms include a sense of fullness, heavyness, aching, cramps, and tiredness. They worsen with standing or walking and are relieved by rest and elevation”.
Mistral-7B-Instruct-v0.1
19.5
“Symptoms include a sense of fullness, heaviness, aching, cramps, pain, tiredness, and tingling in the legs; these symptoms worsen with standing or walking and are relieved by rest and elevation”.
Appendix
Table 6: Dataset: Med-EASi . Control Attribute: ARI. Instance ID: medeasi_000434. Simplifications generated by the model are very similar, further highlighting the effect of sentence splitting on the ARI score. Llama-3.2-1B-Instruct and Ministral-3b-instruct split the long sentence into two and drastically reduce the ARI score without any further transformations.
Figure 20: Formatted prompt examples.
Hyperparameters
Shared Hyperparams
batch size
4
grad. accumulation steps
4
cumulative batch size
16
weight decay
0.01
warmup steps
30
max epochs
3
Appendix
Table 7: Left: training hyperparameters selected for smaller ( ≤ 4B) models, fine-tuned without PEFT. Right: training hyperparameters selected for larger ( > 4B) models, fine-tuned with PEFT.
Although automatic text simplification (ATS) is critical for accessibility, its progress has not matched the rapid evolution of broader natural language processing techniques. This paper investigates the application of reinforcement learning (RL) to improve the quality of ATS for low-resource languages using Large Language Models (LLMs). The paper introduces a novel reward function, designed to guide LLMs toward a targeted simplification style with Group Relative Policy Optimization (GRPO), that combines the SARI metric with specific penalty components. The effectiveness of GRPO with this reward function is motivated and demonstrated by post-training IberianLLM-7B-Instruct on the ASSET dataset. After post-training on the English ASSET, the model's ATS performance improves on two curated Catalan benchmarks while also successfully suppressing previously observed negative behaviors. Cross-lingual transfer learning is explored by translating ASSET into Catalan and Spanish and post-training the model on each version, but these fail to show a significant improvement on the out-of-domain benchmark.
Arnau Ayguadé Domingo, Stefan Bott, Horacio Saggion
Universitat Pompeu Fabra · Barcelona Supercomputing Center
Text simplification supports second language (L2) learning by providing comprehensible input, consistent with the Input Hypothesis. However, constructing personalized parallel corpora is costly, while existing large language model (LLM)-based readability control methods rely on pre-labeled sentence corpora and primarily target English. We propose Re-RIGHT, a unified reinforcement learning framework for adaptive multilingual text simplification without parallel corpus supervision. We first show that prompting-based lexical simplification at target proficiency levels (CEFR, JLPT, TOPIK, and HSK) performs poorly at easier levels and for non-English languages, even with state-of-the-art LLMs such as GPT-5.2 and Gemini 2.5. To address this, we collect 43K vocabulary-level data across four languages (English, Japanese, Korean, and Chinese) and train a compact 4B policy model using Re-RIGHT, which integrates three reward modules: vocabulary coverage, semantic preservation, and coherence. Compared to the stronger LLM baselines, Re-RIGHT achieves higher lexical coverage at target proficiency levels while maintaining original meaning and fluency.
Text simplification aims to make complex texts easier to understand while preserving their original meaning. Recent large language models can perform simplification through prompting, but it remains unclear whether adding explicit linguistic information about sentence complexity to the prompt improves their outputs. We investigate this question for multilingual sentence-level Easy-to-Read simplification in Catalan, Spanish, and Italian. Using Qwen3-8B, we compare a baseline prompt, a gold-cue prompt enriched with gold linguistic cues, and a predicted-cue prompt enriched with automatically predicted cues. We evaluate the outputs using SARI, BLEU, chrF, and BERTScore, and complement this evaluation with a manual qualitative analysis. Predicted-cue prompting obtains the best overall scores across all four metrics, although the gains over the baseline are small. Gold-cue prompting does not consistently improve over the baseline, and results vary across languages. These findings indicate that cue-based prompting can influence multilingual Easy-to-Read simplification, but its benefits are modest, metric-dependent, and language-dependent.