Reinforcement Learning for improving Large Language Models' Catalan text simplification capabilities
Organizations: Universitat Pompeu Fabra · Barcelona Supercomputing Center
Abstract
Although automatic text simplification (ATS) is critical for accessibility, its progress has not matched the rapid evolution of broader natural language processing techniques. This paper investigates the application of reinforcement learning (RL) to improve the quality of ATS for low-resource languages using Large Language Models (LLMs). The paper introduces a novel reward function, designed to guide LLMs toward a targeted simplification style with Group Relative Policy Optimization (GRPO), that combines the SARI metric with specific penalty components. The effectiveness of GRPO with this reward function is motivated and demonstrated by post-training IberianLLM-7B-Instruct on the ASSET dataset. After post-training on the English ASSET, the model's ATS performance improves on two curated Catalan benchmarks while also successfully suppressing previously observed negative behaviors. Cross-lingual transfer learning is explored by translating ASSET into Catalan and Spanish and post-training the model on each version, but these fail to show a significant improvement on the out-of-domain benchmark.
Figures & tables
| Benchmark | Original | CA | ES | EN |
|---|---|---|---|---|
| ASSET | 47.75 | 50.42 | 49.77 | 50.41 |
| iDEM | 43.31 | 42.82 | 42.47 | 44.50 |
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
| Comparison | Diff. | 95% CI |
|---|---|---|
| CA vs. Original | ||
| ES vs. Original | ||
| EN vs. Original* | ||
| ES vs. CA | ||
| EN vs. CA* | ||
| EN vs. ES* |
Low-Resource Language Processing
Low-resource language processing builds and evaluates language technologies for languages with limited data, tools, or digital representation.
See all 548 papers