Language Model Generation Evaluation

Follow this topic Ask about this topic

Momentum

30 papers in the last four weeks, up 100% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28

Latest papers 209

All topics

Document-Level Text Simplification in Estonian Using Large Language Models
Oct 7, 2026Meeri-Ly Muru, Eduard BarbuLow-Resource Language Processing Language Model Generation Evaluation
National Library of Estonia · Institute of Computer Science, University of Tartu Tartu, Estonia
I would rather quit NLP than read another paper like this: The rise of antithesis in NLP papers
Oct 7, 2026Olga Zamaraeva, Adrián Gude, Roi Santos-Ríos +1Language Model Generation Evaluation Language Model Post-Training
Universidade da Coruña, CITIC
What pass@k Cannot Measure: Evaluating Diversity and Capability Retention after Post-Training
Oct 5, 2026Subham Rath, Raj Dandekar, Rajat Dandekar +1LLM Evaluation Language Model Generation Evaluation
Accenture Strategy & Consulting · Vizuara AI Labs (vizuara.com) · Vizuara AI Labs (vizuara.ai)
SOL: Measuring Gaps between Text Distributions by Double Sliced Wasserstein Metrics
Oct 5, 2026Gregor Kornhardt, Moritz Piening, Jannis Chemseddine +1Non-Autoregressive Text Generation Language Model Generation Evaluation
Institute of Mathematics, TU Berlin, Berlin, Germany · Department of Computer Science, University of Göttingen, Göttingen, Germany
Keyword Harnesses Fail Open: A Cheap Diagnostic Ladder for Tool-Use Claims in Small Language Models
Oct 1, 2026Juan S. SantillanaSupervised Fine-Tuning Tool-Use Evaluation
Independent Researcher
Generalization Is Stability, Not Accuracy: Multi-Axis Evaluation of LLMs
Oct 1, 2026Nagham Omar, Mahmoud Jabarin, Maya Rozenshtein +3Prompt Sensitivity LLM Evaluation
Technion – Israel Institute of Technology
Beyond Leaderboards: Tokenomics of Agentic Small Language Model Ensembles
Oct 1, 2026Alexei N. Skurikhin, Emily M. Taylor, Nathan A. DeBardelebenLLM Inference Efficiency Language Model Generation Evaluation
Los Alamos National Laboratory Los Alamos, New Mexico, USA
LLM-as-a-Judge for Low-Resource Languages: Adapting Ragas and Comparative Ranking for Romanian
Sep 30, 2026Claudiu Creanga, Liviu P. DinuLLM Evaluation LLM-as-a-Judge
Interdisciplinary School of Doctoral Studies, University of Bucharest, Romania · HLT Research Center, University of Bucharest, Romania · Faculty of Mathematics and Computer Science, University of Bucharest, Romania
Mubric: Mutation Testing-Guided Rubric Generation for LLM Evaluation
Sep 29, 2026Jiayuxuan Yang, Jie M. Zhang, Yiling Lou +1Language Model Generation Evaluation Rubric-Based Evaluation
Tsinghua University · King’s College London · University of Illinois Urbana-Champaign
Dating the Model: Hidden Dates in System Prompts Affect LLM Evaluation
Sep 29, 2026Mario Sanz-Guerrero, Minh Duc Bui, Manuel Mager +1LLM Evaluation Language Model Generation Evaluation
Johannes Gutenberg University Mainz, Germany · Universidad Iberoamericana, Mexico · University of Colorado Boulder, USA
Generating Edit-Inducing Questions for AI Research Manuscripts
Sep 29, 2026Sebastian Joseph, Zichao Wang, Jennifer Healey +3Text Generation Language Model Generation Evaluation
The University of Texas at Austin · Adobe Research
Coherence-Aware Distributional Evaluation of Open-Ended Text Generation
Sep 28, 2026Jinnuo Liu, Junhao Zhu, Weifeng Jiang +2Language Model Generation Evaluation Generative AI Evaluation
New York University · Center for Data Science, NYU Shanghai · Georgia Institute of Technology
Simple Diffusion Language Models Are More Effective Few-Step Generators Than Reported
Sep 27, 2026Hasan Amin, Ming Yin, Rajiv KhannaMasked Diffusion Models Language Model Generation Evaluation
Purdue University
SlopBench: How Well Can We Rank Language Models by Slop? A Multi-Domain Benchmark of Repetitive AI Writing
Sep 27, 2026Dhruv Roongta, Harsha Gaddipati, Anh Tuan HuynhLLM Evaluation Language Model Generation Evaluation
Slashy, San Francisco, California, USA
MISHAP-Bench: A Hallucination Benchmark for Large Audio-Language Models
Sep 27, 2026Zhi Wen Soi, Giulio Segalini, Jian-Jia Chen +1Audio-Language Model Evaluation Audio-Language Model Hallucination Detection
University of Neuchâtel · TU Dortmund University · RWTH Aachen University +1
Self-Designed Evaluators and Warm Memory for Long-Horizon Agents
Sep 27, 2026Saeid Asgari, Emre Kiciman, Leonardo de Oliveira Nunes +1Language Model Generation Evaluation Long-Horizon Agent Evaluation
Microsoft
Knowing Is Not Choosing: What Explicit Verification Adds Beyond Generative Preference
Sep 27, 2026Yilong Li, Chengpo Yan, Aayan Arish +1LLM Evaluation LLM Answer Verification
Department of Computer Science University of Wisconsin-Madison Madison, WI 53703, USA
Style, Not Self: Surface Cues Explain Zero-Shot Code Attribution by Large Language Models
Sep 24, 2026Ehsan Barkhordar, Surendrabikram ThapaLLM Evaluation Language Model Generation Evaluation
Koç University · Virginia Tech
Two Emojis of Difference: What Multilingual Affective Generation Benchmarks Actually Measure
Sep 24, 2026Fardeen Sadab, Adib SakhawatMultilingual Language Model Evaluation Language Model Generation Evaluation
Department of Computer Science and Engineering Islamic University of Technology, Dhaka, Bangladesh
Likelihood Ranking doesn't Scale Like Prompting in LLMs
Sep 24, 2026Alessandro Bondielli, Lucia Passaro, Davide Bacciu +1LLM Evaluation Language Model Generation Evaluation
CoLingLab, Department of Philology, Literature and Linguistics, University of Pisa · Department of Computer Science, University of Pisa
Reasoning Instructions Can Break Answer Decoding in Vision--Language Models
Sep 24, 2026Zeyan Li, Siyuan Qiu, Jianfeng XuVLM Evaluation Language Model Generation Evaluation
Shanghai Jiao Tong University
Reference-Based Analysis of Coherence and Diversity in Open-Ended Text Generation
Sep 23, 2026Esteban Garcés AriasOpen-Ended Generation Language Model Generation Evaluation
Department of Statistics, LMU Munich Munich Center for Machine Learning (MCML)
Speculative Evaluation of Stochastic LLMs
Sep 23, 2026Qianli Shen, Xiang Li, Ruomeng Ding +3LLM Evaluation Language Model Generation Evaluation
Alibaba Group · National University of Singapore · University of North Carolina at Chapel Hill
Recognized but Not Produced: A Generation Benchmark for Culturally Specific Kinship Terms
Sep 22, 2026Sahil Pardasani, Madhusudan SinghMultilingual Language Model Evaluation Language Model Generation Evaluation
Blockchain Data Intelligence Lab The Pennsylvania State University University Park, PA, USA
JEV-as-a-Judge: Accept When Confident, Escalate When Unsure
Sep 22, 2026Yubo Li, Yidi Miao, Ramayya Krishnan +1LLM-as-a-Judge Language Model Generation Evaluation
Carnegie Mellon University
A Semiotics-Aware Framework for Evaluating Fidelity and Coverage in Natural Language Generation
Sep 22, 2026Lorenzo Zangari, Davide PiccaLLM Evaluation Language Model Generation Evaluation
University of Lausanne Lausanne, Switzerland
Efficient Cost-Aware LLM Evaluation via Bayesian Bandit Gittins Indices
Sep 22, 2026Qian Xie, Yueli He, Nairen CaoMulti-Armed Bandits LLM Evaluation
Cornell University · Columbia University · New York University +1
Made in Hungary: Comments on the performance of generative language models
Sep 16, 2026Mátyás Osváth, Enikő Héja, Noémi Ligeti-NagyLLM Evaluation Language Model Generation Evaluation
Institute for Language Technologies and Applied Linguistics ELTE Research Centre for Linguistics Budapest, Hungary
A Calibrated Instrument for Measuring How Inference Optimizations Affect Output Quality
Sep 16, 2026Jerry KaplanLLM Quantization LLM Evaluation
Adjunct Lecturer, Computer Science and Masters in International Policy, Stanford University
EviScope: Paired Counterfactual Evidence Diagnostics for Faithful and Efficient Grounded Language Models
Sep 15, 2026Suryadeep Singh DeswalCounterfactual Evaluation LLM Grounding
Indian Institute of Technology Roorkee Roorkee, India

Oct 7, 2026, cs.CLJ/K move · Enter open · S save

Document-Level Text Simplification in Estonian Using Large Language Models

Meeri-Ly Muru, Eduard Barbu

National Library of Estonia · Institute of Computer Science, University of Tartu Tartu, Estonia

Low-Resource Language Processing Language Model Generation Evaluation Multilingual Language Model Evaluation

Open paper AskPDF arXiv

Document-level text simplification involves transformations that go beyond sentence-internal edits, addressing discourse coherence, anaphora resolution, and cross-paragraph consistency. Despite advances in sentence-level simplification for high-resource languages, document-level simplification in morphologically rich, low-resource languages such as Estonian remains largely unexplored. This study presents a comprehensive evaluation of five state-of-the-art multilingual large language models (LLMs) for document-level simplification in Estonian. Three prompting strategies are examined: single-pass generation, pipeline-based modular agents, and guideline-augmented pipelines. The evaluation framework integrates automatic metrics assessing readability, semantic preservation, and discourse coherence, alongside a structured manual annotation protocol. The findings indicate that Gemini-2.0 and LLaMA-3.3 produce outputs with near-native fluency and strong meaning preservation, whereas other models display notable grammatical and semantic limitations. This work contributes novel document-level coherence metrics, evidence-based prompting strategies, and publicly available resources for reproducibility.

Momentum

Related topics

Latest papers 209