Language Model Robustness

Ask about this topic

Overview

Language model robustness asks whether a model gives dependable answers when meaning-preserving changes, noise, misleading context, or adversarial inputs challenge it. Evaluations use paraphrases, reordered answer choices, typos, multilingual and dialectal text, noisy retrieval, and multi-turn interactions, often measuring per-example consistency as well as accuracy. Approaches include perturbation and adversarial training, consistency-based fine-tuning, calibration and selective prediction, and symbolic verification or mechanistic analysis of model internals. A central tension is that strong performance on standard benchmarks often hides fragile behavior, while fixes that improve stability can reduce clean-task utility, over-refuse, or fail to transfer across tasks and deployment settings.

Latest papers

CardsList

495 papers53 papers in the last four weeks, down 4% on the four weeks before. 0.3% of all new papers.

  1. Models Know Their Shortcuts: Deployment-Time Shortcut Mitigation

    Apr 14, 2026Jiayi Li, Shijie Tang, Gün Kaynar +2Shortcut LearningLanguage Model Robustness

    Ray and Stephanie Lane Computational Biology Department, School of Computer Science, Carnegie Mellon University, Pittsburgh, PA

  2. Leveraging Weighted Syntactic and Semantic Context Assessment Summary (wSSAS) Towards Text Categorization Using LLMs

    Apr 13, 2026Shreeya Verma Kathuria, Nitin Mayande, Sharookh Daruwalla +2Text ClusteringText Classification

    Tellagence Inc. · Villanova School of Business, Villanova University

  3. A Robust Evaluation of Probe Robustness: Lessons for Reliable OOD Uncertainty Quantification

    Apr 13, 2026Joe Stacey, Hadas Orgad, Kentaro Inui +2LLM EvaluationUncertainty Quantification

    University of Sheffield · Kempner Institute at Harvard University · MBZUAI +2

  4. MIXAR: Scaling Autoregressive Pixel-based Language Models to Multiple Languages and Scripts

    Apr 13, 2026Chen Hu, Yintao Tai, Antonio Vergari +2Multilingual Language ModelsAutoregressive Language Modeling

    School of Informatics, University of Edinburgh

  5. Toward Robust In-Context Learning: Leveraging Out-of-distribution Proxies for Target Inaccessible Demonstration Retrieval

    Apr 13, 2026Hao Xu, Rite Bo, Fausto Giunchiglia +2In-Context LearningLanguage Model Robustness

    College of Computer Science and Technology, Jilin University, China · Department of Information Engineering and Computer Science, University of Trento, Italy

  6. How Robust Are Large Language Models for Clinical Numeracy? An Empirical Study on Numerical Reasoning Abilities in Clinical Contexts

    Apr 13, 2026Minh-Vuong Nguyen, Fatemeh Shiri, Zhuang Li +1Numerical Reasoning in Language ModelsClinical Language Model Evaluation

    School of Computing Technologies, RMIT University, Australia

  7. CARO: Chain-of-Analogy Reasoning Optimization for Robust Content Moderation

    Apr 12, 2026Bingzhe Wu, Haotian Lu, Yuchen MouAnalogical Reasoning for Language ModelsContent Moderation

    Shenzhen University · Tsinghua University · National University of Singapore

  8. VeriSim: A Configurable Framework for Stress-Testing Medical AI Under Patient Communication Noise

    Apr 12, 2026Sina Mansouri, Mohit Marvania, Vibhavari Ashok Shihorkar +5HealthcareLLM Reliability

    Department of Computer Science, George Mason University · Department of Health Administration and Policy, College of Public Health, George Mason University · Department of Science, George Mason University

  9. Anchored Sliding Window: Toward Robust and Imperceptible Linguistic Steganography

    Apr 10, 2026Ruiyi Yan, Shiao Meng, Yugo MurawakiLanguage Model SteganographyLanguage Model Robustness

    Graduate School of Informatics, Kyoto University · School of Software, Tsinghua University

  10. PerMix-RLVR: Preserving Persona Expressivity under Verifiable-Reward Alignment

    Apr 10, 2026Jihwan Oh, Soowon Oh, Murad Aghazada +3Personalized Language Model AlignmentLanguage Model Robustness

    KAIST AI · Samsung Advanced Institute of Technology

  11. Bridging SFT and RL: Dynamic Policy Optimization for Robust Reasoning

    Apr 10, 2026Taojie Zhu, Dongyang Xu, Ding Zou +4LLM Fine-TuningRL for Language Model Reasoning

    Shenzhen International Graduate School, Tsinghua University · Intelligent System Department, Zhongxing Telecom Equipment (ZTE) · Institute of Advanced Interdisciplinary Studies, Chongqing University of Posts and Telecommunications

  12. Activation Steering for Aligned Open-ended Generation without Sacrificing Coherence

    Apr 9, 2026Niklas Herbster, Martin Zborowski, Alberto Tosato +2LLM Safety AlignmentLanguage Model Robustness

    Tara Research · Technical University of Munich · Mila Quebec AI Institute

  13. Guardian-as-an-Advisor: Advancing Next-Generation Guardian Models for Trustworthy LLMs

    Apr 8, 2026Yue Huang, Haomin Zhuang, Jiayi Ye +6LLM GuardrailsLLM Safety Alignment

    University of Notre Dame · University of California, Los Angeles · MIT-IBM Watson AI Lab +1

  14. To Adapt or not to Adapt, Rethinking the Value of Medical Knowledge-Aware Large Language Models

    Apr 8, 2026Ane G. Domingo-Aldama, Iker De La Iglesia, Maitane Urruela +2Medical LLMsDomain Adaptation for LLMs

  15. DIA-HARM: Dialectal Disparities in Harmful Content Detection Across 50 English Dialects

    Apr 7, 2026Jason Lucas, Matt Murtagh, Ali Al-Lawati +3Fake News DetectionDialect Bias in Language Models

    The Pennsylvania State University, USA · Trinity College Dublin, Ireland · MIT Lincoln Laboratory, USA

  16. Individual and Combined Effects of English as a Second Language and Typos on LLM Performance

    Apr 6, 2026Serena Liu, Yutong Yang, Prisha Sheth +9Language Model RobustnessLLM Evaluation

    Harvard University, Boston, MA 02138, US

  17. Empirical Characterization of Rationale Stability Under Controlled Perturbations for Explainable Pattern Recognition

    Apr 6, 2026Abu Noman Md Sakib, Zhensen Wang, Merjulah Roby +1Explanation StabilityExplainability Evaluation

    Department of Computer Science, The University of Texas at San Antonio, San Antonio, TX, 78249, USA · Department of Mechanical, Aerospace, and Industrial Engineering, The University of Texas at San Antonio, San Antonio, TX, 78249, USA

  18. Shorter, but Still Trustworthy? An Empirical Study of Chain-of-Thought Compression

    Apr 5, 2026Lingjie Zeng, Xiaofan Chen, Yanbo Wang +1CoT CompressionLLM Compression

    Mohamed bin Zayed University of Artificial Intelligence

  19. Parallel LLM Reasoning for Bias-Resilient, Robust Conceptual Abstraction

    Apr 4, 2026Aisvarya Adeseye, Jouni Isoaho, Adeyemi AdeseyeLong-Context Reasoning in Language ModelsParallel Reasoning in Language Models

    University of Turku, Turku, Finland · Brilloconnetz Partners avoin yhtiö, Turku, Finland

  20. Mitigating LLM biases toward spurious social contexts using direct preference optimization

    Apr 2, 2026Hyunji Nam, Dorottya DemszkyLanguage Model RobustnessLanguage Model Bias Evaluation

    Stanford University

  21. Where Does Robustness Live? Neuron-Guided Adaptation for Retrieval-Augmented Language Models

    Apr 2, 2026Jae O Lee, Jaemin Kim, Sumyeong Ahn +1Retrieval-Augmented GenerationFine-Tuning

    1Hanyang University · 2KENTECH

  22. Fragile Reasoning: A Mechanistic Analysis of LLM Sensitivity to Meaning-Preserving Perturbations

    Apr 2, 2026Shou-Tzu Han, Rodrigue Rizk, KC SantoshLanguage Model RobustnessLLM Reasoning

    Department of Computer Science, University of South Dakota

  23. Contrast Matters: Understanding Robustness of In-Context Fine-Tuning to Target-Context Relatedness

    Apr 2, 2026Deeptanshu Malu, Deevyanshu Malu, Aditya Nemiwal +1In-Context Example SelectionLLM Fine-Tuning

    IIT Bombay

  24. Reasoning Shift: How Context Silently Shortens LLM Reasoning

    Apr 1, 2026Gleb Rodionov, Roman Garipov, George YakushevLanguage Model RobustnessLLM Context Management

    Yandex · HSE University

  25. Structured Intent as a Protocol-Like Communication Layer: Cross-Model Robustness, Framework Comparison, and the Weak-Model Compensation Effect

    Mar 31, 2026Peng GangStructured PromptingHuman-AI Interaction

    Huizhou Lateni AI Technology Co., Ltd., Huizhou, China · Huizhou University, Huizhou, China

  26. Beyond Idealized Patients: Evaluating LLMs under Challenging Patient Behaviors in Medical Consultations

    Mar 31, 2026Yahan Li, Xinyi Jie, Wanjia Ruan +5HealthcareClinical Reasoning

    University of Southern California

  27. Distributionally Robust Token Optimization in RLHF

    Mar 27, 2026Yeping Jin, Jiaming Hu, Ioannis Ch. PaschalidisLanguage Model RobustnessReasoning Consistency in Language Models

    Department of System Engineering Boston University, Boston, MA 02215, USA · Department of Math & Statistics Boston University, Boston, MA 02215, USA

  28. When Chain-of-Thought Backfires: Evaluating Prompt Sensitivity in Medical Language Models

    Mar 26, 2026Binesh Sadanandan, Vahid BehzadanPrompt SensitivityLLM Prompting

    SAIL Lab, University of New Haven, West Haven, CT 06516, USA

  29. Robust Reasoning Benchmark

    Mar 26, 2026Pavel Golikov, Evgenii Opryshko, Gennady Pekhimenko +1Language Model RobustnessLLM Evaluation

    University of Toronto · Vector Institute

  30. On Integrating Resilience and Human Oversight into LLM-Assisted Modeling Workflows for Digital Twins

    Mar 26, 2026Lekshmi P, Neha KaranjkarDigital TwinsHuman-in-the-Loop AI

    Indian Institute of Technology Goa Goa, India