Overview

Language model robustness asks whether a model gives dependable answers when meaning-preserving changes, noise, misleading context, or adversarial inputs challenge it. Evaluations use paraphrases, reordered answer choices, typos, multilingual and dialectal text, noisy retrieval, and multi-turn interactions, often measuring per-example consistency as well as accuracy. Approaches include perturbation and adversarial training, consistency-based fine-tuning, calibration and selective prediction, and symbolic verification or mechanistic analysis of model internals. A central tension is that strong performance on standard benchmarks often hides fragile behavior, while fixes that improve stability can reduce clean-task utility, over-refuse, or fail to transfer across tasks and deployment settings.

Latest papers

CardsList

521 papers35 papers in the last four weeks, down 43% on the four weeks before. 0.3% of all new papers.

  1. Reliability is a user-specific property of large language models

    Nov 18, 2026Xingyu ZhaoLLM ReliabilityLLM Evaluation

  2. Evaluating Cultural Reasoning and Robustness in LLMs for Spanish Political Discourse

    Nov 16, 2026Jorge Valverde-Rebaza, Nathan Garcia-Larios, Karime Ruvalcaba-Perez +3Sentiment AnalysisLanguage Model Robustness

  3. HireWatch: Evaluating LLM Compliance with U.S. Employment Law Under Contextual Pressure

    Nov 16, 2026Huy Nghiem, Maria Kostylew, Noam KoltLegal Language Model EvaluationAI Law

  4. Model Immunization Beyond Condition Numbers: The Importance of Plateau Regions

    Nov 16, 2026Quoc Nguyen, Trung Le, Dinh Phung +1Language Model RobustnessAdversarial Robustness of Language Models

  5. Disentangling Spurious Correlations: How IRM Handles Diverse Shortcuts in Text Classification

    Nov 12, 2026Paul Grunenwald, Benoît Crabbé, Guillaume Wisniewski +1Shortcut Learning in Language ModelsOOD Generalization

  6. Shortcuts or Memorisation? Revisiting Data Corruption in Natural Language Understanding with Large Language Models

    Nov 12, 2026Aarne Talman, Stergios ChatzikyriakidisLanguage Model RobustnessShortcut Learning

  7. Breakdown of Autoregressive Multi-Step Reasoning Under Concentrated vs. Diffuse Attention Head Dropout

    Nov 10, 2026Karol StruniawskiLanguage Model RobustnessLarge Reasoning Models

  8. Language model capabilities fail in a shared, predictable order under compression and perturbation

    Nov 8, 2026Zhixiang Wang, Ulas BagciLanguage Model RobustnessLanguage Model Performance Prediction

  9. A Layer-wise Study of How Style Rewriting Breaks the Calibration of Zero-Shot LLM Fake-News Detectors

    Nov 8, 2026Prabhjot Singh, Praney Goyal, Raksha SharmaLanguage Model RobustnessLanguage Model Calibration

  10. On the Effectiveness and Limitations of Open-Weight LLMs for Low-Resource Offensive Language Detection in Danish

    Nov 6, 2026Elnur Adl Zarabi, Amit Kumar, Adi Singh +3Hate Speech DetectionOpen-Weight Language Models

  11. The Intrusion Pathway: How Non-Evidential Context Degrades Language Model Performance

    Nov 5, 2026Dikshant Kukreja, Gautam Gupta, Avni Mittal +8Language Model RobustnessMechanistic Interpretability

  12. Beyond accuracy: what AI assurance needs to establish in public services

    Nov 4, 2026Louise YorkeAI AssuranceLLM Governance

  13. Mind Control of LLMs via Closed-Loop Stimulation: Out-of-Distribution Robustness

    Nov 4, 2026Kasra Fallah, Rudramani Singha, Hannah Ghanei +3Language Model SteeringLanguage Model Robustness

  14. Voice-Native LLMs Hear Frustration but Do Not Listen to It

    Nov 2, 2026Hanjun Luo, Yuhan Lu, Hanan SalamAudio-Language Model EvaluationSpeech Language Models

  15. EMBERS: Empirically Motivated Robustness Testing of Deep Research Systems via Defect-Guided Fuzzing

    Nov 1, 2026Shuochuan Li, Zan Wang, Dong Wang +5Fuzz TestingLanguage Model Robustness

  16. Robust LLM Data Selection via Content-Invariant Gradient Attribution

    Oct 27, 2026Jia Gu, Liang Pang, Huawei ShenInstruction-Tuning Data SelectionGradient-Based Attribution

  17. High Scores, Hidden Failures: Evaluating Codebook-Based LLM Content Analysis

    Oct 26, 2026Elena Senger, Yuri Campbell, Rob van der Goot +1LLM-Assisted AnnotationLLM Reliability

  18. Beyond Belief: Evaluating and Mitigating LLM Susceptibility to Misinformation

    Oct 25, 2026Emran Kanaan, Jiho Lee, Ibrahim Odat +2Language Model Safety EvaluationLanguage Model Robustness

  19. Counterfactual Test-Time Augmentation for Shortcut Removal in Language Models

    Oct 24, 2026Katie Matton, Rosalind W. Picard, John Guttag +1Shortcut Learning in Language ModelsTest-Time Augmentation