Language Model Robustness

Ask about this topic

Overview

Language model robustness asks whether a model gives dependable answers when meaning-preserving changes, noise, misleading context, or adversarial inputs challenge it. Evaluations use paraphrases, reordered answer choices, typos, multilingual and dialectal text, noisy retrieval, and multi-turn interactions, often measuring per-example consistency as well as accuracy. Approaches include perturbation and adversarial training, consistency-based fine-tuning, calibration and selective prediction, and symbolic verification or mechanistic analysis of model internals. A central tension is that strong performance on standard benchmarks often hides fragile behavior, while fixes that improve stability can reduce clean-task utility, over-refuse, or fail to transfer across tasks and deployment settings.

Latest papers

CardsList

495 papers53 papers in the last four weeks, down 4% on the four weeks before. 0.3% of all new papers.

  1. Flip-Flop Consistency: Unsupervised Training for Robustness to Prompt Perturbations in LLMs

    Oct 16, 2025Parsa Hejabi, Elnaz Rahmati, Alireza S. Ziabari +1Unsupervised LearningLanguage Model Robustness

    University of Southern California

  2. VCB Bench: An Evaluation Benchmark for Audio-Grounded Large Language Model Conversational Agents

    Oct 13, 2025Jiliang Hu, Wenfu Wang, Zuchao Li +6Audio-Language Model EvaluationVoice Agent Evaluation

    School of Artificial Intelligence, Wuhan University, Wuhan, China · Tencent AI Lab, Beijing, China

  3. Black-Box Detection of LLM-Generated Text Using Generalized Jensen-Shannon Divergence

    Oct 8, 2025Shuangyi Chen, Ashish KhistiAI-Generated Text DetectionLanguage Model Robustness

    Department of Electrical and Computer Engineering, University of Toronto, Toronto, Canada.

  4. Are LLMs Reliable Rankers? Rank Manipulation via Two-Stage Token Optimization

    Oct 8, 2025Tiancheng Xing, Jerry Li, Yixuan Du +1Adversarial Attacks on LLMsLLM Reranking

    National University of Singapore · University of Southern California · Georgetown University +1

  5. Robustness assessment of large audio language models in multiple-choice evaluation

    Oct 6, 2025Fernando López, Santosh Kesiraju, Jordi LuqueAudio-Language Model EvaluationAudio QA

    Scientific Research, Telef´onica Innovaci´on Digital, Spain · Brno University of Technology, Czech Republic

  6. VietBinoculars: A Zero-Shot Approach for Detecting Vietnamese LLM-Generated Text

    Sep 30, 2025Trieu Hai Nguyen, Sivaswamy AkileshAI-Generated Text DetectionLanguage Model Robustness

    Faculty of Information Technology, Nha Trang University, 02 Nguyen Dinh Chieu Street, North Nha Trang 57100, Vietnam · Faculty of Business Administration, Swiss School of Business and Management Geneva, 12 Avenue des Morgines, Geneva 1213, Switzerland

  7. From Construction to Injection: Edit-Based Fingerprints for Large Language Models

    Sep 3, 2025Yue Li, Xin Yi, Dongsheng Shi +3Language Model FingerprintingLLM Security

    East China Normal University · Hasso Plattner Institute/University of Potsdam

  8. Enhancing Speech Large Language Models through Reinforced Behavior Alignment

    Aug 25, 2025Yansong Liu, Yuan LiuSpeech Language ModelsLanguage Model Robustness

    Xi’an Jiaotong-Liverpool University · The Chinese University of Hong Kong

  9. Data Security in Large Language Models: Risks, Defense, and Directions

    Aug 4, 2025Kang Chen, Xiuze Zhou, Yuanhui Yu +4Adversarial Attacks on LLMsLLM Security

    School of Computer Engineering, Jimei University, Xiamen, 361021, China · The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, 511453, China · School of Informatics, Xiamen University, Xiamen, 361102, China +1

  10. Measuring Intent Comprehension in LLMs

    Jun 19, 2025Nadav Kunievsky, James A. EvansPrompt SensitivityLLM Evaluation

    Knowlesdge Lab, University of Chicago, Chicago, Illinois, USA · Knowledge Lab, Data Science Institute, Department of Sociology, University of Chicago, Chicago, Illinois, USA.

  11. LLMs are Brittle to Simple Code Transformations: Introducing CETBench - A Benchmark for Code-Equivalence Checking

    Jun 4, 2025Neeva Oza, Ishaan Govil, Parul Gupta +3LLM EvaluationLanguage Model Robustness

    Indian Institute of Technology Delhi · IBM Research AI

  12. RAFP: Identifying LLM Lineages via Rare-Region Fingerprints

    May 19, 2025Yun-Yun Tsai, Jia Hao Liang, Chuan Guo +2Language Model FingerprintingLanguage Model Robustness

    Columbia University / Meta · New York / Menlo Park, USA · MIT +5

  13. Unified Enhancement of the Generalization and Robustness of Language Models via Bi-Stage Optimization

    Mar 19, 2025Yudao Sun, Juan Yin, Juan Zhao +3Adversarial TrainingNeural Network Generalization

    Department of New Networks, Peng Cheng Laboratory, Shenzhen 518055, China · Department of Industrial Systems Engineering and Management, National University of Singapore, Singapore 117576 · School of Data Science and Artificial Intelligence, Chang’an University, Shaanxi, 710064, China

  14. Homogeneity Bias in Open-Weight LLMs Is Robust to Decoding Hyperparameters

    Jan 4, 2025Messi H. J. LeeSocial Bias in Language ModelsOpen-Weight Language Models

    Independent Researcher · Seoul, Republic of Korea

  15. BA-LoRA: Bias-Alleviating Low-Rank Adaptation to Mitigate Catastrophic Inheritance in Large Language Models

    Aug 8, 2024Yupeng Chang, Yi Chang, Yuan WuSocial Bias in Language ModelsLow-Rank Adaptation

    School of Artificial Intelligence, Jilin University · Key Laboratory of Symbolic Computation and Knowledge Engineering, Jilin University · International Center of Future Science, Jilin University