Prompt Robustness

Follow this topic Ask about this topic

Prompt robustness studies how changes in wording, formatting, paraphrases, typos, or other input variations affect model answers, safety decisions, and actions. Approaches include consistency and perturbation-based fine-tuning, adversarial and wrapper-based safety training, prompt optimization, and defenses that distinguish harmful intent from harmless changes in form. Evaluations compare multiple prompt variants across tasks such as question answering, retrieval, tool use, safety, and image segmentation rather than relying on one carefully chosen prompt. Results show that robustness depends on the model, task, and type of change, while training gains may not transfer to new prompts or domains and can trade off against clean-task performance or over-refusal.

Latest papers

18 papers1 paper in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

Lost in the Request: How Communication Variation Disrupts Retrieval and Action in Email Agents
Oct 2, 2026Feng Chen, Ritam Dutt, Atnaz Taheri +1LLM Agent Reliability LLM Agent Evaluation
Evaluating the Effects of Prompt Perturbation on Bias and Hallucination in Large Language Models
Sep 18, 2026Mamehgol Yousefi, Ahmad Shahi, Mos Sharifi +3LLM Evaluation Hallucination in Language Models
School of Product Design, Faculty of Engineering, University of Canterbury, Christchurch, New Zealand · Unitec Institute of Technology, Auckland, New Zealand · AgResearch Ltd., Lincoln Research Centre, Lincoln, New Zealand
Prompt-Robust Language Models: Which Training Strategies Work?
Sep 1, 2026Frederic Sadrieh, Michal ŠtefánikPrompt Sensitivity LLM Fine-Tuning
Center for Information and Language Processing, LMU Munich · R&D Centre for Large Language Models, National Institute of Informatics, Japan
When Adaptation Hurts: Connecting Representational Drift to OOD Failures in MedSAM Fine-Tuning
Aug 21, 2026Marko Haralović, Sounic Akkaraju, Carlo Baretta +2Vision Foundation Model Adaptation Medical Imaging Foundation Models
University of Zagreb, Faculty of Electrical Engineering and Computing, Zagreb, Croatia · University of Twente, Enschede, The Netherlands
Refusing Intent, Not Form: Wrapper-Based Intent-Group Supervision for LLM Safety
Aug 13, 2026Ping Wu, Haibo Tong, Feifei Zhao +7LLM Safety Alignment LLM Refusal Behavior
BrainCog Lab, Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · Beijing Key Laboratory of Safe AI and Superalignment +1
Improving Generalization Robustness of Multimodal RLVR
Aug 9, 2026Pengfei Zhou, Zhiwei Tang, Xiaopeng Peng +11Multimodal Robustness Multimodal Large Language Models
1National University of Singapore, HPC-AI Lab · 2DAMO Academy, Alibaba Group · 3Hupan Lab +7
BASIS: Breach-Aware Selective Prompt Injection Shielding with Prefill Attention Probes
Aug 8, 2026Laiqiao Qin, Tianqing Zhu, Longxiang Gao +1Prompt Injection Defense Prompt Injection Attacks on LLMs
City University of Macau, Macau · Qilu University of Technology
Prompt Robustness Is Task-Dependent: Comparing Objective and Belief-Style Questions in LLM Evaluation
Jul 6, 2026Sadia Kamal, Arefa Patwary, Anthony Marchiafava +2LLM Evaluation Language Model Generation Evaluation
Oklahoma State University · University of North Texas
MPSelectTune: Prompt-type Selection for Fine-tuning improves Concept Unlearning in LLMs
Jul 4, 2026Shubhadip Nag, Srinjoy Das, Agniva Saha +5LLM Fine-Tuning Machine Unlearning
*IIT Kharagpur · †MPI-SWS · ‡HPE Labs
Domain-Conditioned Safety in Frontier Computer-Using Agents: A 793-Episode Browser Benchmark, a Coding-Domain Cross-Reference, and a Reproducibility Audit of Recent Red-Teaming
Jun 3, 2026Nicholas SabanComputer-Use Agents AI Agent Safety
Patronus AI · University of California, Berkeley
Harnessing non-adversarial robustness in large language models
May 28, 2026Qinghua Zhou, Ellina Aleshina, Andrey Lovyagin +6LLM Fine-Tuning Neural Network Robustness
International Joint Laboratory of AI for Industry, QUST, Qingdao, China · King’s College London, London, UK · Applied AI Institute, Moscow, Russia +2
One prompt is not enough: Instruction Sensitivity Undermines Embedding Model Evaluation
May 21, 2026Yevhen Kostiuk, Kenneth EnevoldsenPrompt Sensitivity Text Embedding Evaluation
Aarhus University
What Single-Prompt Accuracy Misses: A Multi-Variant Reliability Audit of Language Models
May 3, 2026Ranit Karmakar, Jayita ChatterjeeLLM Evaluation Language Model Calibration
Harvard University, Boston, MA 02115 · Boston, MA 02134
Where Do Prompt Perturbations Break Generation? A Segment-Level View of Robustness in LoRA-Tuned Language Models
May 2, 2026Zhuoyun Li, Boxuan Wang, Jinwei Hu +6Neural Network Robustness Noise Robustness
School of Computer Science and Informatics, University of Liverpool, UK
Beyond the Answer: Decoding the Behavior of LLMs as Scientific Reasoners
Mar 30, 2026Rohan Pandey, Eric Ye, Michael LiLLM Prompting LLM Interpretability
Carnegie Mellon University
POaaS: Minimal-Edit Prompt Optimization as a Service to Lift Accuracy and Cut Hallucinations on On-Device sLLMs
Mar 17, 2026Jungwoo Shim, Dae Won Kim, Sun Wook Kim +4Automatic Prompt Optimization LLM Hallucination Mitigation
Electronics and Telecommunications Research Institute, Republic of Korea
Prompting with the human-touch: evaluating model-sensitivity of foundation models for musculoskeletal CT segmentation
Mar 11, 2026Caroline Magg, Maaike A. ter Wee, Johannes G. G. Dobbe +4Medical Image Segmentation Medical Imaging Foundation Models
Quantitative Healthcare Analysis (QurAI) Group, University of Amsterdam, Science Park 900, Amsterdam, 1098 XH, The Netherlands · Department Biomedical Engineering and Physics, Amsterdam UMC, Meibergdreef 9, Amsterdam, 1105 AZ, The Netherlands · Department Orthopaedics, Amsterdam UMC, Meibergdreef 9, Amsterdam, 1105 AZ, The Netherlands
Flip-Flop Consistency: Unsupervised Training for Robustness to Prompt Perturbations in LLMs
Oct 16, 2025Parsa Hejabi, Elnaz Rahmati, Alireza S. Ziabari +1Unsupervised Learning Language Model Robustness
University of Southern California

Oct 2, 2026J/K move · Enter open · S save

Lost in the Request: How Communication Variation Disrupts Retrieval and Action in Email Agents

Feng Chen, Ritam Dutt, Atnaz Taheri, Alex Williams

LLM Agent Reliability LLM Agent Evaluation Prompt Robustness LLM Agents RAG Robustness

Open paper AskPDF arXiv

An email assistant should not complete less work simply because a user phrases the same request differently. Yet most benchmarks test each task with only one canonical request, leaving this form of robustness largely unmeasured. We test whether email assistants remain reliable when the requested information, available evidence, and expected outcome stay fixed, but the communication style or English variety changes. We construct validated variants along five communication-style axes and four rule-based dialect conditions, and evaluate them on three benchmarks: a retrieval-augmented generation (RAG) pipeline and two tool-using agents. Indirect requests reduce performance on all three benchmarks, while formal requests reduce performance on both agentic benchmarks. Examining the systems more closely shows that these failures have different causes. Verbose requests mainly hurt a lexical retriever by making the relevant email harder to find. By contrast, indirect and dialect variants remain harmful even when the relevant email is retrieved. In the agentic setting, indirect and formal requests mainly cause the agents to omit required actions, not to take more unsupported actions. These results show that a successful response is not enough to establish robustness: evaluations should vary how requests are expressed and separately measure whether agents complete the requested work.