Prompt Sensitivity

Follow this topic Ask about this topic

Momentum

9 papers in the last four weeks, down 10% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 68

All topics

As Easy as Rocket Science: Assessing the Ability of Large Language Models to Interpret Negation in Figurative Language
Jun 17, 2026Jasmine Owers, Edwin Simpson, Martha LewisFigurative Language Understanding Prompt Sensitivity
Intelligent Systems Lab University of Bristol UK · ILLC University of Amsterdam The Netherlands
Demystifying Variance in Circuit Discovery of LLMs
Jun 15, 2026Frank Zhengqing Wu, Francesco Tonin, Volkan CevherPrompt Sensitivity Circuit Discovery
Laboratory for Information and Inference Systems (LIONS) École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland · Laboratory for Informationand Inference Systems (LIONS) École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland
When Large Language Models Fail in Healthcare: Evaluating Sensitivity to Prompt Variations
Jun 5, 2026Mahdi AlkaeedPrompt Sensitivity Adversarial Attacks on LLMs
Department of Computer Science and Engineering, Doha, Qatar
Measuring the sensitivity of LLM-based structured extraction to prompt, model, and schema choices in clinical discharge summaries
Jun 4, 2026Martin MurinPrompt Sensitivity Clinical Information Extraction
DryLabz GmbH, Basel, Switzerland
Discourse-Role Labels as Presentation-Time Variables for Context Use in Language Models
Jun 2, 2026Jianguo Zhu, Xiangmei Li, Wenjie LiuPrompt Sensitivity LLM Evaluation
School of Cybersecurity (Xin Gu Industrial College), Chengdu University of Information, Chengdu, 610225, China · Sichuan Provincial Key Laboratory of Cyberspace Security, Chengdu, 610225, China · Advanced Cryptography and System Security Key Laboratory of Sichuan Province, Chengdu, 610225, China +1
The Invisible Lottery: How Subtle Cues Steer Algorithm Choice in LLM Code Generation
Jun 2, 2026Akanksha Narula, Mofasshara Binte Rafique, Laurent BindschaedlerPrompt Sensitivity LLM Prompting
Max Planck Institute for Software Systems (MPI-SWS), Saarbrücken, Germany · Independent Researcher, Switzerland.
The Unsampled Truth: Quantifying Prompt Artifacts in LM Psychometrics
Jun 2, 2026Nils Schwager, Christoph Hau, Simon Münker +1Prompt Sensitivity LLM Prompting
Trier University, Trier, Germany
Lingo_Research_Group at SemEval-2026 Task 9: Evaluating Prompt Variants for Polarization Detection
Jun 2, 2026Pritam Kadasi, Anuj Tiwari, Mayank SinghPrompt Sensitivity Text Classification
Lingo Research Group, Indian Institute of Technology Gandhinagar · Noida Institute of Engineering and Technology 2ML Collective
Toxic HallucinAItions: Perturbing Prompts and Tracing LLM Circuits
May 29, 2026Soorya Ram Shimgekar, Agam Goyal, Amruta Parulekar +6Prompt Sensitivity LLM Interpretability
University of Illinois Urbana-Champaign · Nimblemind
Same Patient, Different Words, Different Diagnosis? Evaluating Semantic Stability in Clinical LLMs
May 28, 2026Mahdi Alkaeed, Adnan Qayyum, Nabeel Abo Kashreef +2Prompt Sensitivity Clinical Language Model Evaluation
Department of Computer Science and Engineering, College of Engineering, Qatar University, Doha, Qatar · College of Science and Engineering, Hamad Bin Khalifa University (HBKU), Doha, Qatar · Primary Health Care Corporation (PHCC), Doha, Qatar +1
Minimal Prompt Perturbations Lead to Code Vulnerabilities: Prompt Fragility and Hidden-State Signals in Coding LLMs
May 28, 2026Alexander Sternfeld, Andrei Kucharavy, Ljiljana DolamicPrompt Sensitivity Adversarial Attacks on LLMs
IEM, HES-SO, Le Foyer, Techno-Pôle 1, Sierre, Switzerland · Cyber-Defence Campus, armasuisse Science and Technology, Thun, Switzerland
Spurious Prompts: Can Irrelevant Prompts Steer Large Language Models?
May 28, 2026Pawel Batorski, Abtin Pourhadi, Jerzy Sarosiek +2Prompt Sensitivity LLM Prompting
1Heinrich Heine University Düsseldorf · 2Jagiellonian University · 3IDEAS Research Institute
When prompt perturbations break your A/B test: A valid statistical test for generative surveying
May 26, 2026Hayden Helm, Carey PriebePrompt Sensitivity A/b Testing
Helivan · Johns Hopkins University
PromptAudit: Auditing Prompt Sensitivity in LLM-Based Vulnerability Detection
May 22, 2026Steffen J. Camarato, Yahya Hmaiti, Mandana Ghadamian +1Prompt Sensitivity LLM Auditing
University of Central Florida Orlando, Florida, USA
Paraphrase Brittleness in Production Retrieval-Augmented Commercial Recommendation: Reproducibility Below the Rerun-Stability Baseline
May 22, 2026Will Jack, Noah Lehman, Keller Maloney +1Prompt Sensitivity LLM Evaluation
Unusual
One prompt is not enough: Instruction Sensitivity Undermines Embedding Model Evaluation
May 21, 2026Yevhen Kostiuk, Kenneth EnevoldsenPrompt Sensitivity Text Embedding Evaluation
Aarhus University
Medical Context Distorts Decisions in Clinical Vision Language Models
May 17, 2026David Restrepo, Ira Ktena, Maria Vakalopoulou +2VLM Evaluation VLM Robustness
1MICS, CentraleSup´elec - Universit´e Paris-Saclay, France · 2Cancer Data Science Unit, IHU PRISM, National Institute in Precision Oncology, University Paris-Saclay, CentraleSupelec, Gustave Roussy, INSERM, Gif-sur-Yvette · 3Ellison Institute of Technology (EIT), Oxford, UK +1
Targeted Tests for LLM Reasoning: An Audit-Constrained Protocol
May 12, 2026Hongmin LiPrompt Sensitivity Language Model Robustness
School of Life Science and Technology, Institute of Science Tokyo · Department of Computational Biology and Medical Sciences, Graduate School of Frontier Sciences, The University of Tokyo
BiAxisAudit: A Novel Framework to Evaluate LLM Bias Across Prompt Sensitivity and Response-Layer Divergence
May 9, 2026Jialing Gan, Junhao Dong, Songze LiPrompt Sensitivity LLM Auditing
Southeast University, China · Nanyang Technological University, Singapore
Paraphrase-Induced Output-Mode Collapse: When LLMs Break Character Under Semantically Equivalent Inputs
May 6, 2026Aofan Liu, Jingxiang MengPrompt Sensitivity LLM Evaluation
1Peking University. · University of Chicago.
Prompt Framing Distorts Count-Based Evaluation of LLM Error Detection: Evidence from Numeric Anchoring
May 3, 2026Dekun YangPrompt Sensitivity LLM Evaluation
Zhejiang University
Format Sensitivity Index: Token-Controlled Prompt Wrapper Robustness and Schema Compliance in LLM Benchmarking
May 2, 2026Deep Pankajbhai MehtaPrompt Sensitivity LLM Evaluation
Adobe Inc.
Impact of Task Phrasing on Presumptions in Large Language Models
May 1, 2026Kenneth J. K. OngPrompt Sensitivity LLM Prompting
AI.DA STC · ST Engineering, Singapore
Prompt-Induced Score Variance in Zero-Shot Binary Vision-Language Safety Classification
May 1, 2026Charles Weng, Dingwen Li, Alexander MartinVLM Evaluation Zero-Shot Learning
Johns Hopkins University · Independent
Investigating Linguistic Steering: An Analysis of Adjectival Effects Across Large Language Model Architectures
Apr 28, 2026Lars MalmqvistPrompt Sensitivity Language Model Steering
Research and Implementation
Green Shielding: A User-Centric Approach Towards Trustworthy AI
Apr 27, 2026Aaron J. Li, Nicolas Sanchez, Hao Huang +8Prompt Sensitivity Clinical Language Model Evaluation
University of California, Berkeley · University of Melbourne · University of California, San Francisco +2
Reliability Auditing for Downstream LLM tasks in Psychiatry: LLM-Generated Hospitalization Risk Scores
Apr 23, 2026Shevya Panda, Shinjini Bose, Ananya JoshiPrompt Sensitivity LLM Auditing
Johns Hopkins University Baltimore, USA
Shared Lexical Task Representations Explain Behavioral Variability In LLMs
Apr 23, 2026Zhuonan Yang, Jacob Xiaochen Li, Francisco Piedrahita Velez +5Prompt Sensitivity LLM Prompting
Brown University · MIT · Northeastern
Less Is More: Cognitive Load and the Single-Prompt Ceiling in LLM Mathematical Reasoning
Apr 20, 2026Manuel Israel CazaresPrompt Sensitivity LLM Prompting
Bytepro AI Mazatlán, Sinaloa, Mexico
Understanding the Prompt Sensitivity
Apr 20, 2026Yang Liu, Chenhui ChuPrompt Sensitivity LLM Prompting
Kyoto University

Jun 17, 2026, cs.CLJ/K move · Enter open · S save

As Easy as Rocket Science: Assessing the Ability of Large Language Models to Interpret Negation in Figurative Language

Jasmine Owers, Edwin Simpson, Martha Lewis

Intelligent Systems Lab University of Bristol UK · ILLC University of Amsterdam The Netherlands

Figurative Language Understanding Prompt Sensitivity LLM Evaluation

Open paper AskPDF arXiv

Figurative language and negation are two areas that challenge current language models, however, both are widely used throughout written and spoken language. Large language models (LLMs) are also widely used in everyday contexts where they cannot necessarily be tuned for a specific dataset. It is therefore essential to understand the ability of LLMs to correctly interpret text that includes both negation and figurative language. To investigate this, we develop a set of new annotations to an existing dataset of figurative language, and test a range of language models on the dataset. We find that the combination of negation and figurativeness can present a particular challenge, and that performance overall and across different negation types is particularly dependent on the prompt style used.

Momentum

Related topics

Latest papers 68