Overview

Prompt sensitivity studies how models respond to changes such as paraphrases, word order, option labels, examples, schemas, and contextual framing, including when the intended meaning stays the same. Researchers measure output flips, score variance, consistency, and ranking changes across controlled prompt variants, and use prompt aggregation, robustness training, and internal-state analyses to explain or reduce these effects. The issue spans language, vision-language, clinical, safety, and benchmark settings, where prompt choices can affect both correctness and behavior. A central tension is that greater consistency does not guarantee correctness or safety, and methods that help in one model or task may fail in another.

Latest papers

CardsList

121 papers9 papers in the last four weeks, down 40% on the four weeks before. 0.1% of all new papers.

  1. Algorithmic Collusion by Large Language Models

    Mar 31, 2024Sara Fish, Yannai A. Gonczarowski, Ran I. ShorrerPrompt SensitivityAlgorithmic Collusion