Language Model Robustness

Language model robustness asks whether a model gives dependable answers when meaning-preserving changes, noise, misleading context, or adversarial inputs challenge it. Evaluations use paraphrases, reordered answer choices, typos, multilingual and dialectal text, noisy retrieval, and multi-turn interactions, often measuring per-example consistency as well as accuracy. Approaches include perturbation and adversarial training, consistency-based fine-tuning, calibration and selective prediction, and symbolic verification or mechanistic analysis of model internals. A central tension is that strong performance on standard benchmarks often hides fragile behavior, while fixes that improve stability can reduce clean-task utility, over-refuse, or fail to transfer across tasks and deployment settings.