cs.CLNov 25, 2025

Structured Prompts Improve Evaluation of Language Models

Authors: Asad Aali, Muhammad Ahmed Mohsin, Vasiliki Bikia, Arnav Singhvi, Richard Gaus, Suhana Bedi, Hejie Cui, Miguel Fuentes, 

Organizations: Stanford University

Abstract

As language models (LMs) are increasingly adopted across domains, high-quality benchmarking frameworks are essential for guiding deployment decisions. However, frameworks such as Holistic Evaluation of Language Models (HELM) typically evaluate models under a single static prompt configuration, even though model behavior depends strongly on prompt choice. As a result, reported scores can reflect prompt choice as much as model capability. Declarative prompting frameworks, such as DSPy, offer a scalable way to operationalize this evaluation protocol by evaluating models under structured prompting strategies rather than static prompts. Here, we conduct a reproducible DSPy+HELM benchmarking study and quantify how prompt choice impacts benchmark outcomes. Using five prompting methods, we evaluate four frontier and two open-source LMs across seven benchmarks against HELM baseline scores and find that prompt choice can materially impact leaderboards. Structured prompting improves performance (by 6% on average), alters comparisons (leaderboard rankings shift on 5/75/7 benchmarks); chain-of-thought provides the strongest cost-performance tradeoff while more advanced optimizers offer smaller additional gains. Beyond the known intuition that prompts affect outputs, our benchmarking study integrates structured prompting into an established evaluation framework and quantifies how prompt choice alone impacts leaderboards. We open-source (i) DSPy+HELM Evaluation (https://github.com/stanford-crfm/helm/pull/3893) and (ii) Prompt Optimization Pipeline (https://github.com/StanfordMIMI/dspy-helm).

Figures & tables

Explore similar work

CardsList
  1. PEEM: Prompt Engineering Evaluation Metrics for Interpretable Joint Evaluation of Prompts and Responses

    Mar 11, 2026Minki Hong, Eunsoo Lee, Sohyun Park +1LLM PromptingLLM Evaluation

  2. Brittlebench: Quantifying LLM robustness via prompt sensitivity

    Feb 27, 2026Angelika Romanou, Mark Ibrahim, Candace Ross +8Prompt SensitivityLLM Evaluation