Structured Prompts Improve Evaluation of Language Models
Authors: Asad Aali, Muhammad Ahmed Mohsin, Vasiliki Bikia, Arnav Singhvi, Richard Gaus, Suhana Bedi, Hejie Cui, Miguel Fuentes, Alyssa Unell, Yifan Mai, Jordan Cahoon, Michael Pfeffer, Roxana Daneshjou, Sanmi Koyejo, Emily Alsentzer, Christopher Potts, Nigam H. Shah, Akshay S. Chaudhari
As language models (LMs) are increasingly adopted across domains, high-quality benchmarking frameworks are essential for guiding deployment decisions. However, frameworks such as Holistic Evaluation of Language Models (HELM) typically evaluate models under a single static prompt configuration, even though model behavior depends strongly on prompt choice. As a result, reported scores can reflect prompt choice as much as model capability. Declarative prompting frameworks, such as DSPy, offer a scalable way to operationalize this evaluation protocol by evaluating models under structured prompting strategies rather than static prompts. Here, we conduct a reproducible DSPy+HELM benchmarking study and quantify how prompt choice impacts benchmark outcomes. Using five prompting methods, we evaluate four frontier and two open-source LMs across seven benchmarks against HELM baseline scores and find that prompt choice can materially impact leaderboards. Structured prompting improves performance (by 6% on average), alters comparisons (leaderboard rankings shift on 5/7 benchmarks); chain-of-thought provides the strongest cost-performance tradeoff while more advanced optimizers offer smaller additional gains. Beyond the known intuition that prompts affect outputs, our benchmarking study integrates structured prompting into an established evaluation framework and quantifies how prompt choice alone impacts leaderboards. We open-source (i) DSPy+HELM Evaluation (https://github.com/stanford-crfm/helm/pull/3893) and (ii) Prompt Optimization Pipeline (https://github.com/StanfordMIMI/dspy-helm).
Figures & tables
Figure 1: Pipeline overview. (a) DSPy takes HELM’s baseline prompt and produces structured prompt variants. (b) HELM evaluates models under each prompt variant. With structured prompting, evaluation improves and benchmark conclusions change materially.
Figure 2: Structured prompting methods evaluated in our study ( Zero-Shot CoT , BFRS , MIPROv2 ). Each box corresponds to a method showing how prompts differ.
Benchmark
Input → Output
Task
Samples
MMLU-Pro
Reasoning Question → Answer
Multi-Task Reasoning
1,000
GPQA
Graduate Question → Answer
Graduate-Level QA
446
GSM8K
Math Problem → Solution
Numeric Problem-Solving
1,000
MedCalc-Bench
Patient Note → Computed Value
Computational Reasoning
1,000
Medec
Medical Narrative → Errors
Error Classification
597
HeadQA
Medical Question → Answer
USMLE-Style QA
1,000
Table 1: HELM benchmarks evaluated in our study. Columns summarize the number of test samples and tasks spanning reasoning, knowledge QA, problem-solving, and error-classification tasks across diverse domains.
Model
API Identifier
Release
Context
Reasoning
Open-Source
Claude 3.7 Sonnet
anthropic/claude-3-7-sonnet-20250219
02/19/2025
200k
✗
✗
Gemini 2.0 Flash
google/gemini-2.0-flash-001
02/01/2025
1000k
✗
✗
GPT 4o
openai/gpt-4o-2024-05-13
05/13/2024
128k
✗
✗
o3 Mini
openai/o3-mini-2025-01-31
01/31/2025
200k
✓
✗
Llama 3.3 70B
meta-llama/Llama-3.3-70B-Instruct
12/06/2024
128k
✗
✓
Qwen3 4B
Qwen/Qwen3-4B-Instruct-2507
08/06/2025
256k
✗
✓
Table 2: Evaluated language models with their API identifier, release, context, reasoning mode, and availability.
Prompting Method
Claude 3.7 Sonnet
Gemini 2.0 Flash
GPT 4o
o3 Mini
Llama 3.3 70B
Qwen3 4B
HELM Baseline
64.8% ± 3.3
61.4% ± 3.4
61.0% ± 3.3
70.9% ± 3.3
57.2% ± 3.4
47.8% ± 3.3
Zero-Shot Predict
65.1% ± 3.4
61.7% ± 3.4
59.7% ± 3.3
73.2% ± 3.2
59.2% ± 3.4
49.3% ± 3.4
Zero-Shot CoT ★
69.4% ± 3.2
66.2% ± 3.4
65.7% ± 3.3
72.7% ± 3.2
64.0% ± 3.4
59.0% ± 3.4
BFRS ★
69.3% ± 3.3
66.2% ± 3.3
65.9% ± 3.2
73.1% ± 3.2
64.2% ± 3.3
60.3% ± 3.4
MIPROv2 ★
69.8% ± 3.3
66.2% ± 3.3
65.3% ± 3.3
73.1% ± 3.1
62.1% ± 3.4
60.4% ± 3.4
Highest − Baseline ★
+5.0%
+4.8%
+4.9%
+2.3%
+7.0%
+12.6%
Table 3: HELM leaderboard (macro-averaged) across six language models and five prompting methods. Green marks the highest performance among evaluated prompting methods. Entries are reported as mean ± 95% bootstrap confidence interval. ★ indicates a statistically significant pooled improvement over the HELM baseline.
Benchmark
Prompting Method
Claude 3.7 Sonnet
Gemini 2.0 Flash
GPT 4o
o3 Mini
Llama 3.3 70B
Qwen3 4B
MMLU-Pro
HELM Baseline
76.3% ± 2.7
66.1% ± 3.0
62.2% ± 3.0
77.1% ± 3.1
64.7% ± 3.0
44.9% ± 3.1
Zero-Shot Predict
77.7% ± 2.7
70.3% ± 2.8
60.7% ± 3.0
78.4% ± 3.1
62.7% ± 3.0
41.7% ± 3.0
Zero-Shot CoT
79.7% ± 2.5
75.3% ± 2.8
67.6% ± 3.0
76.2% ± 3.1
68.5% ± 2.9
66.2% ± 3.0
BFRS
80.1% ± 2.5
75.4% ± 2.7
71.1% ± 2.8
76.5% ± 3.1
68.5% ± 2.9
68.6% ± 2.9
MIPROv2
80.6% ± 2.4
75.3% ± 2.6
68.7% ± 2.9
76.1% ± 3.1
56.5% ± 3.1
68.6% ± 2.9
GPQA
HELM Baseline
57.0% ± 4.7
53.4% ± 4.7
45.5% ± 4.7
57.6% ± 4.7
40.4% ± 4.5
34.3% ± 4.3
Table 4: HELM leaderboard (general) across six language models and five prompting methods. Green marks the highest performance among evaluated prompting methods. Entries are reported as mean ± 95% bootstrap CI.
Figure 3: Heat map showing increase in accuracy of each prompting method over HELM’s baseline (light=small, dark=large). Across six models, x-axis lists prompting methods, y-axis lists benchmarks. All structured prompting methods exhibit similar improvements, o3 Mini remains relatively insensitive, and smaller open-source models achieve largest gains.
Benchmark
Prompting Method
Claude 3.7 Sonnet
Gemini 2.0 Flash
GPT 4o
o3 Mini
Llama 3.3 70B
Qwen3 4B
MedCalc-Bench
HELM Baseline
21.0% ± 2.5
15.8% ± 2.3
18.8% ± 2.4
34.0% ± 2.9
11.3% ± 2.0
4.7% ± 1.4
Zero-Shot Predict
20.6% ± 2.5
17.0% ± 2.4
15.7% ± 2.3
33.4% ± 3.0
9.9% ± 1.9
11.5% ± 2.0
Zero-Shot CoT
35.3% ± 3.0
26.3% ± 2.7
26.6% ± 2.8
34.2% ± 3.0
22.5% ± 2.6
20.8% ± 2.5
BFRS
34.1% ± 3.0
25.2% ± 2.7
27.0% ± 2.8
34.7% ± 3.0
20.0% ± 2.5
22.7% ± 2.6
MIPROv2
34.7% ± 3.0
25.4% ± 2.8
26.8% ± 2.8
34.3% ± 2.9
21.0% ± 2.5
21.5% ± 2.5
Medec
HELM Baseline
62.8% ± 3.9
59.6% ± 4.0
58.0% ± 3.9
68.7% ± 3.7
52.9% ± 4.0
52.1% ± 4.0
Table 5: MedHELM leaderboard (medical) across six language models and five prompting methods. Green marks the highest performance among evaluated prompting methods. Entries are reported as mean ± 95% bootstrap CI.
Figure 4: Accuracy vs cost tradeoff across prompting methods. Each point represents a model-prompt pair, with x-axis showing additional prompt tokens (relative to HELM baseline) and y-axis showing macro-averaged accuracy across benchmarks. Zero-Shot CoT achieves the most cost-effective improvements.
Figure 5: Accuracy vs cost tradeoff across prompting methods and benchmarks. Each point represents a model-prompt pair, with x-axis showing additional prompt tokens (relative to HELM baseline) and y-axis showing accuracy. Zero-Shot CoT achieves the most cost-effective improvements.