Streamlined Reflective Evolution for Task-Adaptive Self-Refinement Pipelines

Xiaofan Zhou; Lu Cheng

cs.CLSep 26, 2026

Streamlined Reflective Evolution for Task-Adaptive Self-Refinement Pipelines

Authors: Xiaofan Zhou, Lu Cheng

Organizations: Pennsylvania State University

Abstract

Reflective prompt optimization improves large language model (LLM) systems without updating model weights, but fixed architectures constrain how self-refinement is organized. We introduce Workflow-Designing Agents (WDA), a framework for streamlined reflective evolution of task-adaptive self-refinement pipelines. Starting from a minimal prompt, WDA jointly evolves stage instructions and their sequential structure. During evolution, we find that repeated revisions can accumulate redundant instructions in a single prompt. In WDA, we propose to address this problem with SPLIT, which redistributes these instructions across specialized stages. Three-example reflection and local screening guide selective search, while calibration scores guide Pareto admission and rollback of unhelpful trailing updates. The resulting pipelines are task-adaptive: their instructions and depth are learned from task data, then fixed for all test inputs within that task. We evaluate WDA on five benchmarks spanning knowledge, mathematical reasoning, multi-hop question answering, and instruction following. On Qwen3.5-9B, WDA achieves an average score of 51.24%, improving over the initial solver by 8.63 percentage points and the variant without SPLIT by 2.60 points. On GPT-4.1-mini, it achieves 49.00%, with corresponding gains of 5.62 and 3.69 points. These results support task-adaptive self-refinement as a complementary direction to broader agentic workflow search.

Figures & tables

Figure 1: WDA: reflective evolution with local screening, Pareto admission, and fixed test-time execution. The example illustrates Split separating constraint analysis from response generation; these excerpts do not establish prompt-length causality. Each stage receives the question and preceding output; Copy forwards that output without a model call.

Method	SuperGPQA	LiveBench Math	IFBench	BBEH	HotpotQA	Avg.
Initial solver	53.00	57.38	38.67	23.33	40.67	42.61
MIPROv2	51.33	51.64	31.33	22.00	49.33	41.13
GEPA	51.67	59.84	41.00	23.33	51.00	45.37
Self-Refine	54.33	65.57	47.67	25.67	39.00	46.45
DPO	50.00	57.38	40.33	21.67	40.00	41.88
WDA (no Split )	53.00	67.21	42.33	29.33	51.33	48.64

Table 1: Test performance (%) of Qwen3.5-9B across five benchmarks, including the DPO comparison. Avg. is the unweighted mean of the five displayed benchmark scores. Bold indicates the highest score in each column.

Method	SuperGPQA	LiveBench Math	IFBench	BBEH	HotpotQA	Avg.
Initial solver	49.67	56.56	37.33	28.33	45.00	43.38
MIPROv2	52.67	54.10	41.33	28.00	47.33	44.69
GEPA	51.00	57.38	42.67	22.33	51.00	44.88
Self-Refine	49.67	59.02	42.33	27.00	45.00	44.60
WDA (no Split )	50.33	58.20	38.67	29.00	50.33	45.31
WDA ( Split )	56.00	60.66	42.33	34.33	51.67	49.00

Table 2: Test performance (%) of GPT-4.1-mini across five benchmarks. Avg. is the unweighted mean of the five displayed benchmark scores. Bold indicates the highest score in each column.

	Qwen3.5-9B		GPT-4.1-mini
Dataset	No Split	Split	No Split	Split
SuperGPQA	1	2	3	3
LiveBench Math	3	2	3	3
IFBench	1	2	1	3
BBEH	3	2	2	3
HotpotQA	3	2	2	3

Table 3: LLM calls per inference for pipelines found with and without Split . Counts include the initial solver and exclude Copy slots. Avg. is the unweighted mean across benchmarks. The Qwen3.5-9B no- Split pipeline on SuperGPQA retains only the initial solver.

Figure 2: Recorded test performance across inference steps for Split -enabled search (blue, solid) and no- Split search (orange, dashed). Rows correspond to Qwen3.5-9B and GPT-4.1-mini; columns correspond to the five benchmarks. Circled endpoints mark the calibration-selected pipeline lengths. These recorded trajectories are distinct from the updated aggregate results in Tables 1 and 2 .

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Dataset	Verified DPO test (%)	Source
supergpqa	50.00	server_runs/qwen35_dpo_preference_s45_20260923/test/supergpqa_summary.json
livebench_math	57.38	server_runs/qwen35_dpo_preference_s45_20260923/test/livebench_math_summary.json
ifbench	40.33	server_runs/qwen35_dpo_preference_s45_20260923/test/ifbench_summary.json
bbeh	21.67	server_runs/qwen35_dpo_preference_s45_20260923/test/bbeh_summary.json
hotpotqa	40.00	server_runs/qwen35_dpo_preference_s45_20260923/test/hotpotqa_summary.json

Appendix

Table 6

LLM Self-Refinement

Large language model self-refinement studies how models can improve outputs through repeated critique, verification, revision, and feedback.

See all 114 papers

Reflective Prompt Tuning through Language Model Function-Calling
May 20, 2026Farima Fatahi Bayat, Moin Aminnaseri, Pouya Pezeshkpour +1LLM Self-Refinement LLM Prompting
Megagon Labs
Explore More, Validate Less: Efficient Reflective Prompt Optimization
Oct 19, 2026Xiaofan Zhou, Lu ChengPrompt Optimization Automatic Prompt Optimization
Pennsylvania State University, USA
REVERE: Reflective Evolving Research Engineer
Mar 21, 2026Balaji Dinesh Gangireddi, Aniketh Garikaparthi, Manasi Patwardhan +1LLM Agent Self-Improvement Prompt Optimization
TCS Research · Yale University

May 20, 2026J/K move · Enter open · S save

Reflective Prompt Tuning through Language Model Function-Calling

Farima Fatahi Bayat, Moin Aminnaseri, Pouya Pezeshkpour, Estevam Hruschka

Megagon Labs

LLM Self-Refinement LLM Prompting Automatic Prompt Optimization

Open paper AskPDF arXiv

Large language models (LLMs) have become increasingly capable of following instructions and complex reasoning, making prompting a flexible interface for adapting models without parameter updates. Yet prompt design remains labor-intensive and highly sensitive to formatting, phrasing, and instruction order, motivating automated prompt optimization methods that reduce manual effort while preserving inference-time flexibility. However, existing methods often search over prompt candidates or use fixed critique-refine pipelines driven by individual examples or small batches, limiting their ability to capture systematic error patterns and make targeted edits grounded in failure history. We propose Reflective Prompt Tuning (RPT), a framework that uses LLM function calling to simulate the iterative workflow of human prompt engineers. An LLM optimizer calls a diagnostic function that evaluates the target model over an entire optimization set, summarizes recurring failure modes, and returns a structured diagnostic report. The optimizer uses this report, together with an accumulated memory of prior reports, to revise the prompt for the next iteration. RPT further supports confidence-aware optimization by using calibration signals in diagnostic feedback and final prompt selection. Across three reasoning tasks, RPT improves over initial prompts by up to 12.9 points, remains competitive with state of the art, and improves confidence calibration. Our analyses show that RPT is especially effective on multi-hop and mathematical reasoning, producing targeted prompt revisions that align with diagnosed failure patterns and lead to gains in task performance and calibration.

Abstract

LLM Self-Refinement

Explore similar work

Reflective Prompt Tuning through Language Model Function-Calling

Explore More, Validate Less: Efficient Reflective Prompt Optimization

REVERE: Reflective Evolving Research Engineer