GFlowPO: Generative Flow Network as a Language Model Prompt Optimizer

Junmo Cho; Suhan Kim; Sangjune An; Minsu Kim; Dong Bok Lee; Heejun Lee; Sung Ju Hwang; Hae Beom Lee

cs.AIFeb 3, 2026

GFlowPO: Generative Flow Network as a Language Model Prompt Optimizer

Authors: Junmo Cho, Suhan Kim, Sangjune An, Minsu Kim, Dong Bok Lee, Heejun Lee, Sung Ju Hwang, Hae Beom Lee

Organizations: Korea Advanced Institute of Science and Technology (KAIST) · Korea University · Mila – Québec AI Institute

Abstract

Finding effective prompts for language models (LMs) is critical yet notoriously difficult: the prompt space is combinatorially large, rewards are sparse due to expensive target-LM evaluation. Yet, existing RL-based prompt optimizers often rely on on-policy updates and a meta-prompt sampled from a fixed distribution, leading to poor sample efficiency. We propose GFlowPO, a probabilistic prompt optimization framework that casts prompt search as a posterior inference problem over latent prompts regularized by a meta-prompted reference-LM prior. In the first step, we fine-tune a lightweight prompt-LM with an off-policy Generative Flow Network (GFlowNet) objective, using a replay-based training policy that reuses past prompt evaluations to enable sample-efficient exploration. In the second step, we introduce Dynamic Memory Update (DMU), a training-free mechanism that updates the meta-prompt by injecting both (i) diverse prompts from a replay buffer and (ii) top-performing prompts from a small priority queue, thereby progressively concentrating the search process on high-reward regions. Across few-shot text classification, instruction induction benchmarks, and question answering tasks, GFlowPO consistently outperforms recent discrete prompt optimization baselines.

Figures & tables

Figure 1: Concepts. Blue contour indicates high performing prompt regions. (a) Existing on-policy RL frameworks fail to explore the huge combinatorial search space with poor sample efficiency. (b) Our GFlowPO can efficiently explore the search space by gradually annealing the posterior density (dotted ellipses) toward more promising areas with off-policy GFlowNet training in step-A and DMU in step-B .

Figure 2: GFlowPO pipeline. The optimizer prompt-LM samples prompts conditioned on a meta-prompt $M$ , and the target LLM provides task rewards. GFlowPO alternates between off-policy GFlowNet training for replay-based posterior inference and Dynamic Memory Update (DMU), which injects diverse and high-reward prompts into $M$ to improve exploration and prompt quality.

Panel of Figure 3 — Figure 3: Meta-prompt template for GFlowPO .

	Method	SST-2	MRPC	RTE	QNLI	MNLI	SNLI	Average
Fixed Prompt	Few-Shot Prompt	$66.2{\scriptstyle\pm 13.9}$	$68.5{\scriptstyle\pm 0.8}$	$79.8{\scriptstyle\pm 2.8}$	$78.7{\scriptstyle\pm 2.7}$	$63.2{\scriptstyle\pm 1.1}$	$67.9{\scriptstyle\pm 5.9}$	$72.8{\scriptstyle\pm 1.0}$
Fixed Prompt	Manual Prompt	$89.1$	$51.0$	$64.0$	$73.0$	$67.0$	$47.0$	$65.2$
Discrete Prompt Tuning	APE	$83.2$	$55.3$	$78.6$	$75.0$	$54.6$	$72.3$	$70.1$
	ProTeGi	$69.2$	$48.8$	$73.2$	$74.2$	$56.6$	$61.3$	$64.0$
	EvoPrompt	$89.8{\scriptstyle\pm 0.0}$	$\underline{71.7}{\scriptstyle\pm 1.2}$	$78.6{\scriptstyle\pm 1.1}$	$78.8{\scriptstyle\pm 0.3}$	$62.2{\scriptstyle\pm 1.9}$	$58.8{\scriptstyle\pm 3.8}$	$73.3{\scriptstyle\pm 0.5}$
	PromptWizard	$89.8{\scriptstyle\pm 0.0}$	$58.5{\scriptstyle\pm 16.2}$	$71.0{\scriptstyle\pm 8.2}$	$64.9{\scriptstyle\pm 1.9}$	$58.4{\scriptstyle\pm 2.1}$	$62.5{\scriptstyle\pm 5.5}$	$67.5{\scriptstyle\pm 3.7}$

Table 1: Results on few-shot text classification datasets. We report average accuracy with standard deviation over three runs using different random seeds. The best results are bolded , and the second-best results are underlined .

Figure 4: Heatmap of performances on few-shot text classification tasks (6 datasets) with various prompt-LM, target LM pairs. Each number is averaged over six tasks. MP : Manual Prompt, G2 : Gemma-2B, G7 : Gemma-7B, M7 : Mistral-7B, L8 : Llama-3-8B, F11 : Falcon-11B.

	Instruction Induction (II)							BBII
	Spelling	Syntax	Semantics	Translation	GLUE	Others	All	Text class.	Text gen.
Method	(4)	(4)	(5)	(4)	(3)	(4)	(24)	(12)	(6)
Fewshot	3.8	0.0	0.6	0.0	35.0	0.5	5.2	51.5	6.2
Manual	29.6	36.8	14.6	23.3	49.7	57.1	33.7	51.6	37.6
APE	43.8	67.2	44.5	35.9	45.9	74.7	51.9	56.5	49.6
ProTeGi	43.8	68.8	27.5	56.3	58.3	62.3	51.6	56.6	55.6

Table 2: Results on instruction induction datasets.

Method	MMLU	OpenBookQA
APE	52.1	70.7
ProTeGi	53.5	71.5
EvoPrompt	53.7	69.4
PromptWizard	47.3	61.6
GRACE	47.2	70.8
StablePrompt	55.8	72.2

Table 3: Results on question answering datasets.

Method	GSM8K	AIME-2025	BBH (5 randomly chosen tasks)
Method	GSM8K	AIME-2025	movie rec.	object count.	tracking five	hyperbaton	causal	Avg. (5)	All. (19)
EvoPrompt	$93.2{\scriptstyle\pm 0.2}$	$46.2{\scriptstyle\pm 3.1}$	$68.7{\scriptstyle\pm 0.6}$	$\underline{97.3}{\scriptstyle\pm 0.6}$	$81.7{\scriptstyle\pm 1.5}$	$\textbf{99.3}{\scriptstyle\pm 0.6}$	$\underline{59.7}{\scriptstyle\pm 0.6}$	$81.3$	$\underline{80.9}{\scriptstyle\pm 0.5}$
PromptWizard	$91.4{\scriptstyle\pm 1.9}$	$45.1{\scriptstyle\pm 2.1}$	$67.7{\scriptstyle\pm 0.6}$	$95.7{\scriptstyle\pm 0.6}$	$79.3{\scriptstyle\pm 1.5}$	$97.3{\scriptstyle\pm 1.2}$	$58.3{\scriptstyle\pm 2.3}$	$79.7$	$78.0{\scriptstyle\pm 0.8}$
GRACE	$93.3{\scriptstyle\pm 0.0}$	$45.1{\scriptstyle\pm 2.8}$	$66.7{\scriptstyle\pm 1.5}$	$96.0{\scriptstyle\pm 0.0}$	$83.3{\scriptstyle\pm 12.1}$	$97.3{\scriptstyle\pm 1.5}$	$56.0{\scriptstyle\pm 2.7}$	$79.9$	$79.4{\scriptstyle\pm 0.7}$
StablePrompt	$87.3{\scriptstyle\pm 6.0}$	$40.2{\scriptstyle\pm 3.7}$	$66.3{\scriptstyle\pm 4.2}$	$92.3{\scriptstyle\pm 5.5}$	$71.3{\scriptstyle\pm 9.8}$	$98.3{\scriptstyle\pm 1.2}$	$59.0{\scriptstyle\pm 1.7}$	$77.4$	$76.8{\scriptstyle\pm 0.3}$
PRL	$92.2{\scriptstyle\pm 1.2}$	$\underline{46.9}{\scriptstyle\pm 1.9}$	$67.3{\scriptstyle\pm 0.6}$	$95.7{\scriptstyle\pm 3.5}$	$70.3{\scriptstyle\pm 3.1}$	$96.3{\scriptstyle\pm 2.5}$	$58.7{\scriptstyle\pm 4.2}$	$77.7$	$76.4{\scriptstyle\pm 1.4}$

Table 4: Results on GSM8K, AIME, and BBH.

Figure 5: Training accuracy as a function of the number of sampled prompts on four II tasks. The first 400 evaluations of GFlowPO correspond to the Pre-Step, where the prompt-LM is not updated (see Appendix P ).

GFlowPO variants	-on-X	-off-X	-on-O	-off-O
Avg. Acc.	74.4	75.2	78.0	80.2

Table 5: Ablation on GFlowPO variants averaged over 10 II text generation tasks. on/off : on-/off-policy training; O/X : with/without DMU.

Appendix figures & tables20 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: Correlation plots on train accuracy vs test accuracy ( left ), and train log-likelihood vs test accuracy ( right ) on six text classification tasks (SST2, RTE, MRPC, MNLI, QNLI, and SNLI).

Task	GFlowPO w/o pre-step	GFlowPO - $\mathcal{B}$	GFlowPO - $\mathcal{Q}$	GFlowPO
Cause and effect	80.0	76.0	52.0	88.0
Negation	83.0	89.0	86.0	87.0
Letters list	94.0	94.0	96.0	96.0
Larger animal	56.0	91.0	90.0	93.0
Sentence similarity	39.0	36.0	36.0	39.0
Num to verbal	95.0	94.0	100.0	98.0

Appendix

Table 6: Comparison of GFlowPO variants. GFlowPO w/o Pre-Step removes the Pre-Step procedure. GFlowPO-

\mathcal{B}

samples reference prompts only from the replay buffer

\mathcal{B}

, while GFlowPO-

\mathcal{Q}

samples reference prompts only from the high-reward buffer

\mathcal{Q}

. GFlowPO achieves the best performance across all variants.

Hyperparameter	Value
Learning rate	$10^{-4}$
Train buffer size	1000
High-reward buffer size	5
Total train steps	200
Pre-steps	100
Max prompt length	150

Appendix

Table 7: Hyperparameters of GFlowPO .

	Sentence Similarity	Hyperbaton
Manual	Rate from 0 (definitely not) to 5 (perfectly) the degree in which both sentences describe the same event. ( Acc: 20.0 )	Which sentence has the correct adjective order? ( Acc: 56.5 )
GEPA	You are an AI assistant that applies a transformation to given inputs. Your task is to identify and extract specific details from two sentences and then concatenate them into one sentence, maintaining the same tense and voice. The sentences will always be provided in a pair. The transformation should result in a clear, concise and grammatically correct answer that is logically equivalent to the original sentences … ( Acc: 18.8 )	The assistant is an AI designed to answer multiple-choice questions by selecting the single best answer. The assistant will be provided with an input and a list of choices, and is expected to output a JSON object with the following format: {"final_answer": <the letter of the correct choice>, "solution_pad": <your reasoning>} . Strictly follow the provided format for outputting your final answer and solution padding … ( Acc: 63.3 )
GFlowPO	I will provide you with two sentences. Based on the given sentences, determine the level of similarity between them. The output should be one of the following numbers: 0 - definitely not 1 - probably not 2 - possibly 3 - probably 4 - almost perfectly Based on the provided input-output pairs, it seems that your friend correctly identified the level of similarity between the two sentences for each pair. ( Acc: 50.0 )	“Rearrange the adjective and the noun in each input.” based on this instruction, the friend’s output for each input is correct. They have successfully rearranged the order of the adjective and the noun in each input to match the expected output. For example, with the first input: Input: a “gold awful knife” b “awful gold knife” Output: A “awful gold knife” Your friend has correctly rearranged the order of the adjective “awful” and the noun “gold” to match the expected output “awful gold”. Similarly, for the other inputs, the friend has also rearranged the order of the adjective and the noun. ( Acc: 73.0 )

Appendix

Table 8: Prompts of Manual Prompt, GEPA, and GFlowPO on Sentence Similarity (II) and Hyperbaton (BBII). Test accuracy is reported in parentheses. Long GEPA prompts are truncated (…).

Figure 7: Meta prompt for prompt generation. (a) Meta-prompt used in StablePrompt, consisting of an instruction and few-shot examples. (b) Meta prompt with a reference prompt sampled only from high reward buffer

\mathcal{Q}

. (c) Meta prompt used in our proposed GFlowPO . Reference prompts are sampled from both high reward buffer

\mathcal{Q}

and replay buffer

\mathcal{B}

, which balances exploitation and exploration during training.

Dataset	Task type	# classes	# training	# test	Verbalizer
SST2	Sentiment	2	32	872	[“yes”, “no”]
MRPC	NLI	2	32	408	[“yes”, “no”]
RTE	NLI	2	32	277	[“yes”, “no”]
QNLI	NLI	2	32	5,460	[“yes”, “no”]
MNLI	NLI	3	48	9,820	[“yes”, “maybe”, “no”]
SNLI	NLI	3	48	9,842	[“yes”, “maybe”, “no”]

Appendix

Table 9: Details of the dataset statistics and verbalizer on few-shot text classification tasks.

Task name	Task type	Metric	# training	# test
Causal judgment	Multiple choice	Accuracy	30	160
Disambiguation qa	Multiple choice	Accuracy	30	228
Epistemic reasoning	Multiple choice	Accuracy	30	1,970
Hyperbaton	Multiple choice	Accuracy	30	49,970
Implicatures	Multiple choice	Accuracy	30	462
Logical fallacy detection	Multiple choice	Accuracy	30	2,770

Appendix

Table 10: Details of BigBench-Hard instruction induction datasets.

Task name	Metric	# training	# test
Antonyms	Exact match	32	100
Word in context	Exact match	32	100
Rhymes	Exact match	32	100
Num to verbal	Exact match	32	100
Cause and effect	Exact match	26	25
Larger animal	Exact match	32	100

Appendix

Table 11: Details of instruction induction datasets.

Type	Subject	# training	# test
STEM	Abstract algebra	11	100
	Anatomy	14	135
	Astronomy	16	152
	College biology	16	144
	College chemistry	8	100
	College computer science	11	100

Appendix

Table 12: MMLU dataset statistics: training and test sizes by category.

Task name	Type	Fewshot	Manual	APE	ProTeGi	EvoPrompt	PromptWizard	GRACE	PRL	GEPA	StablePrompt	GFlowPO
Causal judgment	Multiple choice	58.8	52.5	58.1	56.7	$50.6{\scriptstyle\pm 0.0}$	$47.5{\scriptstyle\pm 0.6}$	$55.4{\scriptstyle\pm 1.4}$	$\textbf{59.8}{\scriptstyle\pm 1.0}$	$58.1{\scriptstyle\pm 1.7}$	$57.1{\scriptstyle\pm 1.8}$	$\underline{58.8}{\scriptstyle\pm 1.1}$
Disambiguation qa	Multiple choice	64.3	52.2	64.0	61.4	$61.0{\scriptstyle\pm 5.3}$	$52.9{\scriptstyle\pm 5.8}$	$51.3{\scriptstyle\pm 5.8}$	$57.3{\scriptstyle\pm 6.4}$	$\textbf{66.2}{\scriptstyle\pm 5.8}$	$64.6{\scriptstyle\pm 2.3}$	$\underline{64.8}{\scriptstyle\pm 1.1}$
Epistemic reasoning	Multiple choice	43.7	57.2	58.4	63.8	$54.0{\scriptstyle\pm 15.0}$	$50.3{\scriptstyle\pm 7.5}$	$50.9{\scriptstyle\pm 14.1}$	$50.9{\scriptstyle\pm 14.5}$	$44.0{\scriptstyle\pm 1.3}$	$\underline{61.0}{\scriptstyle\pm 1.4}$	$59.0{\scriptstyle\pm 5.2}$
Hyperbaton	Multiple choice	47.9	56.5	75.6	76.1	$\textbf{79.2}{\scriptstyle\pm 1.4}$	$75.7{\scriptstyle\pm 3.5}$	$65.8{\scriptstyle\pm 6.6}$	$70.2{\scriptstyle\pm 7.2}$	$64.5{\scriptstyle\pm 7.2}$	$67.6{\scriptstyle\pm 7.1}$	$74.8{\scriptstyle\pm 3.7}$
Implicatures	Multiple choice	83.3	83.1	81.0	73.6	$82.3{\scriptstyle\pm 1.0}$	$74.9{\scriptstyle\pm 8.4}$	$79.9{\scriptstyle\pm 0.0}$	$79.0{\scriptstyle\pm 3.6}$	$82.8{\scriptstyle\pm 0.3}$	$82.3{\scriptstyle\pm 0.6}$	$\textbf{84.5}{\scriptstyle\pm 1.0}$
Logical fallacy detection	Multiple choice	58.2	63.5	56.5	58.2	$\textbf{63.6}{\scriptstyle\pm 6.3}$	$53.2{\scriptstyle\pm 0.3}$	$56.8{\scriptstyle\pm 0.5}$	$60.3{\scriptstyle\pm 4.5}$	$\underline{60.9}{\scriptstyle\pm 0.1}$	$52.9{\scriptstyle\pm 0.1}$	$53.2{\scriptstyle\pm 0.2}$

Appendix

Table 13: Full experimental results on the Big-Bench Instruction Induction (BBII) datasets.

Task name	Metric	Fewshot	Manual	APE	ProTeGi	EvoPrompt	PromptWizard	GRACE	PRL	GEPA	StablePrompt	GFlowPO
Antonyms	Exact match	0.0	43.0	62.5	25.0	$47.3{\scriptstyle\pm 15.0}$	$56.0{\scriptstyle\pm 19.0}$	$64.3{\scriptstyle\pm 4.9}$	$37.7{\scriptstyle\pm 32.9}$	$45.7{\scriptstyle\pm 18.2}$	$\underline{67.7}{\scriptstyle\pm 3.2}$	$\textbf{74.3}{\scriptstyle\pm 2.3}$
Word in context	Exact match	55.0	46.0	37.5	50.0	$56.3{\scriptstyle\pm 3.5}$	$60.0{\scriptstyle\pm 3.6}$	$44.3{\scriptstyle\pm 11.7}$	$\textbf{61.0}{\scriptstyle\pm 4.6}$	$58.3{\scriptstyle\pm 1.5}$	$\underline{60.3}{\scriptstyle\pm 5.0}$	$\textbf{61.0}{\scriptstyle\pm 4.4}$
Rhymes	Exact match	0.0	3.0	6.3	25.0	$6.7{\scriptstyle\pm 3.2}$	$6.7{\scriptstyle\pm 6.0}$	$9.3{\scriptstyle\pm 3.1}$	$\textbf{12.0}{\scriptstyle\pm 3.5}$	$4.7{\scriptstyle\pm 1.5}$	$3.7{\scriptstyle\pm 3.8}$	$\underline{8.7}{\scriptstyle\pm 2.1}$
Num to verbal	Exact match	0.0	61.0	93.8	100.0	$73.7{\scriptstyle\pm 6.4}$	$83.7{\scriptstyle\pm 4.0}$	$82.7{\scriptstyle\pm 18.6}$	$81.3{\scriptstyle\pm 12.7}$	$82.3{\scriptstyle\pm 14.2}$	$\underline{89.0}{\scriptstyle\pm 4.4}$	$\textbf{97.7}{\scriptstyle\pm 1.5}$
Cause and effect	Exact match	0.0	24.0	60.0	0.0	$24.0{\scriptstyle\pm 0.0}$	$48.0{\scriptstyle\pm 10.6}$	$25.3{\scriptstyle\pm 22.0}$	$49.3{\scriptstyle\pm 30.0}$	$52.0{\scriptstyle\pm 17.4}$	$54.7{\scriptstyle\pm 10.1}$	$\textbf{81.3}{\scriptstyle\pm 11.5}$
Larger animal	Exact match	0.0	3.0	56.3	25.0	$6.0{\scriptstyle\pm 0.0}$	$47.3{\scriptstyle\pm 29.0}$	$51.7{\scriptstyle\pm 5.8}$	$72.7{\scriptstyle\pm 25.8}$	$48.0{\scriptstyle\pm 10.2}$	$\underline{86.7}{\scriptstyle\pm 8.5}$	$\textbf{92.7}{\scriptstyle\pm 0.6}$

Appendix

Table 14: Full experimental results on the Instruction Induction (II).

Task group	Task name	Metric	Fewshot	Manual	APE	ProTeGi	Evo Prompt	Prompt Wizard	GRACE	PRL	GEPA	Stable Prompt	GFlowPO
Spelling	First word letter	Exact match	3.0	73.0	87.8	100.0	$92.7{\scriptstyle\pm 6.4}$	$96.7{\scriptstyle\pm 2.9}$	$97.0{\scriptstyle\pm 2.0}$	$\underline{98.0}{\scriptstyle\pm 0.0}$	$80.7{\scriptstyle\pm 15.0}$	97.0	99.0
	Second word letter	Exact match	12.0	8.0	6.3	25.0	$17.7{\scriptstyle\pm 4.0}$	$18.7{\scriptstyle\pm 10.1}$	$\underline{21.0}{\scriptstyle\pm 15.1}$	$8.0{\scriptstyle\pm 0.0}$	$7.0{\scriptstyle\pm 5.6}$	9.7	27.3
	Letters list	Exact match	0.0	0.0	68.8	50.0	$1.7{\scriptstyle\pm 1.5}$	$12.7{\scriptstyle\pm 21.9}$	$49.3{\scriptstyle\pm 47.2}$	$0.0{\scriptstyle\pm 0.0}$	$77.0{\scriptstyle\pm 17.8}$	82.3	95.3
	Orthography starts with	Exact set	0.0	37.5	12.5	0.0	$\textbf{60.7}{\scriptstyle\pm 2.5}$	$\underline{58.3}{\scriptstyle\pm 4.0}$	$41.3{\scriptstyle\pm 13.9}$	$35.3{\scriptstyle\pm 1.2}$	$8.0{\scriptstyle\pm 1.0}$	13.3	15.0
	Average		3.8	29.6	43.8	43.8	42.2	46.6	52.2	35.3	43.2	50.6	59.2
Syntax	Singular to plural	Exact match	0.0	75.0	93.8	100.0	$93.0{\scriptstyle\pm 0.0}$	$95.0{\scriptstyle\pm 0.0}$	$95.0{\scriptstyle\pm 3.5}$	$79.3{\scriptstyle\pm 27.2}$	$\textbf{99.3}{\scriptstyle\pm 1.2}$	96.0	98.3

Appendix

Table 15: Category-wise results on the Instruction Induction (II) tasks.

Type	Subject	Metric	Fewshot +Manual	CoT	APE	ProTeGi	Evo Prompt	Prompt Wizard	GRACE	PRL	GEPA	Stable Prompt	GFlowPO
STEM	Abstract algebra	Accuracy	30.0	33.0	31.0	35.0	31.0	30.0	32.0	29.0	38.0	33.0	31.0
	Anatomy	Accuracy	50.4	51.9	49.6	53.0	54.1	55.6	48.2	56.3	54.8	53.3	52.6
	Astronomy	Accuracy	57.9	64.5	54.0	56.6	61.2	57.9	52.6	58.6	57.2	63.2	57.9
	College biology	Accuracy	66.7	67.4	57.0	65.8	55.0	46.0	45.0	60.4	68.1	65.3	56.9
	College chemistry	Accuracy	38.0	34.0	39.0	40.0	60.4	46.4	45.3	32.0	35.0	39.0	36.0
	College computer science	Accuracy	41.0	48.0	32.8	37.0	60.4	56.3	59.0	41.0	45.0	40.0	39.0

Appendix

Table 16: Full experimental results on the MMLU.

Figure 8: Learning curves of GFlowPO , StablePrompt, PRL, and GEPA. Given the same prompt sampling budgets, GFlowPO consistently shows higher sample efficiency in average test accuracy.

Method	w/o DMU	w/ DMU
Frozen Prompt-LM	69.9	74.4
Reward-weighted off-policy MLE	73.3	74.4
On-policy GFlowNet	74.4	78.0
Off-policy GFlowNet	75.2	80.2

Appendix

Table 18: Average accuracy of different prompt-LM training strategies with and without DMU.

Method	Zero shot	Few shot	Manual	Stable Prompt	GEPA	GFlowPO
Avg. (6 tasks)	71.2	71.5	71.2	$72.3{\scriptstyle\pm 0.7}$	$\underline{76.8}{\scriptstyle\pm 1.8}$	$\textbf{77.5}{\scriptstyle\pm 1.4}$

Appendix

Table 19: Average accuracy on six BBH tasks with GPT-4.1 as the target LM.

	Disambiguation QA	Geometric Shapes	Movie Recommendation
Manual	Identify what the pronoun refers to. Weigh each possible antecedent against the sentence; if the reference is genuinely ambiguous, say so rather than forcing a choice. ( Acc: 64.0 )	Treat the SVG path as a figure defined by vertices and line segments. Trace the path, count the distinct vertices and edges, and identify the shape. ( Acc: 65.0 )	Recall the genre, tone, era, and reception of the listed movies. Compare each option against these shared traits and pick the most similar one. ( Acc: 80.0 )
GFlowPO	For each sentence, identify the antecedent of the pronoun (which noun phrase it refers to) and select the correct option. The pronoun refers to the subject of the sentence unless context clearly indicates otherwise. If the antecedent is ambiguous or unclear, choose (C) Ambiguous. Provide only the letter (A, B, or C) as the output. ( Acc: 67.0 )	Identify the geometric shape described by the SVG path element. Count the number of distinct vertices connected by line segments to determine the shape. If the shape has 6 vertices, output ‘C’ for hexagon. If it has 8 vertices, output ‘F’ for octagon. If it contains an arc and is not a polygon, output ‘K’ for ellipse. If it has 5 vertices, output ‘G’ for pentagon. If it contains an arc and forms a sector, output ‘I’. If it contains an arc and forms a kite, output ‘D’. If it contains an arc and forms a circle, output ‘A’. If it has 7 vertices, output ‘B’ for heptagon. If it … ( Acc: 88.0 )	Select the movie from the options that is most similar in theme, genre, or narrative to the listed films, with a focus on critical acclaim and shared storytelling elements. Output the letter corresponding to the most appropriate choice. ( Acc: 88.0 )

Appendix

Table 20: Comparison of manual prompts and prompts discovered by GFlowPO with GPT-4.1 as the target LM. Test accuracy is reported in parentheses.

	Sentence Similarity	Hyperbaton
Manual	Rate from 0 (definitely not) to 5 (perfectly) the degree in which both sentences describe the same event. ( Acc: 20.0 )	Which sentence has the correct adjective order? ( Acc: 56.5 )
GFlowPO	I will provide you with two sentences. Based on the given sentences, determine the level of similarity between them. The output should be one of the following numbers: 0 - definitely not 1 - probably not 2 - possibly 3 - probably 4 - almost perfectly Based on the provided input-output pairs, it seems that your friend correctly identified the level of similarity between the two sentences for each pair. ( Acc: 50.0 )	“Rearrange the adjective and the noun in each input.” based on this instruction, the friend’s output for each input is correct. They have successfully rearranged the order of the adjective and the noun in each input to match the expected output. For example, with the first input: Input: a “gold awful knife” b “awful gold knife” Output: A “awful gold knife” Your friend has correctly rearranged the order of the adjective “awful” and the noun “gold” to match the expected output “awful gold”. Similarly, for the other inputs, the friend has also rearranged the order of the adjective and the noun. ( Acc: 73.0 )

Appendix

Table 21: Comparison of Manual Prompts and prompts discovered by GFlowPO on Sentence Similarity (II) and Hyperbaton (BBII). Test accuracy is reported in parentheses.

$k$	2	3	4	5
$(k_{b},k_{q})$	(1, 1)	(2, 1)	(2, 2)	(3, 2)
Avg. accuracy	77.6	80.2	78.9	79.2

Appendix

Table 22: Sensitivity of GFlowPO to the number of reference prompts

k=k_{b}+k_{q}

in DMU. We report average accuracy over the same 10 II tasks as in Table 6 .

Figure 9: Prompt evolution during GFlowPO training. The prompts become increasingly precise and specific as training steps change. Initially (at step 0), prompts tend to include ambiguous or irrelevant context, but by step 50 they are refined to focus on the core task rules, and by step 100 they converge to a more explicit form with finer-grained criteria.

LLM Input Optimization

See all 26 papers

RLMOpt: Adaptive Prompt Optimization via Recursive Language Models
Aug 11, 2026Subhash Bangalore Satheesha, Nirvik Pande, Deepthi Duddempudi +1Large Language Model-Guided Optimization LLM Agents
Autonomize AI · Carnegie Mellon University
PrefPO: Pairwise Preference Prompt Optimization
Mar 13, 2026Rahul Singhal, Pradyumna Tambwekar, Karime MaamariAutomatic Prompt Optimization Prompt Optimization
Distyl AI
ESPO: Error-Structured Prompt Optimization via Diagnose, Diversify, and Stabilize
Sep 3, 2026Lihao Liu, Peng Tang, Kunwar Yashraj Singh +1Evolutionary Optimization Large Language Model-Guided Optimization
AWS Agentic AI

Aug 11, 2026J/K move · Enter open · S save

RLMOpt: Adaptive Prompt Optimization via Recursive Language Models

Subhash Bangalore Satheesha, Nirvik Pande, Deepthi Duddempudi, Bharath Dandala

Autonomize AI · Carnegie Mellon University

Large Language Model-Guided Optimization LLM Agents Prompt Optimization Recursive Language Models LLM Input Optimization

Open paper AskPDF arXiv

Prompt optimizers automate the search for prompts that improve language-model performance, but existing methods rely on a predefined optimization procedure: the algorithm determines which candidates to explore and how the search progresses, while the language model generates or refines prompt proposals. We introduce RLMOpt, a prompt optimizer that makes the search policy itself language-model-driven through a recursive language model (RLM). The RLM agent operates over a tool-based environment, inspecting task information, analyzing failures, generating candidates, allocating evaluation budget, and deciding when to stop. A deterministic harness complements the agent by enforcing objective scoring, Pareto-based selection, and regression constraints. We evaluate RLMOpt across four benchmarks spanning structured clinical information extraction (Chia), multi-hop question answering (HotpotQA), verifiable instruction following (IFBench-2025), and multi-turn tool-calling agents (BFCL). In a matched comparison at a single seed, RLMOpt obtains the best held-out score on all four benchmarks and leads the four-task mean (0.610 against 0.589 for GEPA). Repeating each benchmark across seeds yields 11 matched benchmark-seed comparisons, in which RLMOpt outperforms GEPA in 9 cases. Across all 11 runs, it never produced a prompt that underperformed its seed, whereas GEPA fell below its starting point twice. It is also more efficient, achieving these results with fewer search rollouts while producing prompts that are 27-79% the size of those produced by GEPA. Our results further show that optimization gains are determined primarily by the headroom available in the seed prompt, rather than by the search budget. Efficient optimization therefore depends on reaching the available headroom reliably and with minimal search

Abstract

LLM Input Optimization

Explore similar work

RLMOpt: Adaptive Prompt Optimization via Recursive Language Models

PrefPO: Pairwise Preference Prompt Optimization

ESPO: Error-Structured Prompt Optimization via Diagnose, Diversify, and Stabilize