BayesPrompt: human readable prompts that make sense
Authors: Franky Kevin Nando Tezoh, Ali Hussaini Umar, Alessandro Laio, Guido Sanguinetti, Riccardo Rende
Organizations: Department of Physics, Scuola Internazionale Superiore di Studi Avanzati (SISSA), Trieste, Italy · Center for Computational Quantum Physics, Flatiron Institute, 162 5th Avenue, New York, NY 10010
Reconstructing prompts that can elicit a desired answer or behaviour in an LLM is an open and important research topic. Optimisation methods which aim at minimising the perplexity of a given answer, however, consistently yield so-called pseudoprompts, unintelligible strings of tokens which can lack human interpretability. We argue that this is a consequence of the ill-posedness of the prompt optimisation task. By reframing the task as a Bayesian posterior inference over prompts, we propose an efficient algorithm to sample prompts which are both efficient (in terms of perplexity) and human readable. We compare our approach with state of the art alternatives showing on a real data set a marked improvement over a range of metrics.
Figures & tables
Figure 1: Comparison of the distribution of probability between the ground truth (black), the optimisation-based methods (GCG, GCG-Reg, GD-PEZ) under random initialisation and the MCMC-sampled approach. Random initialisation results are computed over 371 question-answer pairs from the NQ-OPEN dataset, using a prompt length of 10 tokens. Left: Distribution of the negative log-conditional probability −logP(a∣q) , measuring how well the generated question recovers the target answer. Right: Distribution of the negative log-fluency, −logP(q) , measuring the linguistic naturalness of the generated prompt.
Figure 2: Effect of initialisation on optimisation-based methods (GCG, GCG-Reg, GD-PEZ), compared against ground truth (black dashed) and MCMC (red), evaluated over 371 question-answer pairs fron NQ-OPEN dataset. Each row shows one method under warm-start (solid) and random (dotted) initialisation, with the same ground truth and the MCMC distributions repeated across rows for reference. Left column (a,c,e): Distributionod the negative log-conditional probability −logP(a∣q) , measuring confidence in the recovered answer. Right column (a,c,e): Distribution of the negative log-fluency −logP(q) , measuring the linguistic naturalness of the recovered prompt.
Figure 3: Qualitative analysis of prompts learned by the optimisation-based methods and the sampling-based MCMC framework . For each target answer, the ground truth question is shown alongside the corresponding question recovered by GCG, GCG-Reg, and GD-PEZ under random initialisation, and by our MCMC based sampler (final sample and forward proposal).
Figure 4: Left: Histogram showing the total distribution of winning instances for question-answer plausibility across optimisation-based methods under random initialisation (GCG, GCG-Reg, GD-PEZ) and sampling framework (reporting the forward variant proposal for our MCMC approach ). Right: Grammar correctness score distribution for the same evaluated methods.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: Comparison of the distribution of probability between the ground truth (blue), the sampling and optimisation frameworks evaluated over 371 question-answer pairs under warm initialisation. Left: Distribution of the negative log-conditional probability −logP(a∣q) . Right: Distribution of the negative log-fluency, −logP(q) .
Figure 6: Left: Mean over 371 questions of the −logP(a,q) as function of the number of steps for the forward and backward/reverse proposals under warm-start initialisation. Right: Mean acceptance rate as function of the number of steps for the same configurations.
Figure 7: Qualitative analysis of prompts learned by the optimisation-based methods and the sampling-based MCMC framework . For each target answer, the ground truth question is shown alongside the corresponding question recovered by GCG, GCG-Reg, and GD-PEZ under warm-start initialisation, and by our MCMC based sampler (final sample and forward proposal).