From Stochastic Answers to Verifiable Reasoning: Interpretable Decision-Making with LLM-Generated Code

Anirudh Jaidev Mahesh; Ben Griffin; Fuat Alican; Joseph Ternasky; Zakari Salifu; Kelvin Amoaba; Yagiz Ihlamur; Aaron Ontoyin Yin

cs.LGFeb 28, 2026

From Stochastic Answers to Verifiable Reasoning: Interpretable Decision-Making with LLM-Generated Code

Authors: Anirudh Jaidev Mahesh, Ben Griffin, Fuat Alican, Joseph Ternasky, Zakari Salifu, Kelvin Amoaba, Yagiz Ihlamur, Aaron Ontoyin Yin,

Organizations: Georgia Institute of Technology · University of Oxford · Vela Research · Amazon

Abstract

Large language models (LLMs) are increasingly used for high-stakes decision-making, yet existing approaches struggle to reconcile scalability, interpretability, and reproducibility. Black-box models obscure their reasoning, while recent LLM-based rule systems rely on per-sample evaluation, causing costs to scale with dataset size and introducing stochastic, hallucination-prone outputs. We propose reframing LLMs as code generators rather than per-instance evaluators. A single LLM call generates executable, human-readable decision logic that runs deterministically over structured data, eliminating per-sample LLM queries while enabling reproducible and auditable predictions. We combine code generation with automated statistical validation using precision lift, binomial significance testing, and coverage filtering, and apply cluster-based gap analysis to iteratively refine decision logic without human annotation. We instantiate this framework in venture capital founder screening, a rare-event prediction task with strong interpretability requirements. On VCBench, a benchmark of 4,500 founders with a 9% base success rate, our approach achieves 37.5% precision and an F0.5 score of 25.0%, outperforming GPT-4o (at 30.0% precision and an F0.5 score of 25.7%) while maintaining full interpretability. Each prediction traces to executable rules over human-readable attributes, demonstrating verifiable and interpretable LLM-based decision-making in practice.

Figures & tables

Method	Cost	Speed	Interpret.	Stat. valid.	Determin.
CAAFE Hollmann et al. (2023)	Low	Fast	Low	CV only	Yes
FeatLLM Balek et al. (2024)	High	Slow	High	None	No
RRF Griffin et al. (2025)	High	Slow	High	None	No
Ours	Low	Fast	High	Built-in	Yes

Table 1: Comparison with related work on automated feature engineering across five key dimensions.

Figure 1: Overview of our LLM-as-code-generator pipeline. The LLM is called once to generate executable Python rules, which then evaluate deterministically across all founders without further LLM involvement. Statistical validation filters low-quality rules, and cluster-based gap analysis guides iterative refinement.

Field	Type	Example Values
industry	string	“Software Development”
educations	list	degree, field, qs_ranking
jobs	list	role, company_size, industry, duration
ipos	list	prior IPO events
acquisitions	list	prior acquisition events

Table 2: Founder profile data structure.

Figure 2: Rule generation process. The LLM receives structured founder profiles and generates Python lambda expressions that encode binary screening rules. These rules execute deterministically across all 4,500 founders without additional LLM calls.

Metric	Threshold	Rationale
Precision lift	$>1.0$	Above-baseline success rate
p-value	$<0.05$	Statistical significance
Coverage	1%–50%	Sufficient power, meaningful discrimination

Table 3: Statistical validation criteria for rule filtering.

Method	Precision	Recall	F 0.5
Random Baseline	9.0%	9.0%	9.0%
Y Combinator	14.0%	6.9%	8.6%
Tier-1 VCs	23.0%	5.2%	10.7%
Claude-3.5-Haiku	15.8%	46.4%	18.2%
GPT-4o	30.0%	16.3%	25.7%
o3	43.2%	7.4%	21.5%

Table 4: Performance comparison on VCBench.

Figure 3: Rule quality distribution. Each point is a generated rule. Significant rules (green) cluster above the lift=1.0 threshold, with a tradeoff between lift and coverage.

Iter	Rule	Lift	Cov.
1	prior_exit	3.05 $\times$	3.16%
1	founder_scaled_company_200plus	2.80 $\times$	2.56%
1	worked_in_vc_pe	2.35 $\times$	3.47%
1	executive_chair_or_operating_partner	2.28 $\times$	2.71%
1	elite_qs_top10	2.18 $\times$	11.91%
2	cybersecurity_domain_depth	6.54 $\times$	0.38%

Table 5: Top rules by precision lift per iteration.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Rule Name	Description	Lift	Coverage	p-value
Iteration 1: Initial generation (no feedback)
prior_exit	Founder has $\geq$ 1 prior IPO or acquisition	3.05 $\times$	3.16%	$<$ .001
founder_scaled_200plus	Founded/co-founded company that grew to 200+ employees	2.80 $\times$	2.56%	$<$ .001
elite_qs_top10	Attended top-10 QS-ranked university	2.18 $\times$	11.91%	$<$ .001
Iteration 2: Gap-targeted generation (with feedback)
cybersecurity_domain_depth	Security industry + technical security role experience	6.54 $\times$	0.38%	$<$ .001

Appendix

Table 6: Representative generated rules with validation statistics. Iter 1 rules were generated without feedback; Iter 2 rules were guided by cluster-based gap analysis targeting underperforming subpopulations.

Metric	Iter 1	Iter 2
Rules generated	30	30
Significant (p $<$ 0.05)	19 (63%)	21 (70%)
Mean lift (sig. rules)	1.70 $\times$	2.58 $\times$
Median lift (sig. rules)	1.46 $\times$	1.88 $\times$
Rules with lift $>$ 2 $\times$	5	10
Rules with lift $>$ 3 $\times$	1	6

Appendix

Table 7: Rule generation statistics per iteration. Feedback-guided generation (Iter 2) produces more significant rules with higher mean precision lift, though with lower average coverage as the LLM targets more specialized subpopulations.

Rare-Event Prediction

See all 11 papers

Random Rule Forest (RRF): Interpretable and Manageable Ensembles of LLM-Generated Questions for Predicting Success from Unstructured Data
May 30, 2025Ben Griffin, Aaron Ontoyin Yin, Diego Vidaurre +4Rule-Based Classification Rare-Event Prediction
University of Oxford, United Kingdom · Vela Research, San Francisco, United States · Aarhus University, Aarhus, Denmark +2
IDEA: An Interpretable and Editable Decision-Making Framework for LLMs via Verbal-to-Numeric Calibration
Apr 14, 2026Yanji He, Yuxin Jiang, Yiwen Wu +3LLM Decision-Making Language Model Calibration
The Hong Kong University of Science and Technology (Guangzhou) · Huawei Technologies Co.,Ltd · The Hong Kong University of Science and Technology
Large Decision Models: Repurposing Large Language Models as Single-Pass Decision Makers
Oct 11, 2026Ziyi Chen, Yan Zhang, Jianhui Wei +2LLM Decision-Making Efficient Language Model Inference
University of Illinois Urbana-Champaign · National University of Singapore · Zhejiang University +2

May 30, 2025J/K move · Enter open · S save

Random Rule Forest (RRF): Interpretable and Manageable Ensembles of LLM-Generated Questions for Predicting Success from Unstructured Data

Ben Griffin, Aaron Ontoyin Yin, Diego Vidaurre, Ugur Koyluoglu+3

University of Oxford, United Kingdom · Vela Research, San Francisco, United States · Aarhus University, Aarhus, Denmark+2

Rule-Based Classification Rare-Event Prediction

Open paper AskPDF arXiv

Many high-stakes screening tasks require predicting rare outcomes from unstructured text, where errors are costly and decisions must be auditable. We introduce Random Rule Forest (RRF), an interpretable ensemble that uses a large language model (LLM) not as an end-to-end predictor but as a generator of simple YES/NO questions. Each question acts as a weak learner, and their responses are combined by a plain unit-weight vote into an auditable ``green-flags'' scorecard: enough independent positive signals indicate a higher chance of success. We argue this deliberate simplicity is a robust default when positives are scarce and learned weights are hard to estimate. We evaluate RRF in two low-base-rate domains. On early-stage startup screening from founder profiles, RRF produces a transparent scorecard whose precision is several times the base rate (with light expert input raising it further) and, unlike direct prompting, its operating point can be controlled directly. On an established Phase~I clinical-trial benchmark, RRF outperforms published baselines on the threshold-independent metrics PR-AUC and ROC-AUC. Together these show that LLMs can serve as auditable feature generators for high-stakes text-based decisions, combining transparency with competitive predictive performance.

Abstract

Rare-Event Prediction

Explore similar work

Random Rule Forest (RRF): Interpretable and Manageable Ensembles of LLM-Generated Questions for Predicting Success from Unstructured Data

IDEA: An Interpretable and Editable Decision-Making Framework for LLMs via Verbal-to-Numeric Calibration

Large Decision Models: Repurposing Large Language Models as Single-Pass Decision Makers