DealsFlow: An Evidence-Gated Agentic Harness for Venture Capital Deal Triage

Ming Ze Tang; David Cutting; Boyd Carson; Vasiliki Carson

cs.AIOct 23, 2026

DealsFlow: An Evidence-Gated Agentic Harness for Venture Capital Deal Triage

Authors: Ming Ze Tang, David Cutting, Boyd Carson, Vasiliki Carson

Organizations: Queen’s University Belfast, Belfast, United Kingdom · Sapphire Capital Partners, Belfast, United Kingdom

Abstract

Large language models (LLMs) can read startup pitch decks, but letting them emit investment scores directly invites score inflation, run-to-run variance and rationales that are hard to audit. We describe DealsFlow, a human-in-the-loop agentic harness deployed at a UK early-stage fund. The LLM reads the deck, cites evidence and rates each category; code combines the ratings, caps them and only ever lowers them. Three mechanisms implement this: a stage-aware corroboration ceiling (78, 70 or 60 out of 100) on factual categories without external evidence; a sceptic call that may lower but never raise a score; and a pre-fetch pattern that retrieves and indexes all evidence before a single generation pass. We evaluate on $N=243$ SEIS/EIS investments with recorded outcomes. Because recent investments have unresolved outcomes, we use the 142 that are at least three years old. There, the full pipeline's score has Spearman $ρ=0.45$ (95% CI 0.32 to 0.57) with the total-value multiple and AUC 0.71 for identifying write-offs, against $ρ=0.38$ and AUC 0.68 for an unconstrained LLM. The gain is small and comes mostly from the sceptic call; we did not test whether its lower-only rule matters. Investments that later exceeded $2\times$ had 64% of factual categories corroborated, against 38% for write-offs (omnibus Kruskal-Wallis test over five outcome groups, $p=0.031$ ), and our retrieval was not date-restricted, so these figures may overstate screening skill. In a study of 20 inbound applications, analyst time per application fell from 44.7 to 18.9 minutes including verification and correction, and 84% of 240 cited claims were fully supported by their sources, but 70% of reports needed at least one correction.

Figures & tables

Figure 1. DealsFlow pipeline. (1) Ingestion and evidence pre-fetch; (2) deterministic triage with the corroboration ceiling and the lower-only sceptic pass; (3) parallel per-criterion evaluation for applications that pass the gate; (4) audit record and mandatory human approval. The ceiling values are stage-dependent (§ 3 ). Components in stage 3 and the portfolio-memory branch of stage 1 are described in § 3.6 but not evaluated in this paper. Flow diagram in four horizontal bands. Band 1: an inbound PDF deck is extracted and profiled, then a context orchestrator queries Companies House, capped web research and portfolio memory and produces an enriched evidence bundle. Band 2: deterministic scoring, then a decision on whether each claim is corroborated. Uncorroborated claims are capped at a stage-dependent ceiling of 78, 70 or 60; corroborated claims keep the raw category score. Both paths feed an asymmetric sceptic pass that can only lower scores, then a recalculated triage score and a gate. Band 3: a parallel fan-out to criterion agents and a specialist panel, a meta-agent that synthesises scenarios, and probabilistic estimates at p10, p50 and p90. Band 4: an append-only audit record, an investment committee review gate, and a final investment decision reached only through human approval.

Outcome class	$n$	Holding period (yrs)	Aggregate multiple	Individual range
Active breakout	15	5.50	4.61 $\times$	3.12–14.74 $\times$
Active up-valued	50	4.06	1.67 $\times$	1.09–2.80 $\times$
Active held at cost	102	1.94	1.00 $\times$	1.00 $\times$
Active impaired	14	4.41	0.54 $\times$	0.04–0.99 $\times$
Fully exited (profitable)	3	2.15	4.44 $\times$	1.04–5.81 $\times$
Fully exited (at cost)	2	1.80	1.00 $\times$	1.00 $\times$

Table 1. Portfolio by outcome class (

N=243

, valuation date 8 July 2026). Holding period is the mean, in years, from first investment to exit, or to the valuation date for active holdings. Aggregate multiple is total value over capital invested within the class.

Bracket	Deals	Value (£m)	Share of total value
Top 1%	2	74.1	17.8%
Top 5%	10	211.1	50.6%
Top 10%	20	274.7	65.8%
Top 25%	49	360.8	86.5%

Table 2. Value concentration among the 196 positions with non-zero total value. Shares are of total portfolio value (£417.22m, realised plus unrealised).

Figure 2. Mean holding period against aggregate multiple by outcome class; bubble area is proportional to capital invested. Generated from the same records as Table 1 . Bubble chart. The horizontal axis is mean holding period in years, from about 1.8 to 5.5, and the vertical axis is aggregate multiple, from 0 to about 4.6 times. Active breakout is top right at about 5.5 years and 4.6 times. Fully exited profitable is top left at about 2.2 years and 4.4 times. Active up-valued is near 4 years and 1.7 times and is the largest bubble. Active held at cost and fully exited at cost sit at 1 times, under 2 years. Active impaired and fully exited loss sit below 1 times at about 4.4 and 4.6 years. Written off sits at 0 times at about 3.4 years.

Years since investment	$n$	Written off	Held at cost	$>2\times$
$<2$	73	0 (0%)	59 (81%)	4
2–3	28	4 (14%)	17 (61%)	0
3–5	93	28 (30%)	24 (26%)	7
$\geq 5$	49	15 (31%)	2 (4%)	15

Table 3. Outcomes by time since first investment (valuation date 8 July 2026).

Outcome class	$n$	Corroborated / total	Rate
$>2\times$	22	56 / 88	63.6%
$>1\times$ to $2\times$	31	54 / 124	43.5%
At cost	27	50 / 108	46.3%
Impaired / loss (excl. write-offs)	19	35 / 76	46.1%
Written off	43	65 / 172	37.8%
Overall	142	260 / 568	45.8%

Table 4. Corroboration by outcome class (142 mature deals). Rate is the pooled share of the four factual categories per deal (market, financials, traction, differentiation) labelled external or registry ;

n

is the number of deals, so each class has

4n

categories. A Kruskal–Wallis test on the 142 per-deal corroboration fractions, rather than on the 568 categories as if independent, gives

H(4)=10.65

p=0.031

Configuration	Spearman $\rho$	AUC 0	AUC 2	Mean score, write-offs / $>2\times$
Unconstrained LLM	0.381 [0.239, 0.512]	0.681 [0.588, 0.768]	0.727 [0.608, 0.838]	67.5 / 79.8
Ceiling only	0.384 [0.238, 0.517]	0.681 [0.586, 0.768]	0.751 [0.628, 0.864]	66.9 / 78.4
Sceptic only	0.435 [0.301, 0.555]	0.705 [0.616, 0.787]	0.762 [0.651, 0.864]	63.3 / 78.6
Full (ceiling + sceptic)	0.451 [0.317, 0.570]	0.709 [0.620, 0.790]	0.791 [0.678, 0.892]	62.7 / 77.2
Corroboration-only	0.216 [0.048, 0.372]	0.605 [0.503, 0.702]	0.696 [0.579, 0.804]	–
Vintage only	$-$ 0.01 [ $-$ 0.19, 0.17]	0.45 [0.34, 0.55]	0.64 [0.50, 0.77]	–

Table 5. Results on the 142 mature deals. 95% bootstrap intervals (10,000 resamples) in brackets. AUC 0 : write-off vs. rest; AUC 2 : multiple

>2\times

vs. rest. The corroboration-only row uses the corroboration labels from the scoring runs; the vintage, structured-field and initial-cheque rows are computed from the portfolio records alone. Each deal was scored once per configuration.

Comparison	Difference in $\rho$	Difference in AUC 0	Difference in AUC 2
Full $-$ Unconstrained	$+$ 0.070 [0.035, 0.105]	$+$ 0.027 [0.006, 0.050]	$+$ 0.064 [0.041, 0.091]
Full $-$ Ceiling only	$+$ 0.067 [0.034, 0.103]	$+$ 0.028 [0.006, 0.051]	$+$ 0.040 [0.015, 0.067]
Full $-$ Sceptic only	$+$ 0.016 [ $-$ 0.006, 0.040]	$+$ 0.004 [ $-$ 0.008, 0.016]	$+$ 0.029 [0.011, 0.051]
Full $-$ Corroboration-only	$+$ 0.235 [0.037, 0.429]	$+$ 0.104 [ $-$ 0.034, 0.242]	$+$ 0.095 [ $-$ 0.037, 0.230]

Table 6. Paired bootstrap differences between configurations on the same 142 deals (10,000 resamples; 95% intervals in brackets).

Outcome group	Mean $\pm$ SD	Median	Q1–Q3
$>2\times$	77.2 $\pm$ 8.3	78.6	73.0–81.6
$>1\times$ to $2\times$	71.4 $\pm$ 7.4	71.3	67.5–76.8
At cost	69.2 $\pm$ 9.3	70.9	60.8–76.2
Impaired / loss	64.4 $\pm$ 13.3	65.0	55.2–78.1
Written off	62.7 $\pm$ 11.0	65.2	58.0–70.3

Table 7. Composite score of the full pipeline by outcome group (142 deals).

Measure	Result
Deals affected by the ceiling	69/142 (48.6%)
Mean reduction from the ceiling, all deals	1.23 points
Deals affected by the sceptic	135/142 (95.1%)
Mean further reduction from the sceptic, all deals	2.99 points
Score increases from either guardrail	0

Table 8. Effect of the guardrails on the 142 composite scores.

Measure	Mean $\pm$ SD	95% CI for mean
Manual triage	44.69 $\pm$ 11.02	40.05–49.53
DealsFlow machine processing, unattended	6.13 $\pm$ 1.58	5.51–6.85
Analyst: reading the report	4.63 $\pm$ 1.26	4.10–5.17
Analyst: verifying sources	7.04 $\pm$ 1.84	6.27–7.86
Analyst: correcting errors	2.41 $\pm$ 1.31	1.85–2.96
Analyst: completing the final assessment	4.77 $\pm$ 1.14	4.30–5.27

Table 9. Time per application in minutes, 20 inbound applications (mean

\pm

standard deviation, with the 95% confidence interval for the mean). Elapsed time adds the machine time to the analyst time, assuming the analyst starts when the machine run ends. The saving is the paired difference between manual triage and total analyst time with DealsFlow, for the same application and different analysts.

Measure	Result
Reports assessed / distinct claims checked	20 / 240
Claims fully supported by the cited source	202 / 240 (84.2%)
Claims partly supported	22 / 240 (9.2%)
Claims not supported	10 / 240 (4.2%)
Cited source not accessible	6 / 240 (2.5%)
External or registry labels confirmed by a cited source	42 / 48 (87.5%)

Table 10. Factual-support assessment of the 20 generated reports. Counts are after reconciliation, except agreement, which is on the reviewers’ initial ratings.

Correction type	Count
Qualifying overstated or partly supported claims	12
Removing or replacing unsupported claims	8
Adding omitted material issues	6
Repairing inaccessible or incorrect citations	3

Table 11. Corrections made to the 20 reports, by type.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Item	Value
Scoring model	GPT-6 Luna ( gpt-6-luna ) via Microsoft Azure, EU region; knowledge cutoff 18 May 2026 ( OpenAI, 2026 )
Sceptic model	Same model; second call; reductions only, at most 20 points per category
Sampling	Temperature 0.1 on every call; all other sampling and reasoning settings at the Azure defaults
Web queries per deck	At most 10 by construction (hard cap 12)
Search	ddgs metasearch, 5 results per query; top 2 pages crawled ( crawl4ai ), 4,000 characters each; results cached for 24 hours
Date filter on search	None in any run

Appendix

Table 12. Settings used in the experiments.

Category (weight)	What is judged	Counts as corroboration	Ceiling
Vision & mission (1.0)	Clarity and ambition of the vision; whether the mission follows from the problem	Judgement-based; rarely externally checkable	No
Market potential (1.5)	Problem severity and realism; market-size claims sanity-checked; pricing realism	A web source on market data, competitors or customers	Yes
Financials & valuation (1.5)	Credibility of the revenue trajectory; valuation reasonable for stage and traction; quality of the raise against the plan	Companies House data, or a public funding or press source	Yes
Founders (2.0)	Depth and relevance of expertise for this problem, from the deck and web research; red flags	Founder web sources or Companies House officer records	No
Traction (1.5)	Real demand (revenue, pilots, signed customers, growth) rather than vanity metrics	Customer, review or press sources, or registry filings	Yes
Differentiation (1.5)	Unique edge or moat; defensibility against alternatives	A web source on competitors or alternatives	Yes

Appendix

Table 13. Category definitions, corroboration sources and ceiling status.

Requirement	How addressed here
Temporal sanitation (weights)	Model cutoff (18 May 2026) disclosed; it postdates most outcomes and is not controlled; residual leakage reported (§ 7 ).
Temporal sanitation (retrieval)	No date filter on web search; Companies House status is current; residual leakage reported (§ 7 ).
Dynamic universe / survivorship	All 243 funded deals included, including 47 write-offs and 10 loss exits; primary set restricted to 142 deals at least 3 years old to reduce the influence of very recent investments (outcomes remain censored) (§ 5.1 ); rejected applications unobservable (§ 7 ).
Rationale robustness	Each category cites numbered sources; the stored source list (title and URL) permits source-level audit, and analysts check whether cited sources support the claims (RQ5).
Epistemic calibration	Stage-3 survival estimates are not evaluated in this paper (§ 3.6 ).
Cost	Total analyst time including verification and correction, and machine time, per application (RQ4); inference cost not measured.

Appendix

Table 14. Evaluation against the structural validity framework of ( Kong et al., 2026 ) .

A $\backslash$ B	Supported	Partly	Unsupported	Inaccessible
Supported	195	6	1	1
Partly supported	4	15	2	0
Unsupported	1	2	7	0
Inaccessible	0	0	0	6

Appendix

Table 15. Initial ratings of the 240 claims by Reviewer A (rows) and Reviewer B (columns).

Financial Language Model Evaluation

See all 30 papers

Feb 19, 2026

DIALECTIC: A Multi-Agent System for Startup Evaluation

Venture capital (VC) investors face a large number of investment opportunities but only invest in few of these, with even fewer ending up successful. Early-stage screening of opportunities is often limited by investor bandwidth, demanding tradeoffs between evaluation diligence and number of opportunities assessed. To ease this tradeoff, we introduce DIALECTIC, an LLM-based multi-agent system for startup evaluation. DIALECTIC first gathers factual knowledge about a startup and organizes these facts into a hierarchical question tree. It then synthesizes the facts into natural-language arguments for and against an investment and iteratively critiques and refines these arguments through a simulated debate, which surfaces only the most convincing arguments. Our system also produces numeric decision scores that allow investors to rank and thus efficiently prioritize opportunities. We evaluate DIALECTIC through backtesting on real investment opportunities aggregated from five VC funds, showing that DIALECTIC matches the precision of human VCs in predicting startup success.

Jae Yoon Bae, Simon Malberg, Joyce Galang +2

Technical University of Munich · Earlybird Venture Capital · UVC Partners

Multi-Agent LLM Systems LLM Decision-Making

Ask PDF

Sep 30, 2026

Better Deck or Different Judge? Evaluating Agentic Harness Gains in Corporate and Investment Banking

Corporate and investment banking teams use presentations to support credit decisions and advise clients on financing and transactions. Producing these decks requires reconciling financial data, tracing sources and turning analysis into a recommendation. We retrospectively study the development of an agentic harness combining a 27B language model, financial calculations, narrative templates and validation checks. LLM judges guide engineering changes and assess the resulting decks, raising the question of whether higher scores reflect better documents or changes in grading. In shared-session text-only grading with template markers removed, five judges score the complete system 20.4 to 33.6 points out of 95 above the same model generating directly from a short prompt. Every judge scores the system higher on all seventeen development deliverables. Margins against direct Opus generation from a short prompt range from -4.7 to +0.8 points. Judges agree on broad progress across development rounds but agree less on final-deck rankings than on pooled scores. Repeated grading also shifts scores on unchanged decks, making small improvements difficult to distinguish from judge variability.

Ludovic Gibert, Matis Despujols, Andre-Louis Rochet

AIData2Action · TW3 Partners

LLM-as-a-Judge Long-Form Document Generation LLM Agent Evaluation LLM Agent Harnesses

Ask PDF

Aug 6, 2026

Evaluating Investment Logic in Large Language Models: A Real-World Benchmark Towards Personalzied Financial Agents

Investment competence is inherently personalized: the same market evidence can justify different actions for investors with different goals, horizons, portfolios, and risk boundaries. Yet financial LLMs are evaluated either by static question answering or by terminal profit and loss. The former omits agency; the latter cannot reveal whether a profitable action was grounded, profile-consistent, or merely lucky. We ask whether the community is using the wrong ruler for consequential agents. We introduce InvestLogicBench, a process-native benchmark containing 201,247 documented decisions from 151 real-world investors. Each episode instantiates a \textbf{P

\rightarrow

\rightarrow

\rightarrow

\rightarrow

O} trace: investor Profile, observable market Events, investment Reasoning, executable Decision, and delayed Outcome. The release includes profile construction, point-in-time event binding, structured logic, horizons, outcomes, and post-mortems, and supports comprehension, profile-conditioned generation, and end-to-end replay. Across four leading LLMs, logical plausibility remains near 4/5 while event grounding is only 0.8--2.8/5; return and process quality also disagree. These results expose polished but weakly grounded reasoning that outcome-only evaluation hides. We further argue that P

\rightarrow

\rightarrow

\rightarrow

\rightarrow

O should be a data-system interface, requiring versioned profiles, temporal provenance, inspectable retrieval, decision ledgers, and replayable outcomes. Finance is our stress test for a broader class of personalized, consequential agents.

Yuanhong Jiang, Jingjie Zou, Zhenghong Lin +4

Financial Decision-Making Financial Reasoning in Language Models LLM Agent Evaluation AI Agent Benchmarks LLM Decision-Making AI-Assisted Decision Making+2

Ask PDF

Abstract

Financial Language Model Evaluation

Explore similar work

DIALECTIC: A Multi-Agent System for Startup Evaluation

Better Deck or Different Judge? Evaluating Agentic Harness Gains in Corporate and Investment Banking

Evaluating Investment Logic in Large Language Models: A Real-World Benchmark Towards Personalzied Financial Agents