As LLM agents tackle increasingly complex tasks, a critical question is whether they can maintain strategic coherence over long horizons: planning under uncertainty, learning from delayed feedback, and adapting when early mistakes compound. We introduce YC-Bench, a benchmark that evaluates these capabilities by tasking an agent with running a simulated startup over a one-year horizon spanning hundreds of turns. The agent must manage employees, select task contracts, and maintain profitability in a partially observable environment where adversarial clients and growing payroll create compounding consequences for poor decisions. We evaluate 12 models, both proprietary and open source, across 3 seeds each. Only three models consistently surpass the starting capital of $200K, with Claude Opus 4.6 achieving the highest average final funds at $1.27 M, followed by GLM-5 at $1.21 M at 11× lower inference cost. Scratchpad usage, the sole mechanism for persisting information across context truncation, is the strongest predictor of success, and adversarial client detection is the primary failure mode, accounting for 47% of bankruptcies. Our analysis reveals that frontier models still fail through distinct failure modes such as over-parallelization, demonstrating the capability gaps for long-horizon performance. YC-Bench is open-source, reproducible, and configurable.
Figures & tables
Figure 1: Overview of YC-Bench . The agent interacts with the environment through CLI commands ( blue ) and receives structured observations ( green ). The environment tracks observable state (tasks, employees, finance, prestige, client trust) and one hidden element: adversarial clients whose work inflation must be inferred from repeated task failures.
Figure 2: Out of the 12 models that we benchmark on YC-Bench , 5 models are profitable and only 3 turn a substantial profit ( 5× profit). The figure plots the funds across time averaged across three seeds for each model. Comprehensive results can be found in Appendix D .
Figure 3: We observe that better models are able to build client trust over time by strategically selecting clients. What is surprising is smaller distilled models (Sonnet-4.6) do worse than contemporaries (Gemini-3-Flash) unlike VB.
Figure 4: Analyzing how the models deal with adversarial clients, who have appealing rewards when accepting a task but have a lot more work than claimed (scope creep).
Figure 5: Other than identifying adversarial clients there are two other primary task failure modes, especially wrong employee assignment. We observe that Kimi-K.5 is the most cost efficient, whereas the second highest ranking model, GLM-5 is substantially ( 10× ) better than Opus-4.6 on cost efficiency.
Figure 6: Representative failure moments for four models. Each panel shows the scratchpad state, agent action, and outcome. From left to right: Opus violates its own blacklist; Flash blindly accepts adversarial tasks through a rigid loop; Sonnet writes a correct rule and immediately ignores it; Grok diagnoses its bankruptcy but takes no corrective action.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Category
Command
Effect
Observe
company status
Funds, prestige, payroll
Observe
employee list
Names, tiers, salaries, productivity
Observe
market browse
Available tasks with client, reward, domains
Observe
task list
Accepted tasks with status and progress
Observe
task inspect --task-id T
Per-domain progress, deadline, assignments
Observe
client list
Client trust levels and tiers
Appendix
Table 1: Agent action space. All actions are CLI commands with arguments constructed as free-form text. Multiple actions may be issued per turn.
Category
Parameter
Value
Description
Simulation
Horizon
1 yr
Evaluation period
Simulation
Auto-advance
5 turns
Idle turns before forced advance
Simulation
Business hours
9–18
Weekdays only
Workforce
Employees
8
Fixed roster
Workforce
Initial funds
$200K
Starting capital
Workforce
Salary bump
1%/task
All assigned employees
Appendix
Table 2: Default configuration hyperparameters for YC-Bench .
Figure 7: Average final funds across three seeds for all 12 models and the greedy baseline, sorted by performance. Only three models (Claude Opus 4.6, GLM-5, GPT-5.4) consistently surpass the starting capital.
Figure 8: Monthly funds trajectory across three seeds.
Figure 9: Average number of tool calls per run for the top three and bottom three performing models, grouped by command type. Top models execute 3-4x more actions overall, with the largest gaps in task inspection and scratchpad usage.
Model
SP/100T
Inspect/Accept
Avg Conc.
Cmd/Turn
Claude Opus 4.6
5.6
1.10
2.0
1.41
GLM-5
2.7
0.26
1.5
2.16
GPT-5.4
10.6
1.43
2.5
2.63
Kimi-K2.5
0.6
0.00
1.3
1.18
Gemini 3 Flash
0.2
0.11
1.2
0.24
Gemini 3.1 Flash Lite
0.9
0.02
1.2
2.16
Appendix
Table 3: Behavioral statistics per model, averaged across 3 seeds and sorted by average final funds. The top three models use the scratchpad and task inspection significantly more than others. Sonnet’s average concurrency of 7.2 is a clear outlier, consistent with its high understaffing failure rate. GPT-5.4 Mini and Nano execute fewer than 0.1 commands per turn despite running thousands of turns, indicating most turns produce reasoning without action. SP/100T : scratchpad writes per 100 turns. Inspect/Accept : ratio of task inspect to task accept calls. Avg Conc. : mean simultaneously active tasks. Cmd/Turn : commands executed per turn.
Figure 10: Average final funds vs. K (number of retained turns), across three seeds.
Model
Bankrupt
Cost ($)
Tokens
Runtime (min)
Claude Opus 4.6
0/3
86.07 ± 41.17
16.7M
70 ± 36
GLM-5
0/3
7.62 ± 1.06
8.3M
40 ± 7
GPT-5.4
0/3
23.08 ± 7.07
9.5M
19 ± 5
Kimi-K2.5
1/3
1.79 ± 2.74
3.8M
21 ± 30
Gemini 3 Flash
0/3
1.83 ± 0.32
3.8M
6 ± 1
Gemini 3.1 Flash Lite
1/3
0.59 ± 0.22
2.6M
6 ± 2
Appendix
Table 4: Computational cost per run, averaged across three seeds ( μ±σ ). Models are sorted by average final funds.