Enterprise adoption of LLMs in agentic workflows is gated by evaluations that span a wide arc of abilities: CRM operations, heterogeneous-document retrieval, personal data understanding, multi-turn planning, tool use, function calling, long-horizon research, and long-context software engineering. These evaluations live today in separate repositories with divergent runners, dependencies, and scoring conventions, which makes cross-benchmark comparison difficult and obscures what a single aggregate number actually represents. We release EGI-BENCH, a unified LLM agent benchmark suite that consolidates eight enterprise facing agent benchmarks (CRMARENA, HERB, PERSONABENCH, USERBENCH, MCP-UNIVERSE, MCPEVAL, BENCHFORCE, LOCOBENCH) behind a single command-line interface with two runtime-budgeted execution modes. On top of the suite we define the Enterprise General Intelligence (EGI) Score, a six-dimensional framework (M1-M6) whose buckets are populated by fine-grained sub-metrics routed from each benchmark rather than by benchmark-level averages. We describe the suite, the sub-metric routing, and the quick/deep budgets, and report results for fourteen frontier and cost-efficient LLMs spanning the GPT, Claude, and Gemini families under quick mode. Top-line EGI ranges from 39.6 (GPT-4o mini) to 65.6 (gpt-5.5), and no single model dominates every capability axis: gpt-5.5 leads task outcome, planning, and retrieval grounding; Claude Sonnet 4.5 leads tool actioning and safety/abstention; and Claude Haiku 4.5 leads engineering quality. Code is available at https://github.com/SalesforceAIResearch/egi-bench.
Table 1: Domain coverage, task count, and M 1 –M 6 enterprise-metric coverage across representative agent/LLM benchmarks. ✓ = the benchmark evaluates this dimension as a core part of its design; ∼ = exercised as part of tasks but not a primary axis; ✗ = outside scope.
Metric
Weight
What it measures
M 1 Task Outcome Accuracy
28%
Did the agent produce the correct business outcome or final answer?
M 2 Tool Actioning Reliability
22%
Correct tool selection, parameters, ordering; no extraneous calls.
M 3 Planning & Workflow
16%
Multi-step and multi-turn execution that is step-efficient.
Abstain when appropriate, flag policy violations, no unsafe actions.
M 6 Engineering & Maintainability
6%
Code quality and structural/process discipline.
Table 2: The six enterprise metrics (M1–M6) and their weights in the EGI score. Weights are inherited from the internal PR #28 proposal and are applied uniformly across the suite.
Figure 1: M 1 –M 6 scores across fourteen LLMs under quick mode. Models are ordered by EGI ; each bucket is a cluster of bars. The M 3 (Planning) dimension is uniformly low across models, while M 2 (Tool Actioning) is the strongest axis for nearly every frontier model. The best M 2 (Tool Actioning, 87.17) and M 5 (Safety & Abstention, 80.28) are held by Claude Sonnet 4.5, while the best M 1 (Task Outcome, 82.95), M 3 (Planning, 30.89), and M 4 (Retrieval & Grounding, 54.86) are all held by gpt-5.5 , illustrating that top-line EGI can mask capability-level specialization.
Figure 4
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Benchmark
Domain
Tasks (quick / deep)
Primary metric
CRMArena
CRM operations
9 types × 6 / × 3
Task accuracy
HERB
Enterprise search
80 Qs / full
F1 + content correctness
PersonaBench
Personal data QA
3 people × 2 / × 4 noises
QA accuracy
UserBench
Travel planning
3 envs × 60 / all envs
Micro-avg. reward
MCP-Universe
MCP tool use
1 dom. × 20 / 3 dom.
Sub-eval pass rate
MCPEval
MCP tool use
1 dom. × 50 / 3 dom. × 50
Flex. avg. score
Appendix
Table 4: The eight benchmarks unified under EGI-Bench . Task counts reflect the quick and deep mode profiles defined in benchmark_config.yaml . Full benchmark descriptions are in Section 4 .
CRMArena (PVI), HERB (unanswerable null rate), Benchforce (unsafe-failure gap)
M 6
LoCoBench (software_engineering, code_quality)
Appendix
Table 5: Which benchmarks contribute to each enterprise metric after fine-grained decomposition. Every benchmark feeds at least one bucket and every bucket receives at least two routed sub-metrics; M 6 is the only bucket whose sub-metrics come from a single benchmark ( LoCoBench ).
Benchmark
Sub-metric
Bucket
Rationale
CRMArena
NCR (Case Routing)
M 1
Business action correctness
HTU (Handle Time)
M 1
Analytical task outcome
TCU (Transfer Count)
M 1
Analytical task outcome
TII (Top Issue ID)
M 1
Analytical task outcome
BRI (Best Region ID)
M 1
Analytical task outcome
MTA (Monthly Trend)
M 1
Task outcome
Appendix
Table 6: Fine-grained sub-metric routing. Each row shows a native sub-metric reported by a benchmark, the enterprise metric (M 1 –M 6 ) it is routed into, and a short rationale. No benchmark’s overall score is consumed directly; the routing is the unit of aggregation.
Enterprise agents increasingly operate inside workspaces: they read heterogeneous files, invoke tools, and deliver business artifacts. We introduce EnterpriseClawBench, an enterprise agent benchmark constructed from proprietary, real-world agent sessions. Starting from a large archive of workplace sessions, the EnterpriseClawBench produces 852 reproducible tasks, each paired with recovered fixtures, rewritten prompts, role classes, skill subclasses, hard rules, and semantic rubrics. Because the sessions contain internal enterprise content, we do not release the benchmark data; instead, our reusable contribution is the construction and evaluation protocol. On EnterpriseClawBench, the best configuration reaches only 0.663 (Codex with GPT-5.5). These results show that enterprise agent evaluation must report harness--model combinations, artifact delivery, visual quality, cost, runtime, and skill-transfer behavior, rather than collapsing performance into a single score. Code: https://github.com/FrontisAI/EnterpriseClawBench
LLM agents for enterprise systems of record cannot be evaluated on customer production data, and no existing substitute provides ground truth. We present the Era by Eon Benchmark for evaluating LLM agents that use enterprise tools. The benchmark is built around a complete fictional company. It includes product simulators, company-specific internal databases, benchmark questions, and computed answer keys. Industry, company size, business model, application portfolio, and a seed define each company. One seeded entity graph supplies shared company data to simulators of Salesforce, Zendesk, Slack, Gong, and other products. A questionconditioned generator creates the schemas and records for internal databases. It takes shared entities, keys, and values from the same graph before generating database-specific facts. Both mechanisms therefore describe one consistent enterprise estate. Every expected answer is computed from the final records, so grading is exact. Design and answer-key checks validate the internal databases. A realism scorecard and adversarial detector validate the entity graph. Across 23 generated companies, the mean realism score rose from 61.8 to 97.0, with zero records flagged as synthetic. In the reported simulator-track comparison, nine models answered the same 33 questions three times each. Accuracy estimates ranged from 42.4% to 76.8%, and three of 36 pairwise differences remained supported after correction.
Benjamin Gruenbaum, Doron Porat, Assaf Natanzon +3
Large Language Models (LLMs) are increasingly deployed as agents that interact with stateful environments over multiple steps: gathering hidden information, composing tool calls, and committing state changes. We refer to this capability as multi-step tool use. Existing benchmarks have advanced tool-use agent evaluation, but often focus on isolated API calls, short trajectories, or settings that are difficult to scale or control. We introduce E-Bench, a fully synthetic benchmark with 323 state-changing tasks across three product domains: Honor of Kings, QQ Music, and Tencent Meeting. E-Bench decouples environment synthesis from task synthesis: graph-guided database filling builds reusable, orphan-free product environments, while generator-solver asymmetry creates tasks with both an information gap and a tool gap, requiring agents to discover hidden data and compose multiple tool calls before changing state. Outcomes are graded deterministically by database-state diffs. Since both environments and tasks are synthetic, E-Bench is controllable at the environment level and scalable at the task level. Benchmarking 11 cutting-edge LLMs shows that multi-step tool use remains challenging: Pass^3 stays below 60% for the strongest models, and even with code execution in the E-Bench-Code extension, reliability (Pass^3) remains below 70%.
Weihuang Zheng, Tianyuan Zou, Eileen Ye +5
Hunyuan Team, Tencent · Institute for AI Industry Research, Tsinghua University · School of Computer Science and Engineering, Southeast University