A Multi-Agent Platform for Automated Enterprise Analytics and Insight Generation
Organizations: Rakuten India Enterprise Private Limited, Bengaluru, India
Abstract
This paper proposes a multi-agent framework built on CrewAI [1] for conversational business intelligence. Five specialized AI agents operate in a sequential pipeline to process natural language queries, retrieve and analyze data, generate visualizations via the Model Context Protocol (MCP) [2], and deliver actionable insights. The platform features a defense-in-depth security architecture for multi-tenant data isolation and a query parameterization mechanism for transforming conversational insights into reusable dashboard components. Evaluation across 300 end-to-end test cases spanning synthetic and production enterprise datasets demonstrates 95.3% functional accuracy, a mean response latency of 24 seconds, and a response quality score of 4.52/5.0 as assessed by an LLM-as-a-Judge framework, with a 93.0% hallucination-free rate, representing a 22.6 percentage point accuracy improvement and 20.2% quality gain over a single-agent baseline. Cross-model evaluation across four LLM backends and human expert validation confirm architectural generalizability and evaluator reliability. An ablation study confirms that the Data Analysis and Report Aggregation agents are the primary drivers of output quality.
Figures & tables
| Category | Tests | Passed | Pass Rate |
|---|---|---|---|
| Simple Aggregations | 65 | 64 | 98.5% |
| Filtered Aggregations | 35 | 35 | 100% |
| Time Series Analysis | 40 | 39 | 97.5% |
| Multi-Table JOINs | 35 | 33 | 94.3% |
| Complex Analytics | 30 | 26 | 86.7% |
| Edge Cases | 30 | 28 | 93.3% |
| Category | Mean | Min | Max | |
|---|---|---|---|---|
| Simple Aggregations | 65 | 20.8 | 8.2 | 68.3 |
| Filtered Aggregations | 35 | 16.2 | 9.1 | 24.5 |
| Time Series Analysis | 40 | 19.4 | 11.8 | 38.2 |
| Multi-Table JOINs | 35 | 26.8 | 13.2 | 52.1 |
| Complex Analytics | 30 | 38.6 | 16.4 | 118.5 |
| Edge Cases | 30 | 28.1 | 8.4 | 65.8 |
| Attack Vector | Detection Layer | Latency |
|---|---|---|
| SQL Injection (DROP) | Input Validation | 0.01s |
| Off-topic Query | Input Validation | 0.02s |
| DELETE Injection | Input Validation | 0.01s |
| UPDATE Injection | LLM Guardrail | 6.2s |
| UNION Injection | LLM Guardrail | 7.5s |
| Prompt Injection | LLM Guardrail | 6.3s |
| Dimension | Core ( =205) | All ( =260) |
|---|---|---|
| Overall Score | 4.52 | 4.38 |
| Factual Accuracy | 4.28 | 4.05 |
| Relevance | 4.85 | 4.62 |
| Completeness | 4.30 | 4.08 |
| Actionability | 4.60 | 4.42 |
| Language Quality | 4.94 | 4.88 |
| Metric | Core ( =205) | All ( =260) |
|---|---|---|
| Hallucination-Free Rate | 93.0% | 89.2% |
| Factual Reliability Score | 4.34/5.0 | 4.12/5.0 |
| Category | Multi-Agent | Single-Agent |
|---|---|---|
| Simple Aggregations | 64/65 (98.5%) | 61/65 (93.8%) |
| Filtered Aggregations | 35/35 (100%) | 29/35 (82.9%) |
| Time Series Analysis | 39/40 (97.5%) | 34/40 (85.0%) |
| Multi-Table JOINs | 33/35 (94.3%) | 26/35 (74.3%) |
| Complex Analytics | 26/30 (86.7%) | 15/30 (50.0%) |
| Edge Cases | 28/30 (93.3%) | 23/30 (76.7%) |
| Metric | Multi-Agent | Single-Agent |
|---|---|---|
| Overall Quality | 4.52/5.0 | 3.76/5.0 |
| Factual Accuracy | 4.28/5.0 | 3.48/5.0 |
| Relevance | 4.85/5.0 | 4.25/5.0 |
| Completeness | 4.30/5.0 | 3.40/5.0 |
| Actionability | 4.60/5.0 | 3.62/5.0 |
| Language Quality | 4.94/5.0 | 4.08/5.0 |
| Configuration | Quality | Lang. | Halluc-Free |
|---|---|---|---|
| Full pipeline (5 agents) | 4.52 | 4.94 | 93.0% |
| w/o Follow-up + Chart | 4.50 | 4.94 | 93.0% |
| w/o Report Aggregation | 3.86 | 3.42 | 90.4% |
| w/o Data Analysis | 3.38 | 4.80 | 85.0% |
| Model | Acc. | Quality | Halluc-Free | Latency |
|---|---|---|---|---|
| GPT-4.1 | 95.3% | 4.52 | 93.0% | 23.8s |
| Claude Sonnet 4 | 93.0% | 4.44 | 90.8% | 28.2s |
| Gemini 2.5 Flash | 86.3% | 4.02 | 82.1% | 17.4s |
| Llama 3.1 70B | 79.7% | 3.65 | 74.2% | 35.8s |