cs.CYJul 6, 2026

KumbhDoot: A Scale-Ready, LLM-Bounded Architecture for Mass-Gathering Public-Service Assistants

Authors: Saurabh Sakalkar, Abhishek Singh, Ramesh Raskar

Organizations: Cisco · Project Nanda · MIT

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Mass religious gatherings such as the Kumbh Mela concentrate tens of millions of people into a single region over a few weeks, producing intense, repetitive, multilingual, and safety-critical demand for information. The default response, a conversational assistant that routes every query to a large language model (LLM), is poorly matched to this setting: it is costly at scale, slow on emergency paths, prone to hallucination on facts that can cause physical harm, and unusable when connectivity fails. We describe KumbhDoot, an agentic pilgrim assistant for the Nashik Simhastha Kumbh Mela built on a different principle. It operates on a foundational design principle that prioritizes semantic similarity over starting with an LLM. Generative models are invoked only in instances where similarity-based retrieval is insufficient to produce a correct answer. The system utilizes a "semantic cache": an embedding-indexed store as a single retrieval primitive, which handles intent routing, answer caching, offline lookups, and multi-agent retrieval. A custom three-tier agent architecture operates directly on this store, ensuring decision paths remain inspectable and avoiding the use of generic multi-agent frameworks that would trigger implicit per-step LLM calls. We present the architecture, an analytical cost model for its per-query economics, and an honest account of where similarity is sufficient and where generative reasoning remains necessary. We argue that for bounded, high-stakes, low-connectivity public-service domains, a similarity-first and LLM-bounded design is not merely cheaper but architecturally more appropriate than an LLM-default one.

Figures & tables

 RequirementPrincipal mechanismSection
R1Cost proportional to noveltyParaphrase-aware response cache; answers formatted from records without an LLM4.3 , 4.5
R2Immediate emergency handlingKeyword-triggered, rule-based fast path ahead of intent routing4.4
R3Grounding in approved contentInfoBins with source and validity metadata; coded formatters; citation validation4.2 , 4.5
R4Offline operationOn-device SQLite bundle with FTS5 and pre-computed embeddings4.7
R5Data minimizationLocal-first storage, coarse and consented location, bucketed attributes7
R6Aggregate operational insightObservability across LLM and non-LLM paths; aggregate dashboards4.1
Table 1: Requirements of the deployment setting and the mechanisms that address them.
Figure 1: Layered runtime. A query is answered by the lowest layer able to answer it reliably and moves upward only when that layer cannot. Observability spans all layers and covers both LLM and non-LLM paths.
FunctionKeysPayload
Knowledge baseEmbeddings of InfoBin contentInfoBin records
Intent classifierEmbeddings of labeled exemplar queriesIntent and domain labels
Response cacheEmbeddings of previously answered queriesApproved answers
Offline bundleDevice-resident subset of InfoBin embeddings, with an FTS5 indexInfoBin records
Per-agent memoryEmbeddings of records in an agent’s domain partitionRecords scoped to that agent
Table 2: Five functions expressed as retrieval against different partitions of the semantic cache.
Figure 2: One retrieval primitive in place of five components. Each function is cosine similarity against a different partition of one embedding-indexed store; the variety lies in the partition, not the algorithm.
 ServerDevice
StorageIn-memory partitions; Redis for query-to-answer pairsSQLite bundle ( ≈\approx 700 KB)
EmbeddingsHigher-dimensional; updated with live contentLower-dimensional; pre-computed, stored as blobs
Lexical index—SQLite FTS5
RetrievalDense retrieval, fusion, rerankingFTS5 and dense scores combined by rank fusion
Query embeddingServer-side encoderQuantized on-device encoder if present; otherwise word-overlap matching against pre-computed query embeddings
Content updatesContinuousAt build time (live synchronization not yet implemented)
Table 3: Realization of the semantic cache on the two tiers.
Figure 3: Escalation ladder. Methods are tried in order of cost. Paths drawn in the dark accent return without a generative model; the warm-colored path is the only one that calls an LLM.
Figure 4: Three-tier orchestration. All tiers retrieve through the same primitive, each over its own partition; only four of the domain agents are shown. Sub-agents annotate retrieved items with the fields _actionable , _card_type , _is_emergency , and _name_match .
StageLLM-defaultConventional RAGKumbhDoot
Intent classificationLLM—Nearest-neighbor over exemplars
Complexity assessmentLLM—Keyword and length heuristic
DecompositionLLM—Hand-written orchestrator
RetrievalLLM-directed toolsDense retrieverHybrid retrieval with reranking
Answer synthesisLLMLLMCoded formatter; LLM only when a gate fires
Table 4: Where a generative model is invoked in three pipeline designs.
Figure 5: Three workload bands. Each query should be answered in the cheapest band that can answer it correctly. Hallucination is reduced rather than eliminated, because Band C still calls a model.
ClaimPlanned measurement
Most traffic needs no generative modelShare of queries answered in Bands A and B on a representative query distribution, with full per-step tracing
Cost and latency by bandTokens, cost, and latency per step on a tiered benchmark of simple, multi-record, and multi-domain queries, each paired with paraphrases
Advantage over alternativesComparison against both an LLM-at-every-step pipeline and a conventional RAG pipeline that uses a model only for synthesis
Multilingual routing accuracyAccuracy and threshold calibration on a labeled test set in English, Hindi, Marathi, and code-mixed input, including speech-recognition noise
Offline usefulnessMinimum useful bundle size and on-device hit rate, measured by replaying representative traffic
Table 5: Claims that remain to be measured and the planned protocol for each.
More in this topic

LLM Agent Serving

Large language model agent serving runs tool-using, multi-step agents efficiently by coordinating inference, memory, and workflow execution.

See all 37 papers

Explore similar work

CardsList
Mar 23, 2026

Reasoner-Executor-Synthesizer: Scalable Agentic Architecture with Static O(1) Context Window

Large Language Models (LLMs) deployed as autonomous agents commonly use Retrieval-Augmented Generation (RAG), feeding retrieved documents into the context window, which creates two problems: the risk of hallucination grows with context length, and token cost scales linearly with dataset size. We propose the Reasoner-Executor-Synthesizer (RES) architecture, a three-layer design that strictly separates intent parsing (Reasoner), deterministic data retrieval and aggregation (Executor), and narrative generation (Synthesizer). The Executor uses zero LLM tokens and passes only fixed-size statistical summaries to the Synthesizer. We formally prove that RES achieves O(1) token complexity with respect to dataset size, and validate this on ScholarSearch, a scholarly research assistant backed by the Crossref API (130M+ articles). Across 100 benchmark runs, RES achieves a mean token cost of 1,574 tokens regardless of whether the dataset contains 42,000 or 16.3 million articles. The architecture eliminates data hallucination by construction: the LLM never sees raw records. KEYWORDS LLM agents; agentic architecture; hallucination elimination; token optimization; context window; retrieval-augmented generation; deterministic execution; scholarly metadata; Crossref API; O(1) complexity.

Ivan Dobrovolskyi

Walmart Global Tech, Sunnyvale, CA, USA

Agentic RAGLLM Hallucination MitigationRetrieval-Augmented GenerationLLM Agent Harnesses
AskPDF
Oct 10, 2026

Persistent LLM Orchestration of Heterogeneous Robot Teams via Partial Delegation to Multi-Agent Protocols

Natural language specifications let non-expert users task robot teams, and large language models (LLMs) have recently proven effective at grounding such semantic specifications into plans for heterogeneous teams. Existing approaches assume missions that terminate and environments that, while unknown, do not change, but both assumptions fail in many emerging applications. For example, a disaster-response mission does not terminate, and its environment changes continuously as hazards spread and evacuees move. We identify three ways an LLM orchestrator breaks in this setting: its prompt grows without bound, its reasoning degrades as agents and targets accumulate, and it stops actively monitoring after covering the search space once. We build on the successful framework of grounded generative intelligence, and address each issue with a classical idea. A Markovian state summary, rebuilt from the semantic map at every invocation, replaces the growing interaction history. Partial delegation lets the orchestrator assign a subset of robots to an autonomous multi-agent protocol and reserve its reasoning for generating subtasks that require detailed parameters and careful ordering. Lastly, observation staleness, surfaced explicitly in the prompt, together with task-and event-triggered re-prompting, keeps the team vigilant. In a simulated urban search-and-rescue mission with spreading fire and moving evacuees, our framework rescues 98% of civilians versus 89% for the strongest baseline while using six times fewer tokens. Additional ablations isolate the contribution of each mechanism.

Maria G. Mendoza, Victoria M. Tuck, Zachary Ravichandran +2

Department of Mechanical Engineering (M.G.M.) and Department of Electrical Engineering and Computer Sciences (S.S.), University of California, Berkeley, CA, USA. · Department of Electrical and Systems Engineering, University of Pennsylvania, Philadelphia, PA, USA.

LLM Agent OrchestrationLarge Language Model-Based Robot PlanningMulti-Robot CoordinationLong-Horizon LLM AgentsMulti-Robot Exploration
AskPDF
May 11, 2026

SOMA: Efficient Multi-turn LLM Serving via Small Language Model

Large Language Models (LLMs) are increasingly deployed in multi-turn dialogue settings where preserving conversational context across turns is essential. A standard serving practice concatenates the full dialogue history at every turn, which reliably maintains coherence but incurs substantial cost in latency, memory, and API expenditure, especially when queries are routed to large proprietary models. Existing approaches often struggle to balance the trade-off between response quality and efficiency. We propose a framework that exploits the early turns of a session to estimate a local response manifold and then adapt a smaller surrogate model to this local region for the remainder of the conversation. Concretely, we learn soft prompts that maximize semantic divergence between the large and surrogate small language models' responses to surface least-aligned local directions, stabilize training with anti-degeneration control, and distill the mined cases into localized LoRA fine-tuning so the surrogate runs without prompts at inference. A simple gate enables a one-time switch with rollback on drift. We further provide a theoretical analysis for key components in SOMA. Extensive experiments show the effectiveness of SOMA. The source code is provided at: https://github.com/LabRAI/SOMA.

Xueqi Cheng, Qiong Wu, Zhengyi Zhou +3

Florida State University · AT&T Chief Data Office · Louisiana State University +1

LLM ServingSmall Language ModelsLanguage Model DistillationEfficient Language Model InferenceMulti-Turn Dialogue Management
AskPDF