Agentic Insight Generation in VSM Simulations

Micha Selak; Dirk Krechel; Adrian Ulges; Sven Spieckermann; Niklas Stoehr; Andreas Loehr

cs.CLApr 14, 2026

Agentic Insight Generation in VSM Simulations

Authors: Micha Selak, Dirk Krechel, Adrian Ulges, Sven Spieckermann, Niklas Stoehr, Andreas Loehr

Organizations: RheinMain University of Applied Sciences, Wiesbaden, GERMANY · SimPlan AG, Hanau, GERMANY

Abstract

Extracting actionable insights from complex value stream map simulations can be challenging, time-consuming, and error-prone. Recent advances in large language models offer new avenues to support users with this task. While existing approaches excel at processing raw data to gain information, they are structurally unfit to pick up on subtle situational differences needed to distinguish similar data sources in this domain. To address this issue, we propose a decoupled, two-step agentic architecture. By separating orchestration from data analysis, the system leverages progressive data discovery infused with domain expert knowledge. This architecture allows the orchestration to intelligently select data sources and perform multi-hop reasoning across data structures while maintaining a slim internal context. Results from multiple state-of-the-art large language models demonstrate the framework's viability: with top-tier models achieving accuracies of up to 86% and demonstrating high robustness across evaluation runs.

Figures & tables

Figure 2: Overview of the decoupled agentic architecture, showcasing four steps of the under-supplied supermarkets example. Red marks tool calls. Green marks the returned information.

Evaluator	Mean Rating	MAE $\downarrow$	Pearson r $\uparrow$
Expert Consensus	76.67	10.67	80.70
Qwen3-30B-A3B-Thinking-2507	72.67	15.67	69.92
GPT-4.1	73.33	7.33	89.99
GPT-5-mini	77.33	11	78.51
GPT-5.4	67.33	12.67	79.77

Table 1: LLM as a judge validation performed on a sub collection of the evaluation dataset. Ratings were multiplied by 100 to set their interval between 0 and 100. Ratings of the for evaluation runs of LLM judges were averaged. Mean rating should be as close as possible to the human base line. MAE should be as low as possible. Pearson r should be as high as possible. Bold marks best LLM metric.

Model	Ministral-3-14B-Reasoning-2512	Qwen3-30B-A3B-Thinking-2507	GLM-5	Claude-Opus 4.6
AVG ± SD Rating	57.34 ± 6.64	72.66 ± 2.1	83.88 ± 3.92	85.96 ± 0.86
Sample Token AVG	39,819	43,482	27,580	38,594

Table 2: Agent evaluation across 4 evaluation runs on the full evaluation dataset. The score in rating row represent the average rating across all samples followed by the standard divination (SD) across evaluation runs. Sample Token AVG denotes the average amount of tokens used to process a single triple.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

VSM name	question	answer
pharma	Which process is the bottleneck in the overall system?	The process bottling is with a utilization rate of 59.75% the bottleneck in the overall system.
automotive-01	Which process has the highest overall throughput?	The Outer_Stamping station has the highest total throughput with 27094 pieces.
spray-02	How long and how often does a storage tank/buffer run empty during the simulation time? (referring to the minimum storage level in parts)	The following warehouses/supermarkets ran out of stock during the simulation (10 days): Warehouse, Warehouse_02, and Warehouse_03 ran empty from the start and remained so. Warehouse_01 was filled on March 4, 2021, and ran empty again from March 8, 2021. Warehouse_04 Component Base_Lid_Skids ran out of stock on March 6, 2021, and then gradually refilled. Component KSWg ran empty from March 4, 2021, for the remainder of the simulation.

Appendix

Table 3: From German to English translated data examples used in this work.

LLM Agent Orchestration

Large language model agent orchestration coordinates models, tools, and workflows to complete complex tasks reliably and efficiently.

See all 458 papers

OrchSLM: Probing the Dynamics of Small Language Model Orchestration
Sep 11, 2026Chengxi Zhang, Yu YaoLLM Agent Orchestration Small Language Models
MIT
Simthesizer: An Agent-Driven Simulation Framework for LLM Serving Systems
Aug 25, 2026Wonung Kim, Hyunmin Choi, Minsu Kim +3LLM Serving
KAIST Daejeon, Republic of Korea
Decoupled Intelligence: A Multi-Agent LLM Framework for Controllable Traffic Scenario Generation in SUMO
May 26, 2026Shuyang Li, Ruimin KeMulti-Agent LLM Systems Intelligent Transportation Systems
Civil and Environmental Engineering, School of Engineering, Rensselaer Polytechnic Institute, Troy, United States.

Sep 11, 2026J/K move · Enter open · S save

OrchSLM: Probing the Dynamics of Small Language Model Orchestration

Chengxi Zhang, Yu Yao

MIT

LLM Agent Orchestration Small Language Models Language Model Ensembles LLM Routing

Open paper AskPDF arXiv

Although large language models (LLMs) have demonstrated remarkable capabilities, their reliance on cloud-scale infrastructure poses fundamental challenges for deployment in agentic pipelines, including latency, privacy, connectivity, and substantial computational cost. Small language models (SLMs) offer a compelling alternative: recent studies suggest that many repetitive and narrowly scoped subtasks in agentic workloads may be better served by specialized SLMs than by monolithic LLMs. However, the limited capacity and context windows of SLMs can constrain long-horizon reasoning and interaction-heavy orchestration strategies such as iterative verification and debate. This motivates a complementary, non-interactive paradigm in which heterogeneous SLMs independently generate candidate solutions and a router orchestrates their cached samples without further model interaction. To further understand the mechanisms of such orchestration, we introduce OrchSLM, a routing framework that unifies existing non-interactive orchestration methods and exposes their underlying design choices as controllable parameters. Using OrchSLM as a systematic probe, we reveal how orchestration behavior emerges from diverse knobs, including the task structure, model-pool composition, and multi-agent consensus.

Abstract

LLM Agent Orchestration

Explore similar work

OrchSLM: Probing the Dynamics of Small Language Model Orchestration

Simthesizer: An Agent-Driven Simulation Framework for LLM Serving Systems

Decoupled Intelligence: A Multi-Agent LLM Framework for Controllable Traffic Scenario Generation in SUMO