Policy-Governed LLM Routing with Intent Matching for Instrument Laboratories

Emmanuel A. Olowe; Danial Chitnis

cs.CYApr 3, 2026

Policy-Governed LLM Routing with Intent Matching for Instrument Laboratories

Authors: Emmanuel A. Olowe, Danial Chitnis

Organizations: School of Engineering, The University of Edinburgh, Edinburgh, UK

Abstract

AI tutoring systems in engineering labs face a tension between providing sufficient assistance and preserving learning opportunities. Existing systems typically offer instructors limited control over assistance timing, content, or cost. This paper describes a routing and governance system for LLM-based lab assistance comprising two components: Routiium, an OpenAI-compatible gateway that manages multiple LLM backends with configurable prompt modifications and usage logging, and EduRouter, a policy-aware routing service that enforces per-lab budgets, approval workflows, and embedding-based question matching. We evaluated the system using trace-driven simulation calibrated from two engineering labs (LED characterization, RC circuit analysis) and a 100-query replay through live models. In simulations, governed policies (P1/P2) increased challenge-alignment index from 0.90 to 0.98 and overlay-adherence score from 0.69 to 0.87 compared to ungoverned operation (P0). The productive-struggle window metric increased from 1.4 to 3.6 simulated turns before high-scaffold hints appeared. In the 100-query replay, EduRouter routed 75% of queries to a local model, reducing token costs by 66% ( $0.087 vs.$ 0.26 for all-premium routing) while maintaining canonical hit rate of 1.0 for the curated 89-intent question bank. We release Routiium, EduRouter, canonical-task tooling, and simulator configurations to support replication and future classroom studies.

Figures & tables

System	Multi-model routing	Budget enforcement	Instructor approvals	Prompt/audit transparency
FrugalGPT [ 8 ]	✓	✓	—	—
RouteLLM [ 9 ]	✓	(varies)	—	—
ThriftLLM [ 10 ]	✓	✓	—	—
This work (Routiium+EduRouter)	✓	✓	✓	✓

TABLE I: Positioning relative to related systems (illustrative dimensions).

Label	Definition	Governance behavior
L0	Validation or minimal prompt	No approval required; conceptual framing
L1	Guided troubleshooting hint	Overlay adds justification prompts
L2	Worked example fragment	Requires budget debit; throttled after configurable retry count
L3	Complete solution	TA approval required plus budget and frequency limits
P0	Ungoverned	No spend limits, overlays disabled, no approvals
P1	Governed	Per-turn spend limits, overlays enabled, L3 frequency caps

TABLE II: Hint levels and policy modes referenced throughout the paper.

Panel of Figure 3 — Fig. 1: LABIIUM educational platform architecture with centralized teacher control, an explicit routing stack (EduRouter + Routiium), and distributed LAMB devices. The teacher console configures policies, approvals, and overlays; EduRouter computes cost- and policy-aware plans ( /route/plan ) while Routiium exposes an OpenAI-compatible proxy with overlays, logging, and streaming telemetry. Students reach their assigned LAMB via secure web tunnels, which in turn talk to instruments and DUTs while upstream assistance is routed through this governable routing and assistance plane.

Panel of Figure 4 — Fig. 2: Protocol-level interaction between student, teacher console, Routiium, EduRouter, and LLM backends. Routiium receives chat requests, consults EduRouter’s /route/plan , dispatches to local vLLM or external OpenAI GPT-5 Mini, and streams the response back with routing headers.

Field	Content
Routing rationale	Reason for model selection (e.g., “canonical match: RC circuit troubleshooting” or “budget limit enforced”)
Question match	Matched library entry IDs and similarity scores
Active overlay	Which prompt modifications were applied
Content fingerprint	Hash for verifying overlay application
Request trace ID	Link to equipment telemetry

TABLE III: Metadata logged for each routing decision.

Panel of Figure 6 — Fig. 3: Trace-driven simulator pipeline used for synthetic evaluation.

Policy	$\mathrm{CAI}$	$\mathrm{OAS}$	$\mathrm{PSW}$	$\mathrm{IIL}$	$\mathrm{EI}$
P0 (Ungoverned)	0.90	0.69	1.45	5.20	0.59
P1 (Governed)	0.98	0.84	2.75	3.20	0.56
P2 (Integrity)	0.98	0.87	3.56	2.39	0.52

TABLE IV: Steerability metrics by policy (simulator, mean across seeds;

\mathrm{IIL}

in turns).

Embedding	$\mathrm{CHR}$	$\mathrm{CRG}$	$\mathrm{FCR}$
gte-large-en-v1.5	0.95	0.19	0.007
bge-large-en-v1.5	0.95	0.17	0.012
nomic-embed-text-v1.5	0.94	0.17	0.011
bge-small-en-v1.5	0.93	0.15	0.015
FastEmbed (edge)	0.72	0.05	0.010
Embedding off	0.00	0.00	0.000

TABLE V: Canonical metrics (simulator, averaged across jobs).

Policy	Freeze TTL	$\mathrm{CSS}$
P0 (Ungoverned)	None	0.00
P1 (Governed)	5 min	0.49
P2 (Integrity)	30 min	0.60

TABLE VI: Canonical Stickiness Stability (

\mathrm{CSS}

) by policy and freeze horizon.

Fig. 4: Embedding ablation in the simulator. Bars show canonical hit rate (CHR, left) and canonical routing gain ( $\mathrm{CRG}$ , right) averaged over cache TTLs and warm/cold starts for five embedding configurations—BAAI/bge-small-en-v1.5, BAAI/bge-large-en-v1.5, Alibaba-NLP/gte-large-en-v1.5, nomic-ai/nomic-embed-text-v1.5, and a FastEmbed deployment (ONNX wrapper around BAAI/bge-small-en-v1.5)—plus a no-embeddings baseline. Direct cloud-style integrations of the larger models achieve the strongest CHR and $\mathrm{CRG}$ , while the FastEmbed edge deployment trades some canonical fidelity for lower infrastructure cost and dependence on external providers.

Panel of Figure 11 — Fig. 5: 100-query replay results. (a) Token cost by routing configuration. (b) End-to-end latency. (c) Time-to-first-token decomposed into routing and execution components. (d) Routing correctness by query difficulty.

Field	Meaning
session_id	Anonymous session identifier
lab_id	Micro-lab ( led_iv , rc_step )
policy	P0 , P1 , or P2
step_id	Current lab step key
hint_req / hint_granted	Request/grant events + level (L0–L3)
justification_len	Characters in student justification

TABLE VII: Telemetry feature dictionary (excerpt).

Knob	Target / Rationale
Hint budgets	$5 total / lab; L3_max =2 to enforce$ \mathrm{PSW}$ goals
Approvals	Two TAs (M/M/2) with p95 $<$ 60 s decision latency
Overlay personas	Default “Socratic troubleshooting”; teacher can swap to “diagnostic” overlay in P2
Boost/freeze	One-click teacher boost to premium tier; freeze TTL default 5 min (extendable to 30 min)
Integrity throttle	Block assistance when integrity_flag triggers for 3 consecutive turns
Canonical embedding	Enabled in P1/P2 via fastembed ; hashed fallback when privacy_mode= hashed

TABLE VIII: Policy targets used in simulation scenarios.

Metric	Interpretation
$\mathrm{CAI}$	Challenge alignment (hint mix vs. targets)
$\mathrm{OAS}$	Overlay adherence (no-leakage rate)
$\mathrm{PSW}$	Productive-struggle window (steps before L2/L3)
$\mathrm{IIL}$	Instructor-influence latency (turns to take effect)
$\mathrm{EI}$	Equity index ( $1-\text{Gini}$ on L3 allocation)
$\mathrm{CHR}$	Canonical hit rate (fraction with canonical match)

TABLE IX: Governance and canonical metric glossary.

LLM Routing

Large language model routing selects models, tools, experts, or inference strategies to balance answer quality, cost, latency, and deployment constraints.

See all 349 papers

Learning to Prompt: Improving Student Engagement with Adaptive LLM-based High-School Tutoring
Jun 18, 2026Po-Chin Chang, Nicholas Hogan, Aske Plaat +1Prompt Learning Intelligent Tutoring Systems
Leiden University · FutureWhiz
LLMRouter: Unified Infrastructure for Developing, Evaluating, and Deploying LLM Routers
Aug 7, 2026Tao Feng, Fangxu Yu, Haozhen Zhang +9LLM Evaluation LLM Routing
University of Illinois Urbana-Champaign · University of Maryland, College Park · 3Nanyang Technological University +2
RouteJudge: An Open Platform for Reproducible and Preference-Aware LLM Routing
Jun 17, 2026Guannan Lai, Haoran Hu, Han-Jia YeLLM Evaluation Pairwise Preference Evaluation
School of Artificial Intelligence, Nanjing University, China · 2National Key Laboratory for Novel Software Technology, Nanjing University, 210023, China · 3SinapisAI

Jun 18, 2026J/K move · Enter open · S save

Learning to Prompt: Improving Student Engagement with Adaptive LLM-based High-School Tutoring

Po-Chin Chang, Nicholas Hogan, Aske Plaat, Michiel T. van der Meer

Leiden University · FutureWhiz

Prompt Learning Intelligent Tutoring Systems Generative AI in Education Educational Dialogue Systems Education Personalized Learning

Open paper AskPDF arXiv

LLMs can personalize education, although current static-prompt tutoring systems struggle to adapt to diverse academic disciplines. We develop and test a system with subject-aware prompting, based on 14 pedagogical features (e.g., tutor scaffolding, student understanding) extracted from raw transcripts. We first train a prompt routing model in a simulation environment, and then deploy it for online adaptation with actual high-school students. The simulation benchmark shows the router outperforming two static baselines (

0.694

vs.

0.647

and

0.64

p<0.001

). A/B testing (

N=656

conversations from 359 students) shows sim-to-real transfer where the model switches from analytical to scaffolding learning strategies. Our adaptive prompt selection mechanism improves instructional efficiency, maintains pedagogical quality and reduces interactions by around 3 turns (

p=0.007

). While a greedy router achieves a comparable exercise conversion rate with the baseline (

19.1\%

vs.

19.6\%

), a stochastic router that samples strategies leads to a higher conversion rate (

28.1\%

Abstract

LLM Routing

Explore similar work

Learning to Prompt: Improving Student Engagement with Adaptive LLM-based High-School Tutoring

LLMRouter: Unified Infrastructure for Developing, Evaluating, and Deploying LLM Routers

RouteJudge: An Open Platform for Reproducible and Preference-Aware LLM Routing