On-Premises Multi-Course RAG Tutoring for Business Education: Hardware-Software Trade-offs in a Campus AI Tutor

Sidney Shapiro; Joshua Lindemann

cs.AIOct 1, 2026

On-Premises Multi-Course RAG Tutoring for Business Education: Hardware-Software Trade-offs in a Campus AI Tutor

Authors: Sidney Shapiro, Joshua Lindemann

Organizations: Dhillon School of Business, University of Lethbridge

Abstract

Campus AI tutors based on retrieval-augmented generation (RAG) must ground answers in assigned course materials while keeping textbooks and student dialogue on institutional infrastructure. We present CourseChat, an on-premises, multi-course RAG tutor for undergraduate business education, deployed behind a campus web gateway and intended for use embedded in Moodle. Six isolated course offerings, each keyed by its own course reference number (CRN), share twin-edge AI hosts running a FastAPI service, a local vector database, and a local large language model (LLM) served by Ollama. We report two generation-model bake-off rounds, a separate fixed-evidence source-fidelity comparison, and conversation and quiz audits. Several larger models failed the classroom speed gate, but a 12B model and a 7B alternative passed. A separate mixture-of-experts candidate improved some corrections while introducing new factual and continuity errors. We therefore retain the 8B production model pending a demonstrated overall improvement, rather than claiming that 8B is universally optimal. Software changes improved follow-up topic resolution while preserving course scope; 435 prebuilt questions across 65 modules decouple practice from live generation. The results support treating model choice, evidence selection, serving compatibility, and product design as a joint engineering decision. They do not establish learning gains: faculty ratings, peak-load capacity, and complete public-gateway acceptance remain separate evaluation needs.

Figures & tables

Panel of Figure 1 — Figure 1: Application request paths. Quiz practice uses stored items and deterministic grading. Course, mixed, and dependent follow-up routes retrieve scoped evidence before generation; social and clarification routes do not retrieve course content. Responses return through the same API and UI. The dashed arrow supplies course configuration.

Panel of Figure 2 — Figure 2: Campus access architecture and deployed twin local stacks. Solid arrows indicate request/response traffic; the dashed path transfers index snapshots. Quiz and application artifacts are deployed separately. The node audits reported here bypass the gateway; complete public course-slug acceptance remained open in the deployment snapshot.

Approach	Observed result	Trade-off	Decision
Larger dense generators (first round)	All passed 12/12 live probes; median completion times were 1.75–3.36 times the 8B baseline.	Some answers were more complete, but every candidate exceeded the $+30\%$ classroom latency limit. Probe passes alone do not establish pedagogical quality.	Retain 8B for classroom service.
Qwen 7B and Mistral Nemo 12B	Both passed 12/12 probes; latency ratios were 1.01 and 1.24. Qwen completeness was 0.93 versus 0.91 for 8B.	Both met the speed gate. Speed passes and small completeness differences did not establish a faculty-reviewed tutoring advantage.	Retain as candidates for paired review.
Structured Qwen mixture-of-experts response	Warm median was 3.00 s versus 3.47 s for 8B, but review found three regressions across twelve fixed-evidence cases.	Faster warm responses and better corrections accompanied a new factual error and an unnecessary refusal. Model and response contract changed together.	Hold activation.
Deterministic follow-up resolution	Topic checks rose from 23.3–40.0% in baseline resolver preflight to 90–100% live across five courses; all 400 scope checks passed.	Better continuity required no generator change. Correct topic selection and scope do not by themselves verify answer fidelity.	Deploy resolver with 8B.
Prepared quiz banks	435 questions cover 65 modules. Warm practice-request medians were 4.8 and 5.8 ms on the two nodes.	Inspectable evidence and deterministic grading require finite banks, maintenance after source changes, and instructor review.	Serve prepared banks; faculty approval remains pending.

Table 1: Decision summary across model trials and system-design evaluations. Results come from distinct protocols and are not a common accuracy ranking. Model latency ratios use the 8B baseline from the corresponding bake-off round; all reported timings are server-local.

Model	Live eval	p50 (ms)	vs. 8B	Speed gate
llama3.1:8b (prod.)	12/12	8392	1.00 $\times$	pass
qwen2.5:14b-instruct	12/12	14726	1.75 $\times$	fail
gemma3:12b	12/12	18855	2.25 $\times$	fail
qwen3:14b	12/12	28194	3.36 $\times$	fail
qwen3.6:35b-A3B (Q4)	0/12	15109	1.80 $\times$	fail †

Table 2: Bake-off round 1 (includes thinking-capable tags).

Model	Live eval	p50 (ms)	vs. 8B	Speed gate
llama3.1:8b	12/12	8556	1.00 $\times$	pass
qwen2.5:7b-instruct	12/12	8668	1.01 $\times$	pass
mistral-nemo:12b	12/12	10589	1.24 $\times$	pass
ibm/granite3.3:8b	0/12	21443	2.51 $\times$	fail ‡

Table 3: Bake-off round 2 (non-thinking / think-disabled candidates).

Panel of Figure 3 — Figure 3: Knowledge preparation and deployment. Automatic quiz building requires passing corpus quality gates. Independently reviewed additions provide a separate path, bound to current primary evidence, without clearing failed chat evaluations. Both paths require artifact validation. Dashed arrows denote deployment transfers; instructor pedagogical review remains separate and pending for the published banks.

Offering	Baseline	Updated live	Updated rate	Scope
Course 1	31/60	60/60	100.0%	80/80
Course 2	24/60	54/60	90.0%	80/80
Course 3	22/60	55/60	91.7%	80/80
Course 4	17/60	58/60	96.7%	80/80
Course 5	14/60	59/60	98.3%	80/80
Course 6	19/60	60/60	100.0%	80/80

Table 4: Intended-topic follow-up checks. Course 1 has a live baseline; Courses 2–6 have resolver-only baselines. Updated results are live for all courses with the unchanged 8B generator. Scope checks also include initial turns.

AI in Education

AI in Education studies how artificial intelligence can support teaching, learning, assessment, and educational decision-making.

See all 261 papers

From 50% to Mastery in 3 Days: A Low-Resource SOP for Localizing Graduate-Level AI Tutors via Shadow-RAG
Mar 21, 2026Zonglin Yang, J. -H. Xie, Lining Zhang +2Generative AI in Education Intelligent Tutoring Systems
School of Mechanics and Engineering Science, Peking University, Beijing, 100871, China · Readraft Intelligent Technology Co., Ltd., China · Graduate School of Education, Peking University, Beijing, 100871, China +1
From Surface Learning to Deep Understanding: A Grounded AI Tutoring System for Moodle
May 7, 2026Anna Ostrowska, Michał Kukla, Gabriela Majstrak +4Intelligent Tutoring Systems AI in Education
Faculty of Mathematics and Information Sciences, Warsaw University of Technology, Warsaw, Poland
Learning Engagement Assistant (LEA): Cross-Course Scalability and Classroom Evaluation of an Agentic AI Tutoring System
Jul 15, 2026Teri Rumble, Javad Zarrin, P. George Lovell +1Retrieval-Augmented Generation Intelligent Tutoring Systems
Faculty of Design, Informatics and Business, Abertay University, Dundee, United Kingdom.

Mar 21, 2026J/K move · Enter open · S save

From 50% to Mastery in 3 Days: A Low-Resource SOP for Localizing Graduate-Level AI Tutors via Shadow-RAG

Zonglin Yang, J. -H. Xie, Lining Zhang, Jiyou Jia+1

School of Mechanics and Engineering Science, Peking University, Beijing, 100871, China · Readraft Intelligent Technology Co., Ltd., China · Graduate School of Education, Peking University, Beijing, 100871, China+1

Generative AI in Education Intelligent Tutoring Systems Retrieval-Augmented Generation LLM Reasoning

Open paper AskPDF arXiv

Deploying high-fidelity AI tutors in schools is often blocked by the Resource Curse -- the need for expensive cloud GPUs and massive data engineering. In this practitioner report, we present a replicable Standard Operating Procedure that breaks this barrier. Using a Vision-Language Model data cleaning strategy and a novel Shadow-RAG architecture, we localized a graduate-level Applied Mathematics tutor using only 3 person-days of non-expert labor and open-weights 32B models deployable on a single consumer-grade GPU. Our pilot study on a full graduate-level final exam reveals a striking emergence phenomenon: while both zero-shot baselines and standard retrieval stagnate around 50-60% accuracy across model generations, the Shadow Agent, which provides structured reasoning guidance, triggers a massive capability surge in newer 32B models, boosting performance from 74% (Naive RAG) to mastery level (90%). In contrast, older models see only modest gains (~10%). This suggests that such guidance is the key to unlocking the latent power of modern small language models. This work offers a cost-effective, scientifically grounded blueprint for ubiquitous AI education.

Abstract

AI in Education

Explore similar work

From 50% to Mastery in 3 Days: A Low-Resource SOP for Localizing Graduate-Level AI Tutors via Shadow-RAG

From Surface Learning to Deep Understanding: A Grounded AI Tutoring System for Moodle

Learning Engagement Assistant (LEA): Cross-Course Scalability and Classroom Evaluation of an Agentic AI Tutoring System