Stan: An LLM-based thermodynamics course assistant
Authors: Eric M. Furst, Vasudevan Venkateshwaran
Organizations: Department of Chemical and Biomolecular Engineering, Allan P. Colburn Laboratory, University of Delaware, Newark, DE 19716, USA · W. L. Gore & Associates Inc., 100 Airport Road, Elkton, MD 21921, USA
Discussions of AI in education focus predominantly on student-facing tools -- chatbots, tutors, and problem generators -- while the potential for the same infrastructure to support instructors remains largely unexplored. We describe Stan, a suite of tools for an undergraduate chemical engineering thermodynamics course built on a data pipeline that we develop and deploy in dual roles: serving students and supporting instructors from a shared foundation of lecture transcripts and a structured textbook index. On the student side, a retrieval-augmented generation (RAG) pipeline answers natural-language queries by extracting technical terms, matching them against the textbook index, and synthesizing grounded responses with specific chapter and page references. On the instructor side, the same transcript corpus is processed through structured analysis pipelines that produce per-lecture summaries, identify student questions and moments of confusion, and catalog the anecdotes and analogies used to motivate difficult material -- providing a searchable, semester-scale record of teaching that supports course reflection, reminders, and improvement. All components, including speech-to-text transcription, structured content extraction, and interactive query answering, run entirely on locally controlled hardware using open-weight models (Whisper large-v3, Llama~3.1 8B) with no dependence on cloud APIs, ensuring predictable costs, full data privacy, and reproducibility independent of third-party services. We describe the design, implementation, and practical failure modes encountered when deploying 7--8 billion parameter models for structured extraction over long lecture transcripts, including context truncation, bimodal output distributions, and schema drift, along with the mitigations that resolved them.
Figures & tables
Level
Title
Primary Role
Capabilities
1
Resource Pointer
Indexer
• Points to textbook sections or course materials • Acts as a smart search tool without explanations
Table 1: Classification of Learning Assistant Intelligence Levels
Figure 1: Stan system architecture. The batch pipeline (left) runs on a GPU workstation, producing transcripts and structured lecture analyses. The interactive query pipeline (right) runs on a consumer laptop, using a dual-path extraction strategy (regex and LLM) to match student queries against the textbook index, then synthesizing grounded responses with chapter and lecture context.
Metric
Baseline
Full corpus
Lectures
1
39
Audio duration
55.0 min
35.7 hr
Transcription time
177 s (18.6 × )
43.7 min (49 × )
Raw segments
826
20,361
Hallucination loops
4
1
Segments removed
53 (6.4%)
4 (0.02%)
Table 2: Transcript quality: baseline (one lecture, no suppression) versus full corpus (39 lectures, all mitigations active).
Metric
Kaltura
Whisper
Mean word count
7,449
7,409
Word count ratio
0.99–1.01
“entropy” (total)
329
331
“enthalpy” (total)
50
49
“reversible” (total)
87
81
“thermodynamic(s)” (total)
117
115
Table 3: Comparison of institutional (Kaltura) and domain-prompted Whisper transcripts for six matched lectures. Domain term counts are summed across all six lectures.
Intended
Kaltura
Whisper
the next module
the next logro
the next module
liquid versus gases
liquid versus gaffy
liquids versus gases
a balance of the enthalpies
a balance of the envelopes of the trees
enthalpies
Gibbs and Helmholtz free energies
ideas and compulse free energies
gives and compulse for energies
keeping your protons
keeping your protein
keeping your protons
keeping your quarks
keeping your torch
keeping your quarks
Table 4: Representative misrecognitions comparing Kaltura and domain-prompted Whisper transcripts (lecture 9). The intended phrase is determined from surrounding context.
We present ThermoQA, a benchmark of 293 open-ended engineering thermodynamics problems in three tiers: property lookups (110 Q), component analysis (101 Q), and full cycle analysis (82 Q). Ground truth is computed programmatically from CoolProp 7.2.0, covering water, R-134a, and variable-cp air. Six frontier LLMs are evaluated across three independent runs each. The composite leaderboard is led by Claude Opus 4.6 (94.1%), GPT-5.4 (93.1%), and Gemini 3.1 Pro (92.5%). Cross-tier degradation ranges from 2.8 pp (Opus) to 32.5 pp (MiniMax), confirming that property memorization does not imply thermodynamic reasoning. Supercritical water, R-134a refrigerant, and combined-cycle gas turbine analysis serve as natural discriminators with 40-60 pp performance spreads. Multi-run sigma ranges from +/-0.1% to +/-2.5%, quantifying reasoning consistency as a distinct evaluation axis. Dataset and code are open-source at https://huggingface.co/datasets/olivenet/thermoqa
Large-language-model (LLM) graders promise to relieve the grading burden of upper-division STEM courses, but most deployments to date send student work to third-party APIs, violating FERPA and exposing institutions to data risk while requiring substantial assignment modification. We present LaTA(LaTeX Teaching Assistant), a drop-in, open-source autograder that runs entirely on commodity on-premises hardware and assumes a LaTeX-native workflow already adopted by many engineering and physics courses. LaTA implements a four-stage pipeline (ingest, segment, grade, report) using a locally hosted open-weight chain-of-thought LLM grader (gpt-oss:120b) that compares student work to an instructor-authored reference solution and applies a YAML rubric with binary per-item scoring. We deployed LaTA in Winter~2026 in ME 373 (Mechanical Engineering Methods) at Oregon State University, grading every weekly assignment for approximately 200 students on a single Mac Studio at $0 marginal cost per assignment and 1--3 minutes of wall-clock time per submission, enabling regrading of corrected assignments and greatly expanded TA office hour offerings. The instructor-confirmed grading-error rate held at roughly 0.02--0.04% per rubric line item across the term. Relative to the same instructor's previous traditionally-graded cohort, the LaTA-graded cohort outperformed by approximately 11% on the midterm exam and 8% on the final exam, and reported large gains in self-assessed confidence on every stated learning objective (N=159 survey responses, Δ≥+1.49 Likert points, p<10−27 on every comparison). We release the code under AGPLv3.
Jesse A. Rodríguez
School of Mechanical, Industrial, and Manufacturing Engineering Oregon State University Corvallis, OR 97331, USA
Campus AI tutors based on retrieval-augmented generation (RAG) must ground answers in assigned course materials while keeping textbooks and student dialogue on institutional infrastructure. We present CourseChat, an on-premises, multi-course RAG tutor for undergraduate business education, deployed behind a campus web gateway and intended for use embedded in Moodle. Six isolated course offerings, each keyed by its own course reference number (CRN), share twin-edge AI hosts running a FastAPI service, a local vector database, and a local large language model (LLM) served by Ollama. We report two generation-model bake-off rounds, a separate fixed-evidence source-fidelity comparison, and conversation and quiz audits. Several larger models failed the classroom speed gate, but a 12B model and a 7B alternative passed. A separate mixture-of-experts candidate improved some corrections while introducing new factual and continuity errors. We therefore retain the 8B production model pending a demonstrated overall improvement, rather than claiming that 8B is universally optimal. Software changes improved follow-up topic resolution while preserving course scope; 435 prebuilt questions across 65 modules decouple practice from live generation. The results support treating model choice, evidence selection, serving compatibility, and product design as a joint engineering decision. They do not establish learning gains: faculty ratings, peak-load capacity, and complete public-gateway acceptance remain separate evaluation needs.
Sidney Shapiro, Joshua Lindemann
Dhillon School of Business, University of Lethbridge