LoRA Fine-Tuned Models for Control Systems Course Q\&A: A Multidimensional Evaluation of Model Scale and Rank Effects

Shaowen Lu; Chengxu Liu; Ping Zhou; Tao Yang

cs.AISep 12, 2026

LoRA Fine-Tuned Models for Control Systems Course Q&A: A Multidimensional Evaluation of Model Scale and Rank Effects

Authors: Shaowen Lu, Chengxu Liu, Ping Zhou, Tao Yang

Organizations: State Key Laboratory of Synthetical Automation for Process Industries Northeastern University Shenyang, China

Abstract

Large language models (LLMs) are increasingly used in specialized university courses, but control-systems questions require coordinated terminology, notation, derivations, and stepwise explanations. Direct general-purpose responses may be inconsistently structured and hard to verify. Using exercises and reference solutions from a Linear Control Systems course, we built a supervised fine-tuning dataset of 360 system-user-assistant conversations. We applied LoRA to Qwen2.5-3B-Instruct and Qwen2.5-7B-Instruct. With identical data splits, inference settings, and evaluation protocols, we compared base and fine-tuned models and tested LoRA ranks r=4, 8, and 16. Evaluation used ROUGE, BERTScore, and structured-output features to measure reference-answer similarity and stability of the Solution-Method-Teaching Points format. LoRA improved both similarity and structured-output stability at both sizes. On the current test set, 7B-r16 achieved the highest ROUGE-L (0.4093) and BERTScore-F1 (0.8643), while r=8 offered a better balance between performance and parameter efficiency. Bootstrap resampling showed ROUGE-L gains of 0.0764 [0.0613, 0.0915] for 3B-r16 and 0.0874 [0.0687, 0.1042] for 7B-r16; both intervals exceeded zero, indicating stable textual-similarity improvements on the current test set. These results suggest LoRA can align open-source instruction-tuned models more closely with the language and pedagogical organization of course reference answers. However, the metrics mainly capture textual similarity and formatting consistency, not domain-specific reasoning or mathematical correctness, which require expert assessment and task-specific rubrics.

Figures & tables

Data subset	Number of samples	Proportion	Primary purpose
Training set	270	75%	Parameter learning during LoRA fine-tuning
Validation set	36	10%	Model selection and hyperparameter tuning
Test set	54	15%	Evaluation of generalization to unseen questions

Table 1: Dataset split for the control systems course question-answering dataset

Item	Description
Data sources	Exercises and reference solutions from Linear Control Systems textbooks
Language	English
Data format	JSONL (chat-style)
Sample structure	System-user-assistant dialogue; four-part user message (Problem Description / Subproblems / Additional Information / Constraints); three-level assistant response (Solution-Method-Teaching Points)
Total samples	360
Training set	270 (75%)

Table 2: Basic information about the control systems course question-answering dataset

Model	Experimental role	Purpose of comparison
Qwen2.5-3B-Instruct	Smaller instruction-tuned model	Evaluate the feasibility of LoRA course adaptation under resource constraints
Qwen2.5-7B-Instruct	Larger instruction-tuned model	Evaluate the effects of greater model size on answer similarity and structured output

Table 3: Base models and experimental purposes

Parameter	Setting
Base models	Qwen2.5-3B-Instruct; Qwen2.5-7B-Instruct
Fine-tuning method	LoRA (SFT)
Training epochs	5
Maximum sequence length	2048
Training batch size	1
Evaluation batch size	1

Table 4: Parameters for LoRA-based supervised fine-tuning

Metric category	Metric	Measured property
Textual similarity	ROUGE-1, ROUGE-2, ROUGE-L	Lexical overlap between generated responses and reference answers
Semantic similarity	BERTScore-F1	Semantic proximity between generated responses and reference answers
Structured output	has_solution, has_method, has_teaching_points	Whether a response follows the pedagogical organization
Structured-output rate	has_solution_rate, has_method_rate, has_teaching_points_rate	Proportion of test samples exhibiting each structural feature

Table 5: Evaluation metrics for the control systems course question-answering models

Model size	LoRA rank	Adapter parameters (M)	Training loss	Best validation loss	Validation-token accuracy (%)	Validation entropy	Best step
3B	4	1.843	0.8552	0.8830	78.09	0.8457	170
3B	8	3.686	0.7826	0.8269	79.03	0.7733	170
3B	16	7.373	0.7239	0.8080	79.36	0.7465	170
7B	4	2.523	0.8156	0.7992	79.57	0.7557	170
7B	8	5.046	0.7403	0.7599	80.31	0.7135	170
7B	16	10.093	0.6800	0.7330	80.71	0.6865	170

Table 6: Training and validation results across model sizes and LoRA ranks

Figure 2: Validation loss across model sizes and LoRA ranks.

Figure 3: Automatic evaluation metrics for the base and LoRA-fine-tuned models. (a) ROUGE-1; (b) ROUGE-2; (c) ROUGE-L; (d) BERTScore-F1

Figure 4: Bootstrap confidence intervals for ROUGE improvements.

Figure 5: Bootstrap confidence intervals for improvements in structured-output coverage.

Model size	LoRA rank	Adapter parameters (M)	$\Delta$ ROUGE-L	$\Delta$ BERTScore-F1	$\Delta$ ROUGE-L per million parameters
3B	4	1.843	+0.0334	+0.0028	0.0181
3B	8	3.686	+0.0579	+0.0151	0.0157
3B	16	7.373	+0.0764	+0.0314	0.0104
7B	4	2.523	+0.0407	+0.0212	0.0161
7B	8	5.046	+0.0726	+0.0325	0.0144
7B	16	10.093	+0.0874	+0.0332	0.0087

Table 7: Comparison of LoRA rank and parameter efficiency

Figure 6: Relationship among LoRA rank, absolute ROUGE-L improvement, and parameter efficiency. Although $r=16$ achieved the largest absolute gain, the ROUGE-L gain per adapter parameter decreased as rank increased.

Figure 7: Coverage of structured instructional output in the base and LoRA-fine-tuned models.

Educational QA

See all 10 papers

Which LoRA? An Empirical Study on the Effectiveness of LoRA Techniques During Multilingual Instruction Tuning
Jun 9, 2026Thamali Wijewardhana, Napoleon H. Reyes, Surangika RanathungaLow-Rank Adaptation Cross-Lingual Transfer
School of Mathematical and Computational Sciences Massey University, Auckland, New Zealand
Beyond LoRA vs. Full Fine-Tuning: Gradient-Guided Optimizer Routing for LLM Adaptation
May 8, 2026Haozhan Tang, Xiuqi Zhu, Xinyin Zhang +3LLM Fine-Tuning Low-Rank Adaptation
Carnegie Mellon University · Tsinghua University · Infinigence AI
Application-Driven Pedagogical Knowledge Optimization of Open-Source LLMs via Reinforcement Learning and Supervised Fine-Tuning
Apr 7, 2026Navan Preet Singh, Xiaokun Wang, Anurag Garikipati +3LLM Fine-Tuning Language Modeling
Forta, Houston, TX · East China Normal University, Shanghai, China · Incept Labs, Houston, TX +1

Jun 9, 2026J/K move · Enter open · S save

Which LoRA? An Empirical Study on the Effectiveness of LoRA Techniques During Multilingual Instruction Tuning

Thamali Wijewardhana, Napoleon H. Reyes, Surangika Ranathunga

School of Mathematical and Computational Sciences Massey University, Auckland, New Zealand

Low-Rank Adaptation Cross-Lingual Transfer

Open paper AskPDF arXiv

We investigate whether commonly available LoRA variants have an advantage over basic LoRA in multilingual instruction tuning. Experiments involving LoRA and four other variants on two datasets across diverse target languages show that there is no significant advantage in using more complex LoRA variants instead of basic LoRA, with respect to balancing cross-lingual transfer and knowledge retention. An analysis of hidden embeddings reveal that layer-wise language representation remains largely similar across LLMs fine-tuned with different LoRA techniques, suggesting that architectural novelty of LoRA techniques may not translate into better cross-lingual adaptation.

Abstract

Educational QA

Explore similar work

Which LoRA? An Empirical Study on the Effectiveness of LoRA Techniques During Multilingual Instruction Tuning

Beyond LoRA vs. Full Fine-Tuning: Gradient-Guided Optimizer Routing for LLM Adaptation

Application-Driven Pedagogical Knowledge Optimization of Open-Source LLMs via Reinforcement Learning and Supervised Fine-Tuning