Constraint-Driven Warm-Freeze for Efficient Transfer Learning in Photovoltaic Systems

Yasmeen Saeed; Ahmed Sharshar; Mohsen Guizani

cs.NEApr 7, 2026

Constraint-Driven Warm-Freeze for Efficient Transfer Learning in Photovoltaic Systems

Authors: Yasmeen Saeed, Ahmed Sharshar, Mohsen Guizani

Organizations: Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, United Arab Emirates

Abstract

Detecting cyberattacks in photovoltaic (PV) monitoring and MPPT control signals requires models that are robust to bias, drift, and transient spikes, yet lightweight enough for resource-constrained edge controllers. While deep learning outperforms traditional physics-based diagnostics and handcrafted features, standard fine-tuning is computationally prohibitive for edge devices. Furthermore, existing Parameter-Efficient Fine-Tuning (PEFT) methods typically apply uniform adaptation or rely on expensive architectural searches, lacking the flexibility to adhere to strict hardware budgets. To bridge this gap, we propose Constraint-Driven Warm-Freeze (CDWF), a budget-aware adaptation framework. CDWF leverages a brief warm-start phase to quantify gradient-based block importance, then solves a constrained optimization problem to dynamically allocate full trainability to high-impact blocks while efficiently adapting the remaining blocks via Low-Rank Adaptation (LoRA). We evaluate CDWF on standard vision benchmarks (CIFAR-10/100) and a novel PV cyberattack dataset, transferring from bias pretraining to drift and spike detection. The experiments demonstrate that CDWF retains 90 to 99% of full fine-tuning performance while reducing trainable parameters by up to 120x. These results establish CDWF as an effective, importance-guided solution for reliable transfer learning under tight edge constraints.

Figures & tables

Panel of Figure 1 — Fig. 1 : Comparison of adaptation strategies. (a) Full fine-tuning updates all blocks. (b) CDWF selectively adapts high-importance blocks under a parameter budget. (c) Uniform LoRA applies identical adapters to all blocks.

Dataset	Method	Test Acc(%)	Test AUC(%)	Retention(%)	Param Count	Param Reduction
CIFAR-10	Full-FT	88.28	–	100.0	23,528,522	1.0 $\times$
	LoRA (rank=8)	81.29	–	92.08	457,738	52.36 $\times$
	CDWF (rank=8)	85.29	–	96.61	401,994	58.5 $\times$
CIFAR-100	Full-FT	64.99	–	100.0	23,528,522	1.0 $\times$
	LoRA (rank=16)	49.32	–	75.89	809,060	29.1 $\times$
	CDWF (rank=16)	59.57	–	91.66	818,596	28.7 $\times$

TABLE I : Performance comparison of CDWF under a forced rank constraint:

Panel of Figure 3 — Fig. 2 : 10-second voltage traces showing (a) drift attack with easy and harder gradual voltage deviation and (b) spike attack with easy and harder transient disturbances.

Dataset	$f_{\max}$	Test Acc (%)	Test AUC (%)	Retention (%)	Trainable (%)	Chosen $(k,r)$
CIFAR-10	0.02	85.47	–	96.82	1.07	(1,4)
	0.05	86.13	–	97.56	2.65	(2,4)
	0.10	87.14	–	98.71	9.0	(3,4)
CIFAR-100	0.02	59.53	–	91.60	1.84	(1,4)
	0.05	59.53	–	91.60	1.84	(1,4)
	0.10	61.82	–	95.12	9.71	(3,4)

TABLE II : Budgeted performance of CDWF across datasets under explicit trainable-parameter budgets

f_{\max}\in\{0.02,0.05,0.10\}

. Retention is computed relative to Full Fine-Tuning on the same dataset (for PV tasks, retention is computed using AUC).

Method	Rank	Test Acc (%)	Retention (%)	Trainable (%)	Pred. Error (%)	Architecture
Full-FT	–	88.28	100.0	100.0	–	All trainable
LoRA-10	1	76.11	86.21	0.32	–	All LoRA-r1
LoRA-10	2	78.34	88.74	0.55	–	All LoRA-r2
LoRA-10	4	79.89	90.50	1.01	–	All LoRA-r4
LoRA-10	8	81.29	92.08	1.91	–	All LoRA-r8
LoRA-10	16	81.23	92.01	3.67	–	All LoRA-r16

TABLE III : Effect of LoRA rank on CIFAR-10 under a fixed 10-epoch budget (3 warm-start + 7 fine-tuning epochs). CDWF is evaluated with fixed LoRA ranks and with adaptive rank selection.

Fig. 3 : Block importance and CDWF block selection across budgets on CIFAR-100.

Warm-start (epochs)	Test Acc. (%)	Retention (%)	Time (s)
1	51.97	79.97	103.50
2	56.62	87.12	103.77
3	59.60	91.71	105.27
4	61.64	94.84	109.34

TABLE IV : Warm-start sensitivity of CDWF on CIFAR-100 under a fixed parameter budget (

f_{\max}=0.05

) and a fixed total training budget of 10 epochs.

Fig. 4 : Epoch-wise CIFAR-100 validation accuracy (10 epoch budget). kW+(10

-

k)F indicates k warm-start epochs followed by fine-tuning epochs

Method	ResNet-18		ResNet-101		ViT-B/16
Method	Acc (%)	Params Red.	Acc (%)	Params Red.	Acc (%)	Params Red.
FL (Full Fine-tuning)	85.63	1.0×	86.94	1.0×	96.33	1.00×
LoRA	77.43	21.14×	81.74	65.12×	97.92	860.28×
CDWF ( $f_{\max}$ =0.05)	83.19	24.34×	84.79	53.03×	96.81	859.35×

TABLE V : Comparison of Accuracy and Parameter Reduction across different backbones (ResNet-18, ResNet-101, ViT). ’FL’ denotes Full Fine-tuning on CIFAR-10

Method	Test Acc (%)	Retention (%)	Param Cnt	Time (min)
Full Fine-Tuning	89.01	100.0	85,875,556	39.46
LoRA (r=8)	89.93	100.03	1,327,104	44.52
DoRA (r=8)	89.97	101.08	1,410,048	81.02
AdaLoRA ( $12\xrightarrow{}8$ )	90.16	101.29	1,991,520	54.00

TABLE VI : Comparison of parameter-efficient fine-tuning methods on CIFAR-100 using ViT-B/16. Retention is measured relative to full fine-tuning.

Fine-Tuning

Fine-tuning adapts pretrained models to new tasks, domains, and behaviors by updating some or all of their parameters.

See all 250 papers

Efficient PEFT Methods with Adaptive Checkpointing for Vision Models and VLMs on Resource Constrained Consumer-GPUs
Jul 2, 2026Altay Toktassyn, Jurn-Gyu ParkFine-Tuning Memory-Efficient Fine-Tuning
Department of Computer Science, Nazarbayev University, Astana 010000, Kazakhstan
Circuit Fine-Tuning for Compute-Efficient Transformer Adaptation
Aug 8, 2026Uri Z. Kialy, Gil Ben-ArtziFine-Tuning Vision Foundation Model Adaptation
School of Computer Science, Ariel University, Israel
TrainDeeploy: Hardware-Accelerated Parameter-Efficient Fine-Tuning of Small Transformer Models at the Extreme Edge
Mar 10, 2026Run Wang, Victor J. B. Jung, Philip Wiese +3On-Device Training Edge ML
Integrated Systems Laboratory (IIS), ETH Zurich, Switzerland · Department of Electrical, Electronic and Information Engineering (DEI), University of Bologna, Italy · Department of Control and Computer Engineering (DAUIN), Politecnico di Torino, Italy

Jul 2, 2026J/K move · Enter open · S save

Efficient PEFT Methods with Adaptive Checkpointing for Vision Models and VLMs on Resource Constrained Consumer-GPUs

Altay Toktassyn, Jurn-Gyu Park

Department of Computer Science, Nazarbayev University, Astana 010000, Kazakhstan

Fine-Tuning Memory-Efficient Fine-Tuning Vision Foundation Model Adaptation Parameter-Efficient Fine-Tuning

Open paper AskPDF arXiv

Modern pretrained vision models achieve strong accuracy but demand substantial GPU memory for fine-tuning, making edge deployment impractical. This paper compares five parameter-efficient fine-tuning (PEFT) methods (Full FT, LoRA, AdaLoRA, QLoRA, BitFit) on Transformers- (ViT-Small, TinyViT) and Mamba-based vision backbones (Vim-Small, MambaVision-T) under an on-device VRAM budget (e.g., 2 GB), together with three gradient-checkpointing strategies (none, static, and a proposed memory-budget-aware adaptive algorithm); and we evaluate three families of foundation-model baselines: zero-shot contrastive vision language models (OpenCLIP, SigLIP), self-supervised vision backbones with lightweight evaluation protocols (DINOv2), and autoregressive VLMs for prompt-based classification (PaliGemma, MobileVLM, SmolVLM). Experiments on CIFAR-100 and DTD report accuracy, training time, energy, and the NetScore family of multi-objective metrics, which we extend with two deployment-aware variants. QLoRA and BitFit cut energy 20-30% at a 1-2% accuracy cost; the adaptive algorithm reduces peak memory 43-79% with 9-30% energy overhead. DINOv2 surpasses fine-tuned models on CIFAR-100 (0.917 vs. 0.897) at a fraction of the energy, while small autoregressive VLMs remain uncompetitive.

Abstract

Fine-Tuning

Explore similar work

Efficient PEFT Methods with Adaptive Checkpointing for Vision Models and VLMs on Resource Constrained Consumer-GPUs

Circuit Fine-Tuning for Compute-Efficient Transformer Adaptation

TrainDeeploy: Hardware-Accelerated Parameter-Efficient Fine-Tuning of Small Transformer Models at the Extreme Edge