Residual-Guided Adaptive Tucker Rank Allocation for Parameter-Efficient MCI Conversion Prediction

Guanghua He; Hancan Zhu; Gaohang Yu; An Zhang

cs.LGOct 19, 2026

Residual-Guided Adaptive Tucker Rank Allocation for Parameter-Efficient MCI Conversion Prediction

Authors: Guanghua He, Hancan Zhu, Gaohang Yu, An Zhang

Organizations: Department of Mathematics, Hangzhou Dianzi University, Hangzhou, 310018, China · School of Mathematics, Physics and Information, Shaoxing University, Shaoxing, 312000, China · Department of Mathematics, Zhejiang University of Science and Technology, Hangzhou, 310023, China

Abstract

Allocating limited trainable capacity across convolutional layers is a central challenge in parameter-efficient fine-tuning (PEFT) of pretrained three-dimensional (3D) networks. We propose Residual-Guided Adaptive Tucker Rank Allocation, which jointly selects input- and output-channel ranks under a fixed adapter-parameter budget while preserving the spatial kernel modes. To guide rank selection, we learn task-specific proxy updates from downstream training data and approximate them using candidate Tucker rank pairs. A perturbation analysis bounds the output discrepancy between the fixed proxy network and its Tucker approximation in terms of layerwise reconstruction residuals, motivating residual-based allocation. We normalize each squared residual by the corresponding proxy-update energy to compare relative approximation errors across layers. Dynamic programming minimizes their sum under a global budget constraint that accounts for both channel factors and the Tucker core. This yields a globally optimal allocation over the specified candidate rank pairs. We evaluate the method on the Alzheimer's Disease Neuroimaging Initiative (ADNI) cohort for predicting conversion from mild cognitive impairment to Alzheimer's disease within 36 months, using hippocampal segmentation pretraining. Across five random seeds, the proposed method achieves a mean area under the receiver operating characteristic curve (AUC) of 0.7706, compared with 0.7239 for fixed-ratio Tucker adaptation and 0.7430 for full fine-tuning. The total number of trainable parameters is limited to approximately 0.618 million, accounting for only 1.45% of the full model. Adaptive Tucker rank allocation thus provides an effective approach to parameter-efficient transfer learning in 3D medical imaging.

Figures & tables

Symbol	Meaning
$\mathcal{W}_{\ell},\Delta\mathcal{W}_{\ell}$	Pretrained kernel and trainable adapter update at layer $\ell$
$\Delta\mathcal{W}_{\ell}^{\mathrm{proxy}}$	Calibration-induced kernel update, held fixed during rank allocation
$\bm{r}_{\ell},\bm{r}_{\ell,q}$	Channel-rank pair at layer $\ell$ and its $q$ th candidate pair
$\bm{r},\mathcal{R}$	Network-wide rank configuration and Cartesian product of layerwise candidate sets
$\mathcal{H}_{\bm{s}}(\mathcal{X})$	Channel-mode HOSVD approximation of $\mathcal{X}$ at rank pair $\bm{s}$
$\mathcal{E}_{\ell}(\bm{r}_{\ell}),\mathcal{E}_{\ell,q}$	Proxy-update reconstruction residual tensor and its candidate-indexed form

Table 1: Main notation for residual-guided Tucker allocation.

Figure 1: Overview of residual-guided Tucker. Task-specific proxy updates provide normalized reconstruction residuals for joint input- and output-channel rank allocation under a fixed adapter-parameter budget. Final PEFT training starts from the original pretrained checkpoint with adapters initialized to produce zero updates at the selected ranks.

Characteristic	sMCI $n=305$	pMCI $n=264$	Total $n=569$
Age (years), mean $\pm$ SD a	$71.69\pm 7.63$	$73.93\pm 7.07$	$72.73\pm 7.45$
Female, $n$ (%)	120 (39.34)	105 (39.77)	225 (39.54)
Male, $n$ (%)	185 (60.66)	159 (60.23)	344 (60.46)

Table 2: Baseline characteristics of the ADNI cohort.

Method	Trainable (M)	AUC	ACC	F1	SEN	SPE
Full fine-tuning	$42.691$	$0.7430$ [-1pt] $(0.0099)$	$0.6684$ [-1pt] $(0.0218)$	$0.6100$ [-1pt] $(0.0229)$	$0.5585$ [-1pt] $(0.0413)$	$\mathbf{0.7639}$ [-1pt] $(0.0539)$
LoRA	$0.646$	$0.7301$ [-1pt] $(0.0243)$	$0.6579$ [-1pt] $(0.0206)$	$0.6194$ [-1pt] $(0.0403)$	$0.6038$ [-1pt] $(0.0789)$	$0.7049$ [-1pt] $(0.0544)$
AdaLoRA	$0.618$	$0.7218$ [-1pt] $(0.0102)$	$0.6632$ [-1pt] $(0.0338)$	$0.6368$ [-1pt] $(0.0457)$	$0.6377$ [-1pt] $(0.0631)$	$0.6852$ [-1pt] $(0.0356)$
PiSSA	$0.646$	$0.7218$ [-1pt] $(0.0186)$	$0.6579$ [-1pt] $(0.0317)$	$0.6022$ [-1pt] $(0.0309)$	$0.5585$ [-1pt] $(0.0605)$	$0.7443$ [-1pt] $(0.0849)$
PoLAR	$0.647$	$0.7203$ [-1pt] $(0.0156)$	$0.6579$ [-1pt] $(0.0240)$	$0.6001$ [-1pt] $(0.0570)$	$0.5623$ [-1pt] $(0.1135)$	$0.7410$ [-1pt] $(0.0700)$
Fixed-ratio Tucker	$0.618$	$0.7239$ [-1pt] $(0.0093)$	$0.6561$ [-1pt] $(0.0209)$	$0.6046$ [-1pt] $(0.0542)$	$0.5736$ [-1pt] $(0.0958)$	$0.7279$ [-1pt] $(0.0526)$

Table 3: Predictive performance of the compared methods. Performance values are means across five seeds, with sample SDs in parentheses below. Bold denotes the highest mean in each performance column. For AdaLoRA, the parameter count refers to the final compact allocation.

Figure 2: Mean test AUC versus trainable parameter count for the compared methods. The horizontal axis is broken to separate the PEFT methods from full fine-tuning; the two segments use different scales. M denotes millions of parameters. For AdaLoRA, the parameter count refers to the final compact allocation.

Configuration	Trainable (M)	AUC	ACC	F1	SEN	SPE
Head-only	$0.066$	$0.6820$ [-1pt] $(0.0107)$	$0.6351$ [-1pt] $(0.0229)$	$0.5520$ [-1pt] $(0.0128)$	$0.4830$ [-1pt] $(0.0103)$	$0.7672$ [-1pt] $(0.0484)$
Common only	$0.250$	$0.7250$ [-1pt] $(0.0133)$	$0.6579$ [-1pt] $(0.0164)$	$0.5903$ [-1pt] $(0.0290)$	$0.5321$ [-1pt] $(0.0488)$	$0.7672$ [-1pt] $(0.0425)$
Fixed-ratio Tucker + common	$0.618$	$0.7239$ [-1pt] $(0.0093)$	$0.6561$ [-1pt] $(0.0209)$	$0.6046$ [-1pt] $(0.0542)$	$0.5736$ [-1pt] $(0.0958)$	$0.7279$ [-1pt] $(0.0526)$
Residual-guided Tucker + common	$0.618$	$0.7706$ [-1pt] $(0.0381)$	$0.7035$ [-1pt] $(0.0474)$	$0.6763$ [-1pt] $(0.0694)$	$0.6755$ [-1pt] $(0.1079)$	$0.7279$ [-1pt] $(0.0587)$

Table 4: Contribution of the trainable components. Values are means across five seeds, with sample SDs in parentheses below. Common denotes the classification head, channel-changing shortcuts, and BatchNorm affine parameters.

Allocation score	AUC	ACC	F1	SEN	SPE
Absolute residual	$0.7524$ [-1pt] $(0.0253)$	$0.6895$ [-1pt] $(0.0395)$	$0.6480$ [-1pt] $(0.0647)$	$0.6226$ [-1pt] $(0.0962)$	$\mathbf{0.7475}$ [-1pt] $(0.0220)$
Normalized residual (proposed)	$\mathbf{0.7706}$ [-1pt] $(0.0381)$	$\mathbf{0.7035}$ [-1pt] $(0.0474)$	$\mathbf{0.6763}$ [-1pt] $(0.0694)$	$\mathbf{0.6755}$ [-1pt] $(0.1079)$	$0.7279$ [-1pt] $(0.0587)$

Table 5: Effect of residual normalization under a common adapter-budget ceiling. Values are means across five seeds, with sample SDs in parentheses below. Bold denotes the higher mean in each column.

Figure 3: Layerwise channel ranks and adapter parameter counts for residual-guided Tucker and fixed-ratio Tucker (seed 42).

Method	Adapter (M)	Trainable (M)	Trainable (%)	FLOPs (G)	Peak memory (GiB)	Test time (s)
Full fine-tuning	–	$42.691$	$100.00$	$313.801$	$2.703$	$4.955$
LoRA	$0.396$	$0.646$	$1.51$	$314.141$	$2.233$	$4.943$
AdaLoRA	$0.368$	$0.618$	$1.45$	$314.095$	$2.258$	$4.948$
PiSSA	$0.396$	$0.646$	$1.51$	$314.141$	$2.233$	$4.998$
PoLAR	$0.397$	$0.647$	$1.51$	$314.141$	$2.233$	$4.994$
Fixed-ratio Tucker	$0.368$	$0.618$	$1.45$	$316.543$	$2.235$	$5.134$

Table 6: Parameter and computational efficiency. FLOPs are measured for a cold forward pass, and memory denotes peak allocated memory during final training. Trainable parameter percentages are relative to full fine-tuning. Test-set processing time includes data loading and transfer. AdaLoRA is evaluated at its final compact allocation. Measurements exclude calibration and initialization.

Adaptive Rank Allocation

Adaptive rank allocation distributes low-rank model capacity across layers, modules, tasks, or inputs according to their estimated needs.

See all 30 papers

GLT-PEFT: Gated Lie-Tucker Parameter-Efficient Fine-Tuning for Alzheimer's Disease Diagnosis with Hippocampal Segmentation Pretraining
May 16, 2026Guanghua He, Hancan Zhu, Gaohang Yu +1Medical Diagnosis Low-Rank Adaptation
Department of Mathematics, Hangzhou Dianzi University, Hangzhou, 310018, China · School of Mathematics, Physics and Information, Shaoxing2026 University, Shaoxing, 312000, China · Department of Mathematics, Zhejiang University of Science and Technology, Hangzhou, 310023, China
Post-Optimization Adaptive Rank Allocation for LoRA
Apr 30, 2026Vishnuprasadh Kumaravelu, Sunil Gupta, P. K. SrijithLow-Rank Adaptation Low-Rank Compression
1Indian Institute of Technology Hyderabad · 2Deakin University
RSLoRA: Training-free Rank Allocation for LoRA via Representational Sensitivity Probing
Jul 5, 2026Jiaqi Liu, Haidong Kang, Qihui Zhao +1Low-Rank Adaptation Adaptive Rank Allocation
School of Computer Science and Technology Dalian University of Technology · Hebei Key Laboratory of Marine Perception Network and Data Processing Northeastern University · School of Computer and Communication Engineering Northeastern University +1

May 16, 2026J/K move · Enter open · S save

GLT-PEFT: Gated Lie-Tucker Parameter-Efficient Fine-Tuning for Alzheimer's Disease Diagnosis with Hippocampal Segmentation Pretraining

Guanghua He, Hancan Zhu, Gaohang Yu, An Zhang

Department of Mathematics, Hangzhou Dianzi University, Hangzhou, 310018, China · School of Mathematics, Physics and Information, Shaoxing2026 University, Shaoxing, 312000, China · Department of Mathematics, Zhejiang University of Science and Technology, Hangzhou, 310023, China

Medical Diagnosis Low-Rank Adaptation Parameter-Efficient Fine-Tuning

Open paper AskPDF arXiv

Parameter-efficient fine-tuning (PEFT) has emerged as a promising paradigm for adapting pretrained models under limited data conditions. However, most existing PEFT methods are designed for matrix-structured parameters and are not well suited for high-dimensional convolutional kernels in medical imaging models. Moreover, they typically rely on additive updates and lack mechanisms to preserve the geometric structure of pretrained parameters, while multiplicative (geometry-aware) updates are difficult to integrate within a unified framework. To address this issue, this paper proposes GLT-PEFT, a gated Lie-Tucker parameter-efficient fine-tuning framework for Alzheimer's disease (AD) diagnosis. The proposed approach transfers a hippocampal segmentation pretrained model to a downstream classification task. Tucker decomposition enables tensor-aware low-rank adaptation of 3D convolutional kernels, while Lie group-based transformations provide structure-preserving multiplicative updates. A gating mechanism further reconciles additive and multiplicative update forms, resulting in a unified and more stable fine-tuning strategy. Extensive experiments demonstrate that GLT-PEFT achieves effective cross-task transfer while significantly reducing trainable parameters, highlighting its effectiveness for efficient and robust adaptation in medical imaging models.

Abstract

Adaptive Rank Allocation

Explore similar work

GLT-PEFT: Gated Lie-Tucker Parameter-Efficient Fine-Tuning for Alzheimer's Disease Diagnosis with Hippocampal Segmentation Pretraining

Post-Optimization Adaptive Rank Allocation for LoRA

RSLoRA: Training-free Rank Allocation for LoRA via Representational Sensitivity Probing