Machine learning models often struggle to generalize across domains with varying data distributions, such as differing noise levels, leading to degraded performance. Traditional strategies like personalized training, which trains separate models per domain, and joint training, which uses a single model for all domains, have significant limitations in flexibility and effectiveness. To address this, we propose two novel domain adaptation methods for regression tasks based on interpretable unrolled networks--deep architectures inspired by iterative optimization algorithms. These models leverage the functional dependence of select tunable parameters on domain variables, enabling controlled adaptation during inference. Our methods include Parametric Tunable-Domain Adaptation (P-TDA), which uses known domain parameters for dynamic tuning, and Data-Driven Tunable-Domain Adaptation (DD-TDA), which infers domain adaptation directly from input data. We validate our approach on compressed sensing problems involving noise-adaptive sparse signal recovery, domain-adaptive gain calibration, and domain-adaptive phase retrieval, demonstrating improved or comparable performance to domain-specific models while surpassing joint training baselines. This work highlights the potential of unrolled networks for effective, interpretable domain adaptation in regression settings.
Figures & tables
Training approach
D1(6 dB)
D2(16 dB)
D3(31 dB)
NMSE (in dB) , HR(%)
Joint train
-6.60 , 54.22
-12.34 , 76.72
-13.33 , 79.81
Retrain last layer
-6.72 , 53.73
-12.64 , 77.87
-13.79 , 81.17
Retrain last 3 layers
-7.10 , 53.93
-13.92 , 83.40
-16.17 , 89.24
TABLE I: Test results of retrained DNNs for domain adaptation
Fig. 1: Layers/iterations of different LISTA architectures: (a) Conventional LISTA, (b), (c) NA-LISTA. (b) Single layer of the P-DTDA model with input data {y,A} and domain parameter σ available during inference. (c) Single layer of DD-TDA model with only input data {y,A} available during inference. Note: Green blocks represent the data available at inference time, and Gray blocks are trainable parameters.
Fig. 2: Comparison of the proposed methods with the JT and PT methods for NA-LISTA for different SNR ranges. PT’s performances are better than JT’s for each domain. The P-TDA/DD-TDA has comparable performance to PT, demonstrating the noise-tunability aspect.
Fig. 3: Comparison of the proposed methods with the JT and PT methods for NA-LISTA for broad SNR ranges, averaged over 5 experiments.
Model
Parameters
FLOPs
Inference Time ( ×10−3 s)
LISTA
1,007,440
15.098 M
0.016
P-TDA
1,721,697
15.811 M
0.020
DD-TDA
1,720,0703
15.810 M
0.019
Tail-LISTA
12,089,280
96.624 M
0.117
DDIM
74,665,729
8.100 G
14.260
TABLE II: Model complexity and inference efficiency for MNIST compressed sensing. FLOPs are reported per single inference. P-TDA and DD-TDA are NA-LISTA variants.
SNR (dB)
Model
NMSE (dB) ↓
PSNR (dB) ↑
SSIM ↑
Cal. Acc. (%) ↑
Raw Acc. (%) ↑
− 10
LISTA [ 15 ] (JT)
− 10.98
11.38
0.4036
35.07
34.55
P-TDA (Ours)
− 12.83
13.21
0.5139
51.92
51.00
DD-TDA (Ours)
− 12.19
12.61
0.5159
49.51
48.65
Tail-LISTA [ 39 ]
− 11.34
11.85
0.2857
24.07
23.55
DDIM [ 40 ]
− 11.19
11.81
0.3121
28.32
27.70
− 5
LISTA [ 15 ] (JT)
− 13.11
13.64
0.5773
63.99
62.90
TABLE III: Per-SNR reconstruction metrics and OCR accuracy on MNIST test set (2,000 samples per SNR level). OCR accuracy ceiling on clean images is 97.50%. P-TDA and DD-TDA are NA-LISTA variants. ↑ indicates higher is better and ↓ indicates lower is better.
Fig. 4: Qualitative MNIST-CS reconstructions across SNR domains. Columns show ground truth (GT) and reconstructions from JT (LISTA), Tail-LISTA (tight), DDIM (diffusion model) and the proposed NA-LISTA with P-TDA and DD-TDA.
Fig. 5: Qualitative MNIST-CS reconstructions at SNR −10 dB. Columns show ground truth (GT) and reconstructions from JT (LISTA), Tail-LISTA (tight), DDIM (diffusion model) and the proposed NA-LISTA with P-TDA and DD-TDA.
Domains (SNR)
D1 (6 dB)
D2 (16 dB)
D3 (32 dB)
S(W1)
16.35
16.1
15.55
S(W2)
15.07
17.77
15.62
β
0.088
0.048
0.037
TABLE IV: S(W) and β values for PT models
Fig. 6: Comparison of learned W1 matrices for different domains.
Fig. 7: Comparison of top (15×15) of learned W1 matrices.
Fig. 8: Comparison of top (15×15) of learned W2 matrices for different domains.
Fig. 9: Results for generalization experiment: The models (except PT) were trained on D1, D3, and D5 domains. Observation: DD-TDA generalizes well and performs better than JT models.
Fig. 10: Tunable model for blind calibration gain with B as a tuned parameter.
Fig. 11: Comparison of the proposed methods with the JT and PT methods for the sparse gain calibration problem. The P-TDA/DD-TDA has comparable performance to PT, demonstrating the domain-adaptability aspect.
Fig. 12: Generalization performance of models, across domains with structured calibration gains. The JT, P-TDA, and DD-TDA models were trained on 45 domains and tested on 5 unseen domains.
Fig. 13: PRα,L() is the tunable model for sparse phase retrieval. Note: Green symbols represent the data available at inference time. Red symbols are trainable parameters, and blue symbols are fixed parameters.
Fig. 14: Comparison of the proposed methods with the JT and PT methods for Phase Retrieval with sparsity [10, 7, 4] and HR tol: 0.05.
Fig. 15: A comparison of NMSE against the number of parameters and FLOPS for different methods.
K
Metric
DD-TDA
DD-TDA Base
10
NMSE (dB)
−20.96
−1.91
HR (%)
89.2
33.1
7
NMSE (dB)
−41.23
−30.41
HR (%)
97.6
98.0
4
NMSE (dB)
−52.14
−44.88
HR (%)
100.0
100.0
TABLE V: Comparison of DD-TDA with and without Cauchy measurements for phase retrieval
Domain adaptation is crucial for addressing distributional shifts that degrade model performance across domains. While most existing research has centered on classification, semi-supervised domain adaptation regression (SSDAR) for continuous-output tasks remains largely unexplored, particularly in practical scenarios with limited labeled target data. To address this gap, we propose semi-supervised domain adaptation regression through whitening transform and dual consistency (WhiteCon), which combines domain-specific whitening transform (DWT) and dual consistency regularization to enhance training stability and domain adaptation. DWT reduces the variance of the model parameters by transforming the feature covariance matrix into an identity matrix, thus stabilizing training under ordinary least squares assumptions. In addition, variance consistency regularization, as part of dual consistency regularization, aligns the variances of weak, strong, and mixup-augmented features to improve resilience against augmentation-induced perturbations. Empirical evaluations on various benchmark datasets under SSDAR settings demonstrate that the proposed WhiteCon achieves state-of-the-art performance compared to existing methods, effectively addressing domain shifts in regression tasks. The code for WhiteCon is available at https://github.com/sejin-sim/WhiteCon.
Se Jin Sim, Seoung Bum Kim
School of Industrial and Management Engineering Korea University, Seoul, Republic of Korea
Applying pre-trained language models to new domains through full fine-tuning is computationally expensive and prone to catastrophic forgetting. To address this limitation, we introduce a novel parameter-efficient strategy for unsupervised domain adaptation that combines a custom PEFT architecture with mixed-objective training. The proposed method integrates invertible adapters with Low-Rank Adaptation (LoRA) and jointly optimizes classification on labeled source-domain data and masked language modeling on unlabeled target-domain data. This joint training scheme supports task adaptation while preserving knowledge of the target domain. We evaluate the method on the Multi-Genre Natural Language Inference (MNLI) dataset across 20 domain shifts. Our approach achieves average performance improvements of 1.41 percentage points over the parameter-efficient state-of-the-art UDapter, 1.26 percentage points over the fully tuned DANN baseline, and 0.86 percentage points over DSN, while updating only 7% of the model parameters. These findings establish a new state-of-the-art result for parameter-efficient unsupervised domain adaptation and demonstrate that carefully designed PEFT combinations with concurrent optimization can outperform both parameter-efficient and conventional fully tuned approaches.
Mohammed Rawhani, Derviş Karaboğa, Özkan Ufuk Nalbantoğlu +2
Computer Engineering Department Erciyes University Kayseri, Türkiye
Test-time adaptation (TTA) aims to align a model to shifting test domains using only unlabeled streaming data. Most existing methods implicitly infer a single global domain distribution, ignoring the multidimensional and sample-specific nature of real-world domain shifts, leading to fragile adaptation. We propose DOME, an effective domain encoder that explicitly models each sample's domain in a zero-shot manner. DOME leverages vision-language pretraining to extract dense, continuous representations, parameterizes domains as distributional variables, and introduces a momentum-updated sparse domain bank for disentangled supervision. By injecting these explicit domain cues into downstream models, even a basic entropy-minimization TTA strategy achieves state-of-the-art performance across ImageNet-C, ImageNet-R, and ImageNet-Sketch, outperforming complex TTA approaches. Our results demonstrate that robust adaptation stems not from intricate adaptation algorithms, but from explicit, structured domain representation.