Transferable Optimization Network for Cross-Domain Image Reconstruction

Yunmei Chen; Chi Ding; Xiaojing Ye

cs.CVMar 8, 2026

Transferable Optimization Network for Cross-Domain Image Reconstruction

Authors: Yunmei Chen, Chi Ding, Xiaojing Ye

Organizations: Department of Mathematics, University of Florida, Gainesville, FL 32611, USA. · Department of Mathematics and Statistics, Georgia State University, Atlanta, GA 30303, USA.

Abstract

We develop a novel transfer learning framework to tackle the challenge of limited training data in image reconstruction problems. The proposed framework consists of two training steps, both of which are formed as bi-level optimizations. In the first step, we train a powerful universal feature-extractor that is capable of learning important knowledge from large, heterogeneous data sets in various domains. In the second step, we train a task-specific domain-adapter for a new target domain or task with only a limited amount of data available for training. Then the composition of the adapter and the universal feature-extractor effectively explores feature which serve as an important component of image regularization for the new domains, and this leads to high-quality reconstruction despite the data limitation issue. We apply this framework to reconstruct under-sampled MR images with limited data by using a collection of diverse data samples from different domains, such as images of other anatomies, measurements of various sampling ratios, and even different image modalities, including natural images. Experimental results demonstrate a promising transfer learning capability of the proposed method.

Figures & tables

Panel of Figure 1 — Figure 1: The flowchart (top) of the $t$ th iteration of Algorithm 1 , the $\mathbf{u}$ -module (middle), and the $\mathbf{v}$ -module (bottom). Each round node represents a variable. Each rectangular node represents a module, mapping, or condition check. The identity mapping is denoted by Id.

Method	PSNR (dB)				SSIM (%)
Method	Brain	Knee	$\text{Cardiac}^{\dagger}$	$\text{Prostate}^{\dagger}$	Brain	Knee	$\text{Cardiac}^{\dagger}$	$\text{Prostate}^{\dagger}$
DnCn Schlemper et al. (2018)	36.02	32.17	33.91	31.32	94.36	80.50	91.66	85.59
LDA Chen et al. (2021)	37.38	32.38	34.78	32.92	95.42	80.63	93.16	89.10
UNet Arshad et al. (2021)	34.05	31.78	32.05	30.74	92.33	79.72	88.50	84.49
U-MRI Liu et al. (2021)	36.40	32.30	34.85	31.69	94.67	80.67	92.88	86.88
Meta Bian et al. (2021)	36.32	32.11	35.61	32.23	94.67	79.81	93.95	87.77

Table 1: Average PSNR and SSIM of reconstructed images on four different anatomies in Section 5.2.1 . The anatomies marked by

\dagger

(cardiac and prostate) have small training datasets. A fixed Cartesian mask of 20% sampling ratio is used to undersample the

k

-spaces of all images.

Panel of Figure 2 — Figure 2: Comparison of cross-anatomy TL reconstruction results using 20% Cartesian sampling ratio in Section 5.2.1 . Top two rows show two instances of reconstructed cardiac images. Bottom rows show two instances of reconstructed prostate images. Blue squares zoom in the corresponding small squares in the images.

Method	PSNR (dB)					SSIM (%)
Method	$10\%$	$20\%$	$30\%$	$15\%^{\dagger}$	$25\%^{\dagger}$	$10\%$	$20\%$	$30\%$	$15\%^{\dagger}$	$25\%^{\dagger}$
DnCn	29.85	35.85	39.29	31.93	35.75	86.82	94.37	96.55	89.92	94.18
LDA	31.27	37.11	40.67	33.26	37.80	89.36	95.36	97.17	92.13	95.83
UNet	26.93	29.16	31.20	28.03	29.97	76.17	79.55	83.49	77.63	81.11
U-MRI	30.60	36.25	39.42	33.58	37.33	86.07	94.54	96.49	91.91	95.22
Meta	27.63	32.97	35.73	30.72	34.44	82.36	91.22	94.06	87.86	92.87

Table 2: Average PSNR and SSIM of reconstructed images with sampling ratios 10%, 15%, 20%, 25%, and 30% in Section 5.2.2 . The sampling ratios marked by

\dagger

(15% and 25%) have small training datasets. For each sampling ratio, the same Cartesian mask is used to undersample the

k

-spaces of all images.

Panel of Figure 3 — Figure 3: Comparison of cross-sampling-ratio TL reconstruction results in Section 5.2.2 . Top two rows show two instances of reconstructed images under 15% sampling ratio. Bottom rows show two instances of reconstructed images under 25% sampling ratio. Blue squares zoom in the corresponding small squares in the images.

Method	PSNR (dB)				SSIM (%)
Method	ImageNet	CIFAR-10	$\text{fastMRI}^{\dagger}$	$\text{Stanford2D}^{\dagger}$	ImageNet	CIFAR-10	$\text{fastMRI}^{\dagger}$	$\text{Stanford2D}^{\dagger}$
DnCn	33.90	38.68	31.90	36.29	92.87	97.09	84.97	92.33
LDA	35.37	41.33	33.42	37.85	94.26	97.99	87.30	94.23
UNet	31.36	36.27	30.88	35.33	89.94	96.40	82.58	91.00
U-MRI	34.33	39.72	30.50	34.48	93.37	97.66	80.96	89.08
Meta	32.81	38.11	30.93	35.04	90.95	96. 26	82.35	90.47

Table 3: Average PSNR and SSIM of reconstructed images on different modality domains in Section 5.2.3 . The modalities marked by

\dagger

(fastMRI and Stanford2D) have small training datasets.

Panel of Figure 4 — Figure 4: Comparison of cross-modality reconstruction results in Section 5.2.3 . Knowledge is transferred from natural image datasets ImageNet and CIFAR-10 to medical image reconstruction. Top two rows show two instances of reconstructed images in the fastMRI dataset. Bottom rows show two instances of reconstructed images in the Stanford2D dataset. Blue squares zoom in the corresponding small squares in the images.

Panel of Figure 5 — Figure 5: Reconstructed images, zoom-in views, and pointwise absolute errors obtained by the compared methods on the cross-anatomy experiment (top three rows), cross-sampling-ratio experiment (middle three rows), and cross-modality experiment (bottom three rows).

Panel of Figure 6 — Figure 6: CKA (left column) and NCC (right column) between $\mathbf{g}_{i}$ trained by LDA on $D_{i}$ and $\mathbf{g}$ trained by U-LDA versus phase number $T$ . Top and bottom rows show the results on brain ( $\mathbf{x}_{1}$ ) and knee ( $\mathbf{x}_{2}$ ) images, respectively.

Panel of Figure 7 — Figure 7: Average PSNR of reconstructed images with three different target domain training sample sizes (5, 40, and 100). Each bar represents the average PSNR and standard deviation (black interval) obtained by the corresponding method over five tests.

Methods			Results (PSNR)
Adapter	Init	AUG	Cross-Anatomy	Cross-Sampling-Rate	Cross-Modality
$\times$	$\times$	$\times$	32.92	33.26	33.42
$\checkmark$	$\times$	$\times$	32.84	33.48	33.45
$\checkmark$	$\checkmark$	$\times$	33.10	33.68	33.47
$\checkmark$	$\checkmark$	$\checkmark$	33.31	34.90	33.58

Table 4: PSNR of images reconstructed by U-LDA with (

\checkmark

) or without (

\times

) adapters (Adapter), initialization trick of

\mathbf{g}

(Init), and data augmentation (AUG).

$\alpha$	0.1	0.01	0.001
PSNR	36.45	37.88	37.69

Table 5: Average PSNR of images reconstructed by U-LDA using different values of SSIM weight parameter

\alpha

$\delta$	0.0001	0.001	0.01	0.1
PSNR	27.32	37.88	37.79	37.43

Table 6: Average PSNR of images reconstructed by U-LDA using different values of smoothing level

\delta

in the smoothed ReLU activation.

Explore similar work

Mar 13, 2026

Open World MRI Reconstruction with Bias-Calibrated Adaptation

Real-world MRI reconstruction systems face the open-world challenge: test data from unseen imaging centers, anatomical structures, or acquisition protocols can differ drastically from training data, causing severe performance degradation. Existing methods struggle with this challenge. To address this, we propose BiasRecon, a bias-calibrated adaptation framework grounded in the minimal intervention principle: preserve what transfers, calibrate what does not. Concretely, BiasRecon formulates open-world adaptation as an alternating optimization framework that jointly optimizes three components: (1) frequency-guided prior calibration that introduces layer-wise calibration variables to selectively modulate frequency-specific features of the pre-trained score network via self-supervised k-space signals, (2) score-based denoising that leverages the calibrated generative prior for high-fidelity image reconstruction, and (3) adaptive regularization that employs Stein's Unbiased Risk Estimator to dynamically balance the prior-measurement trade-off, matching test-time noise characteristics without requiring ground truth. By intervening minimally and precisely through this alternating scheme, BiasRecon achieves robust adaptation with fewer than 100 tunable parameters. Extensive experiments across four datasets demonstrate state-of-the-art performance on open-world reconstruction tasks.

Jiyao Liu, Shangqi Gao, Lihao Liu +5

Fudan University, Shanghai, China · University of Cambridge, Cambridge, United Kingdom · Shanghai Artificial Intelligence Laboratory, Shanghai, China

Image Reconstruction Domain Adaptation Accelerated MRI Reconstruction Diffusion-Based Inverse Problems

Ask PDF

May 17, 2025

Bridging the Inter-Domain Gap through Low-Level Features for Cross-Modal Medical Image Segmentation

This paper addresses cross-modal medical image segmentation, focusing on MRI-CT transfer in a source-only domain generalization setting. During training, only source-modality samples are available, while unlabeled target-modality images are used for testing. We propose LowBridge, which builds on the observation that cross-modal images share similar low-level features (e.g. edges) as they depict the same types of anatomical structures. Specifically, we first train a generative model to recover the source images from their edge features, followed by training a segmentation model on the generated source images, separately. At test time, edge features from the target images are input to the pretrained generative model to generate source-style target domain images, which are then segmented using the pretrained segmentation network. Experiments on various public datasets demonstrate that LowBridge achieves state-of-the-art performance, outperforming ten existing approaches. Ablation studies further show that LowBridge is compatible with different types of generative and segmentation models, suggesting its generalizability and potential to benefit from future advances in these models. The code will be available at https://github.com/JoshuaLPF/LowBridge.

Pengfei Lyu, Pak-Hei Yeung, Jing Xia +5

Northeastern University, Shenyang, China · Nanyang Technological University, Singapore

Cross-Modal Learning Cross-Modality Medical Image Segmentation Image Segmentation Domain Generalization Cross-Domain Transfer Learning

Ask PDF

Sep 2, 2026

Data-Efficient Networks for Multi-Contrast MRI Reconstruction based on a Generalized Content/Style Prior

Multi-contrast MR scans contain redundant structural information that can be leveraged during reconstruction and potentially accelerate acquisition times. This idea has inspired end-to-end guided reconstruction models, leveraging one or more contrasts to guide the reconstruction of a different contrast. However, these models require large paired multi-contrast raw datasets for training, limiting their application in low-data regimes. In this work, we propose a modular framework, namely CoSMo-RecNet, for learning guided reconstruction models in the low-data regime. At its core is a reusable multi-contrast representation based on a content/style model, which can be learned from large-scale, publicly accessible, unpaired multi-contrast image datasets, without available k-space data. Using this frozen model as a multi-contrast prior and using a set of reference contrasts, the reconstruction problem reduces to a much simpler refinement problem that can be solved by a lightweight unrolled network and thus learned from small, task-specific reconstruction datasets. We demonstrate the efficacy of CoSMo-RecNet by evaluating it on the low-field 0.3 T M4Raw dataset, showing stable reconstruction quality on decreasing the raw training data budget. CoSMo-RecNet achieved higher reconstruction quality with 5 training subjects or lower compared to a parameter-count-matched MoDL trained on 100 subjects. On a data-limited and severely out-of-distribution ultra-low-field 47 mT Halbach scanner dataset, CoSMo-RecNet was superior to other viable strategies, including classical reconstruction, transfer learning, and zero-shot reconstruction.

Chinmay Rao, Efe Ilıcak, Matthias J. P. van Osch +5

Department of Radiology, Leiden University Medical Center, Leiden, The Netherlands · Philips Innovative Technologies, Hamburg, Germany · Philips Cardiologs, Paris, France +1

Accelerated MRI Reconstruction Magnetic Resonance Imaging Data-Efficient Learning

Ask PDF