Adapting Frozen Mono-modal Backbones for Multi-modal Registration via Contrast-Agnostic Instance Optimization

Yi Zhang; Yidong Zhao; Qian Tao

eess.IVMar 27, 2026

Adapting Frozen Mono-modal Backbones for Multi-modal Registration via Contrast-Agnostic Instance Optimization

Authors: Yi Zhang, Yidong Zhao, Qian Tao

Organizations: Department of Imaging Physics, Delft University of Technology, The Netherlands

Abstract

Deformable image registration remains a central challenge in medical image analysis, particularly under multi-modal scenarios where intensity distributions vary significantly across scans. While deep learning methods provide efficient feed-forward predictions, they often fail to generalize robustly under distribution shifts at test time. A straightforward remedy is full network fine-tuning, yet for modern architectures such as Transformers or deep U-Nets, this adaptation is prohibitively expensive in both memory and runtime when operating in 3D. Meanwhile, the naive fine-tuning struggles more with potential degradation in performance in the existence of drastic domain shifts. In this work, we propose a registration framework that integrates a frozen pretrained mono-modal registration model with a lightweight adaptation pipeline for multi-modal image registration. Specifically, we employ style transfer based on contrast-agnostic representation generation and refinement modules to bridge modality and domain gaps with instance optimization at test time. This design is orthogonal to the choice of backbone mono-modal model, thus avoids the computational burden of full fine-tuning while retaining the flexibility to adapt to unseen domains. We evaluate our approach on the Learn2Reg 2025 LUMIR validation set and observe consistent improvements over the pretrained state-of-the-art mono-modal backbone. In particular, the method ranks second on the multi-modal subset, third on the out-of-domain subset, and achieves fourth place overall in Dice score. These results demonstrate that combining frozen mono-modal models with modality adaptation and lightweight instance optimization offers an effective and practical pathway toward robust multi-modal registration.

Figures & tables

Figure 1: Overview of the proposed framework. Input images are optionally passed through a contrast-agnostic preprocessing (Brain-ID) when large modality differences are detected. A frozen pretrained backbone $f_{\theta}$ produces an initialization $\phi_{0}$ , which is refined by instance optimization with lightweight multi-scale modules. The final deformation field $\phi_{T}$ is obtained by composing these refinements.

Method	Overall DICE $\uparrow$	NDV (%) $\downarrow$	In-domain $\uparrow$	Out-of-domain $\uparrow$	Multi-modal $\uparrow$
Zero Displacement	$0.5416\pm 0.0341$	-	0.5668	0.5189	0.5391
ConvexAdam-MIND [ 24 ]	$0.6732\pm 0.0275$	0.0010	0.6998	0.6621	0.6578
SynthMorph [ 10 ]	$0.7012\pm 0.0265$	0.0000	0.7262	0.6888	0.6888
VFA-SynthSR [ 11 ]	$0.7536\pm 0.0260$	0.0074	0.7744	0.7545	0.7320
MultiGradICON [ 7 ]	$0.7429\pm 0.0236$	0.0017	0.7611	0.7453	0.7222
VFA-Brain-ID (no IO)	$0.7563\pm 0.0239$	0.0791	0.7743	0.7548	0.7397

Table 1: Performance of official baselines and our variants on the validation leaderboard. Due to space constraints, TRE and HdDist95 are omitted; the full results are available on the official leaderboard.

Method	Overall DSC $\uparrow$	In-domain $\uparrow$	Out-of-domain $\uparrow$	Multi-modal $\uparrow$
VFA Multi-modal (w/o IO)	$0.8874\pm 0.0182$	$0.9001\pm 0.0085$	$0.8828\pm 0.0195$	$0.8837\pm 0.0187$
VFA-Brain-ID (w/o IO)	$0.8896\pm 0.0180$	$0.9017\pm 0.0088$	$0.8896\pm 0.0187$	$0.8830\pm 0.0190$
VFA-Brain-ID (full finetune)	$0.8863\pm 0.0188$	$0.8990\pm 0.0079$	$0.8817\pm 0.0204$	$0.8820\pm 0.0200$
VFA-Brain-ID (w/o pretrain)	$0.8930\pm 0.0184$	$0.9057\pm 0.0086$	$0.8941\pm 0.0200$	$0.8857\pm 0.0181$
VFA-Brain-ID (1K)	$\textbf{0.8938}\pm\textbf{0.0180}$	$\textbf{0.9068}\pm\textbf{0.0072}$	$\textbf{0.8945}\pm\textbf{0.0200}$	$\textbf{0.8861}\pm\textbf{0.0178}$
VFA-Brain-ID (5K)	$0.8929\pm 0.0181$	$0.9048\pm 0.0074$	$0.8933\pm 0.0205$	$0.8861\pm 0.0180$

Table 2: Local validation results on the organizers’ reduced 10-class label set. Note that these results are for development only and may not perfectly correlate with the official leaderboard metrics.

Figure 2: Dynamics of instance optimization on the 10-class validation subset with VFA-Brain-ID (1K). Dice score is recorded per 5 steps.

Explore similar work

Apr 10, 2026

Search-MIND: Training-Free Multi-Modal Medical Image Registration

Multi-modal image registration plays a critical role in precision medicine but faces challenges from non-linear intensity relationships and local optima. While deep learning models enable rapid inference, they often suffer from generalization collapse on unseen modalities. To address this, we propose Search-MIND, a training-free, iterative optimization framework for instance-specific registration. Our pipeline utilizes a coarse-to-fine strategy: a hierarchical coarse alignment stage followed by deformable refinement. We introduce two novel loss functions: Variance-Weighted Mutual Information (VWMI), which prioritizes informative tissue regions to shield global alignment from background noise and uniform regions, and Search-MIND (S-MIND), which broadens the convergence basin of structural descriptors by considering larger local search range. Evaluations on CARE Liver 2025 and CHAOS Challenge datasets show that Search-MIND consistently outperforms classical baselines like ANTs and foundation model-based approaches like DINO-reg, offering superior stability across diverse modalities.

Boya Wang, Ruizhe Li, Chao Chen +1

Intelligent Modelling & Analysis Group (IMA), School of Computer Science, University of Nottingham, UK · Lab for Uncertainty in Data and Decision Making (LUCID), Nottingham Biomedical Research Centre (BRC), School of Medicine, University of Nottingham, UK

Medical Image Registration Cross-Modal Image Registration Deformable Image Registration

Ask PDF

Mar 20, 2026

Disentangle-then-Align: Non-Iterative Hybrid Multimodal Image Registration via Cross-Scale Feature Disentanglement

Multimodal image registration is a fundamental task and a prerequisite for downstream cross-modal analysis. Despite recent progress in shared feature extraction and multi-scale architectures, two key limitations remain. First, some methods use disentanglement to learn shared features but mainly regularize the shared part, allowing modality-private cues to leak into the shared space. Second, most multi-scale frameworks support only a single transformation type, limiting their applicability when global misalignment and local deformation coexist. To address these issues, we formulate hybrid multimodal registration as jointly learning a stable shared feature space and a unified hybrid transformation. Based on this view, we propose HRNet, a Hybrid Registration Network that couples representation disentanglement with hybrid parameter prediction. A shared backbone with Modality-Specific Batch Normalization (MSBN) extracts multi-scale features, while a Cross-scale Disentanglement and Adaptive Projection (CDAP) module suppresses modality-private cues and projects shared features into a stable subspace for matching. Built on this shared space, a Hybrid Parameter Prediction Module (HPPM) performs non-iterative coarse-to-fine estimation of global rigid parameters and deformation fields, which are fused into a coherent deformation field. Extensive experiments on four multimodal datasets demonstrate state-of-the-art performance on rigid and non-rigid registration tasks. The code is available at the project website.

Chunlei Zhang, Jiahao Xia, Yun Xiao +2

Faculty of Engineering and IT, University of Technology Sydney, Australia · School of Artificial Intelligence, Anhui University, Hefei, China · School of Computer Science and Technology, Anhui University, Hefei, China

Cross-Modal Image Registration Deformable Image Registration Multimodal Disentangled Representation Learning

Ask PDF

Apr 7, 2026

CRFT: Consistent-Recurrent Feature Flow Transformer for Cross-Modal Image Registration

We present Consistent-Recurrent Feature Flow Transformer (CRFT), a unified coarse-to-fine framework based on feature flow learning for robust cross-modal image registration. CRFT learns a modality-independent feature flow representation within a transformer-based architecture that jointly performs feature alignment and flow estimation. The coarse stage establishes global correspondences through multi-scale feature correlation, while the fine stage refines local details via hierarchical feature fusion and adaptive spatial reasoning. To enhance geometric adaptability, an iterative discrepancy-guided attention mechanism with a Spatial Geometric Transform (SGT) recurrently refines the flow field, progressively capturing subtle spatial inconsistencies and enforcing feature-level consistency. This design enables accurate alignment under large affine and scale variations while maintaining structural coherence across modalities. Extensive experiments on diverse cross-modal datasets demonstrate that CRFT consistently outperforms state-of-the-art registration methods in both accuracy and robustness. Beyond registration, CRFT provides a generalizable paradigm for multimodal spatial correspondence, offering broad applicability to remote sensing, autonomous navigation, and medical imaging. Code and datasets are publicly available at https://github.com/NEU-Liuxuecong/CRFT.

Xuecong Liu, Mengzhu Ding, Zixuan Sun +2

Northeastern University, China · National University of Defense Technology, China

Cross-Modal Image Registration Cross-Modal Feature Matching

Ask PDF