cs.CVMar 13, 2026

NOIR: Neural Operator mapping for Implicit Representations

Authors: Sidaty El Hadramy, Nazim Haouchine, Michael Wehrli, Philippe C. Cattin

Organizations: Department of Biomedical Engineering, University of Basel, Allschwil, Switzerland · Harvard Medical School, Brigham and Women’s Hospital, Boston, United States

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

This paper presents NOIR, a framework that reframes core medical imaging tasks as operator learning between continuous function spaces, challenging the prevailing paradigm of discrete grid-based deep learning. Instead of operating on fixed pixel or voxel grids, NOIR embeds discrete medical signals into shared Implicit Neural Representations and learns a Neural Operator that maps between their latent modulations, enabling resolution-independent function-to-function transformations. We evaluate NOIR across multiple 2D and 3D downstream tasks, including segmentation, shape completion, image-to-image translation, and image synthesis, on several public datasets such as Shenzhen, OASIS-4, SkullBreak, fastMRI, as well as an in-house clinical dataset. It achieves competitive performance at native resolution while demonstrating strong robustness to unseen discretizations, and empirically satisfies key theoretical properties of neural operators. The project page is available here: https://github.com/Sidaty1/NOIR-io.

Figures & tables

Figure 1: Downstream tasks solved by NOIR . By formulating each task as operator learning between function spaces, NOIR addresses segmentation, completion, synthesis, and translation across in 2D and 3D, with inherent robustness to resolution changes.
Figure 2: Architecture of NOIR . Discrete input ( fidf^{d}_{i} ) and output ( gidg^{d}_{i} ) signals are embedded as continuous functions ( fi∣Xf_{i|\mathcal{X}} and gi∣Xg_{i|\mathcal{X}} ) using INRs with shared dataset-level parameters and signal-specific modulations. A neural operator maps the modulations of the input ( zinz_{in} ) and output ( zoutz_{out} ) INRs, resulting in a continuous function-to-function mapping applicable to multiple downstream tasks.
Figure 3
DatasetMethodRes = 10%10\%Res = 25%25\%Res = 100%100\%
DSC ↑\uparrowIoU ↑\uparrowDSC ↑\uparrowIoU ↑\uparrowDSC ↑\uparrowIoU ↑\uparrow
Shenzhen [ 28 ]U-Net [ 51 ]0.750.620.920.870.950.91
ViT [ 8 ]0.910.840.930.870.940.90
AttU-Net [ 46 ]0.010.010.740.640.950.91
FNO [ 37 ]0.920.860.940.880.940.89
 NOIR (Ours)0.930.880.940.880.940.88
Table 1: Performance of NOIR and baseline methods across segmentation and anatomy completion tasks.
Figure 3: Qualitative results for segmentation and anatomy completion. Rows correspond to Shenzhen binary segmentation, OASIS-4 multi-label segmentation, and SkullBreak anatomy completion, respectively. A randomly selected test sample is shown with the input image, ground truth, NOIR results, and comparison methods.
Figure 4: Qualitative results for image synthesis (top row) and image translation (bottom row). A randomly selected test sample is shown, including the input image, ground truth, NOIR results, and results from comparison methods.
TaskMethodRes = 25%25\%Res = 50%50\%Res = 100%100\%
PSNR ↑\uparrowSSIM ↑\uparrowPSNR ↑\uparrowSSIM ↑\uparrowPSNR ↑\uparrowSSIM ↑\uparrow
US SynthesisU-Net [ 51 ]17.510.8417.450.8417.540.89
ViT [ 8 ]28.220.7828.310.7928.360.79
AttU-Net [ 46 ]27.760.8728.290.8928.550.90
FNO [ 37 ]27.370.7327.940.7528.150.76
DDPM [ 26 ]26.350.6526.510.6526.910.64
Table 2: Image-to-image translation performance for domain adaptation and ultrasound synthesis. PSNR and SSIM are reported across different resolutions.
Figure 5: Resolution robustness of NOIR. (a) Dice distributions across discretizations show stable performance with mean Dice around 0.930.93 – 0.940.94 . (b) Qualitative example demonstrating consistent segmentation predictions across different input resolutions.
DatasetOperator T\mathcal{T}DSC ↑\uparrowIoU ↑\uparrow
Shenzhen ( p=64p=64 )Linear Regression0.930.87
SVR0.940.88
Random Forest0.930.86
MLP0.940.88
OASIS-4 ( p=2048p=2048 )Linear Regression0.800.68
SVR0.790.66
Table 3: Ablation study on NO model selection.
DatasetInOut#HLHSHActLActLat#HHLHHSLRLLROpt
Shenzhen216256SineSigmoid6416410−210^{-2}5×10−65\times 10^{-6}AdamW
OASIS-4216256SineSigmoid204816410−210^{-2}5×10−65\times 10^{-6}AdamW
SkullBreak3110256SineSoftmax6416410−210^{-2}5×10−65\times 10^{-6}AdamW
fastMRI2110256SineSigmoid409616410−210^{-2}5×10−65\times 10^{-6}AdamW
Ultrasound2110256SineSigmoid6416410−210^{-2}5×10−65\times 10^{-6}AdamW
Table S1 : Training configuration for input implicit neural representations (INRs).
DatasetInOut#HLHSHActLActLat#HHLHHSLRLLROpt
Shenzhen226256SineSoftmax6416410−210^{-2}5×10−65\times 10^{-6}AdamW
OASIS-42510256SineSoftmax204816410−210^{-2}5×10−65\times 10^{-6}AdamW
SkullBreak326256SineSoftmax6416410−210^{-2}5×10−65\times 10^{-6}AdamW
fastMRI2110256SineSigmoid409616410−210^{-2}5×10−65\times 10^{-6}AdamW
Ultrasound2110256SineSigmoid6416410−210^{-2}5×10−65\times 10^{-6}AdamW
Table S2 : Training configuration for output implicit neural representations (INRs).
DatasetInOut#HLHDActResLROptWDDropout
Shenzhen64641128SiLUTrue10−410^{-4}AdamW10−410^{-4}0
OASIS-42048204832048SiLUTrue10−410^{-4}AdamW10−410^{-4}0.2
SkullBreak646412048SiLUTrue10−410^{-4}AdamW10−410^{-4}0
fastMRI4096409612048SiLUTrue5×10−55\times 10^{-5}AdamW10−410^{-4}0.1
Ultrasound64643512ReLUTrue7×10−47\times 10^{-4}AdamW10−510^{-5}0
Table S3 : Training configuration for the neural operator (NO) mapping input to output INRs.
DatasetImage SizeTrainValTestNum. Points
Shenzhen [ 28 ]200×200200\times 2000.80.10.140000
OASIS-4 [ 42 ]160×192160\times 1920.70.150.1530720
SkullBreak [ 32 ]512×512×512512\times 512\times 512From [ 32 ]From [ 32 ]From [ 32 ]40000
fastMRI [ 31 ]128×128128\times 1280.80.10.110000
Ultrasound (in-house)64×6464\times 640.80.10.11820
Table S4 : Dataset information including training, validation, and test splits, image sizes, and number of points/slices used for each experiment.
 Input INROutput INR
DatasetPSNR ↑\uparrowSSIM ↑\uparrowDSCIoUPSNR ↑\uparrowSSIM ↑\uparrowDSC ↑\uparrowIoU ↑\uparrow
Shenzhen [ 28 ]22.720.66––––0.980.97
OASIS-4 [ 42 ]24.760.79––––0.910.84
SkullBreak [ 32 ]––0.870.77––0.900.83
fastMRI [ 31 ]22.260.65––29.090.71––
Ultrasound33.770.94––33.480.70––
Table S5 : Reconstruction performance of the input and output INRs across datasets.
Figure S1 : Reconstruction results from the output INR for the Shenzhen dataset. Each example shows the ground truth image and the reconstruction produced by the output INR.
Figure S2 : Reconstruction results from the output INR for the OASIS-4 dataset. Ground truth segmentations are compared with the corresponding INR reconstructions.
Figure S3 : Reconstruction results from the output INR for the fastMRI dataset. The reconstructed images are compared with the ground truth MRI slices.
Figure S4 : Reconstruction results from the output INR for the ultrasound dataset, showing the ability of the INR to approximate the ultrasound signal from the learned latent representation.
Figure S5 : Confusion matrices summarizing pairwise discrepancies between latent modulation vectors obtained at resolutions 32×3232\times 32 , 64×6464\times 64 , 128×128128\times 128 , 160×160160\times 160 , and 200×200200\times 200 . Left: discrepancies between optimized input modulation vectors. Right: discrepancies between predicted output modulation vectors produced by the neural operator.
DatasetNOIRU-NetViTAttU-NetFNODDPM
Shenzhen [ 28 ]8.30×10−18.30\times 10^{-1}3.06×10−33.06\times 10^{-3}3.07×10−33.07\times 10^{-3}4.72×10−34.72\times 10^{-3}2.94×10−32.94\times 10^{-3}–
OASIS-4 [ 42 ]2.41×10−12.41\times 10^{-1}2.69×10−32.69\times 10^{-3}5.75×10−35.75\times 10^{-3}3.57×10−33.57\times 10^{-3}2.63×10−32.63\times 10^{-3}–
SkullBreak [ 32 ]3.45×10−13.45\times 10^{-1}2.13×10−12.13\times 10^{-1}1.06×10−11.06\times 10^{-1}2.38×10−12.38\times 10^{-1}1.23×10−11.23\times 10^{-1}–
fastMRI [ 31 ]1.581.581.89×10−31.89\times 10^{-3}2.71×10−32.71\times 10^{-3}2.78×10−32.78\times 10^{-3}1.59×10−31.59\times 10^{-3}5.275.27
Ultrasound5.93×10−15.93\times 10^{-1}3.00×10−33.00\times 10^{-3}4.04×10−34.04\times 10^{-3}4.69×10−34.69\times 10^{-3}1.62×10−31.62\times 10^{-3}2.72×10−12.72\times 10^{-1}
Table S6 : Prediction time comparison (in seconds).
More in this topic

Neural Operators

Neural operators learn mappings between functions, enabling fast surrogates for PDEs and other continuous physical systems.

See all 310 papers

Explore similar work

CardsList

  1. Disentangled Learning Improves Implicit Neural Representations for Medical Reconstruction

    May 5, 2026Qing Wu, Xuanyu Tian, Chenhe Du +4Disentangled Representation LearningImplicit Neural Representations

    Medical AI Lab, Ant Group · ShanghaiTech University · Shanghai Jiao Tong University

  2. Implicit representations are dead. Long live explicit primitives!

    Aug 7, 2026Nil Stolt-Ansó, Maik Dannecker, Wenqi Huang +23DGS CompressionImplicit Neural Representations

    Technical University Munich, Germany · University of Zurich, Switzerland · Imperial College London, United Kingdom

  3. Neural Operators as Efficient Function Interpolators

    May 8, 2026Vasilis Niarchos, Angelos Sirbu, Sokratis TrifinopoulosNeural Surrogate ModelingFourier Neural Operator

    Department of Physics, CCTP and ITCP, University of Crete, 71303, Greece · Theoretical Physics Department, CERN, Geneva, Switzerland · Physik-Institut, Universität Zürich, 8057 Zürich, Switzerland