sFRC for assessing hallucinations in medical image restoration

Prabhat Kc; Rongping Zeng; Nirmal Soni; Aldo Badano

cs.CVMar 4, 2026

sFRC for assessing hallucinations in medical image restoration

Authors: Prabhat Kc, Rongping Zeng, Nirmal Soni, Aldo Badano

Organizations: Center for Devices and Radiological Health in FDA, Silver Spring, MD, 20993, USA

Abstract

Deep learning (DL) methods are currently being explored to restore images from sparse-view-, limited-data-, and undersampled-based acquisitions in medical applications. Although outputs from DL may appear visually appealing based on likability/subjective criteria (such as less noise, smooth features), they may also suffer from hallucinations. This issue is further exacerbated by a lack of easy-to-use techniques and robust metrics for the identification of hallucinations in DL outputs. In this work, we propose performing Fourier Ring Correlation (FRC) analysis over small patches and concomitantly (s)canning across DL outputs and their reference counterparts to detect hallucinations (termed as sFRC). We describe the rationale behind sFRC and provide its mathematical formulation. The parameters essential to sFRC may be set using predefined hallucinated features annotated by subject matter experts or using imaging theory-based hallucination maps. We use sFRC to detect hallucinations for three undersampled medical imaging problems: CT super-resolution, CT sparse view, and MRI subsampled restoration. In the testing phase, we demonstrate sFRC's effectiveness in detecting hallucinated features for the CT problem and sFRC's agreement with imaging theory-based outputs on hallucinated feature maps for the MR problem. Finally, we quantify the hallucination rates of DL methods on in-distribution versus out-of-distribution data and under increasing subsampling rates to characterize the robustness of DL methods. Beyond DL-based methods, sFRC's effectiveness in detecting hallucinations for a conventional regularization-based restoration method and a state-of-the-art unrolled method is also shown.

Figures & tables

Fig. 1: A range of artifacts, including (a) patient implant-based artifact [ 1 ] ), (b) missing wedge-based acquisition artifacts (described in SI section S2), and (c) hallucinations due to the application of an AI-assisted SRGAN model (described in IV-B1 ) to upsample the low-resolution counterpart of (d) by a factor of $4$ . As opposed to artifacts in (a) and (b), hallucinations in (c) in the yellow box (with the two loops of bowel instead of one contiguous loop) and in the green box (an unwarranted plaque-like structure), only become concretely apparent after a thorough comparison against its normal-resolution reference in (d) . Display windows for (a) is (W: $\,1402$ L: $\,-1358$ ), (b) is (W: $\,780$ L: $\,260$ , and (c-d) is (W: $\,700$ L: $\,50$ ).

	Data Fidelity Analysis	Physical Quality Analysis	Task-based Image Quality Approach
Examples	Image fidelity, visual perceptual quality, distance, and distribution metrics (e.g., PSNR, SSIM, RMSE, FID, KL divergence, etc.).	Traditional Bench testing approaches (e.g., NPS, MTF, HU accuracy, geometric accuracy, signal ghosting, position accuracy, etc.) [ 2 , 3 ] .	Numerical model observers (e.g., Channelized Hotelling Observer, Non-Prewhitening Matched Filter); forced-choice experiments (e.g., 2AFC, 4AFC); multireader multicase (MRMC) studies [ 4 , 5 ] .
Task dependent	No	No	Yes
Designed for or effective in analyzing non-linear methods	No	No	Yes
Purpose	Inference of generic, task-agnostic performance; commonly used for system appraisal [ 6 ] .	Inference of generic, task-agnostic performance; commonly used for system appraisal [ 6 ] .	Inference of human or model performance for specific tasks (e.g., detection or discrimination of subtle lesions or signals) [ 7 , 8 ] .
Hallucinatory inference	No	No	No or not explored

TABLE I: Performance testing methodologies, their primary purposes, and their ability to assess hallucinations in image restoration problems

Fig. 2: Decomposition of a patch containing a hallucinated structure in (g) - indicated by a red arrow - into its frequency-based components in (h-l). (m) is a normal-resolution CT patch. (g) is obtained by upsampling (using SRGAN described in section IV-B1 ) the low-resolution ( downsampled four times) counterpart of (m). (a) is the Fourier transform of (g). (b-f) are different bandpass filters . (h-l) are the image components of (g) obtained by convolving the bandpass filters, as indicated in (b-f), with (g). Similarly, (n-r) are the image components of (m) obtained by convolving the bandpass filters, as shown in (b-f), with (m).

Fig. 3: A visual depiction of our sFRC analysis to detect hallucinated regions. Complimentary pairs of patches (i.e., over the same x-y coordinate) of the same dimension are scanned across the image pairs from the two methods (reference and deep learning images) in (a). FRC is calculated in (b) across all patch pairs from (a). In addition to the FRC threshold used in a typical FRC calculation, we introduce a new threshold called $x_{h_{t}}$ , a line parallel to $y-$ axis. Patches corresponding to the deep learning method whose FRC comparison against their reference counterparts lead to $x_{c_{t}}\leq x_{h_{t}}$ are labeled as candidates to exhibit hallucinations. As shown in fig. 4 , $x_{h_{t}}$ is set a priori using hallucinated ROIs/maps estimated by an imaging theory or annotated by subject matter experts using tuning or developmental set.

Fig. 4: An illustration for setting $x_{h_{t}}$ as an upper bound of $x_{c_{t}}$ s using patches that are labeled as hallucinations by subject matter experts or using an imaging theory.

Fig. 5: An illustration of the imaging theory-based approach to set $x_{h_{t}}$ . In our sFRC analysis of the MRI output post-processed, using a U-Net in (c), $0.75$ is set as its FRC threshold and $0.16$ as its $x_{h_{t}}$ such that the patches labeled as hallucinations from our analysis (indicated by the red bounding boxes in (d)) overlap with the false structured obtained using a theoretical approach proposed by Bhadra et al. in [ 15 ] .

Fig. 6: The purple ROIs indicate hallucinated patches in the SRGAN outputs that were initially missed by our sFRC analysis in the top right scan (000077.IMA). The regions corresponding to the missed hallucinated patches get flagged in the subsequent CT scans in the center right (000080.IMA) and the bottom right (000082.IMA) within the abdominal CT volume of patient L $067$ [ 1 ] . Display window is (W: $400$ L: $50$ ).

Model (test set kernel)	No. of patches detected as	sFRC hallucination
	hallucinations	rate (in $[0,1]$ )
SRGAN (sharp)	$787$	$0.0653$
SRGAN (smooth)	$451$	$0.0374$
SR-WGAN (sharp)	$901$	$0.0748$
SR-WGAN (smooth)	$502$	$0.0417$

TABLE II: sFRC-detected hallucinations on sharp and smooth test sets for SRGAN and SR-WGAN models. Both test sets consist of

188

CT images (i.e.,

188\times 64

patches)

Fig. 7: An explicit illustration of the decay in performance of the SRGAN super-resolution model - trained using smooth data - when analyzed against (b) sharp and (d) smooth kernel-based normal-resolution patches using our sFRC analysis. The smoothly trained SRGAN generated hallucinated indentations in (c), which were detected by our sFRC analysis. In contrast, the same SRGAN upsampled smooth test data more faithfully in (e) and is not flagged by our sFRC analysis. The patches in (b-e) correspond to an ROI of an abdominal CT scan in (a). Display window is (W: $400$ L: $50$ ).

Fig. 8: Plaque-like hallucination that is wrongly added along a bordering region (displayed using the arrow (1) and underfitting of fatty attenuation to air (displayed using the arrow (2)) in the SRGAN-based output as compared to its normal-resolution FBP counterpart. Radiologists are trained to look for even small amounts of air. Such air-based hallucinations may mislead a radiologist’s clinical decision. Display window is (W: $700$ and L: $50$ ).

Fig. 9: A ground truth (in (a)) was used to perform subsampled MRI acquisition by an acceleration factor of 3. The subsampled output is post-processed by a U-Net (in (b), (d)) and reconstructed using PLS-TV (in (c), (e)). Bhadra et. al’s linear operator theory-based approach (in (b), (c)) and our sFRC analysis (in (d), (e)) are applied on the U-Net and PLS-TV outputs to identify hallucinations. In fig. (f), the first column depicts hallucinated ROIs missed (false negative) by our and Bhadra et al.’s [ 15 ] approaches; the second column depicts an ROI missed by our approach on the PLS-TV output; and the third depicts an ROI missed by Bhadra et al’s approach on the U-Net.

Fig. 10: (a) Red bounding boxes as hallucinations detected by our sFRC analysis on a U-Net output. It was restored from a subsampled MRI data acquired using an acceleration factor of three ( $3$ x). (b) Zoomed view of the purple dotted region from (a). The first two rows in (b) correspond to the ground truth and iFFT ( $1$ x) counterparts of the purple dotted region from (a).

Fig. 11: (a) Ground truth image and its corresponding (b) iFFT, (c) the U-Net, and (d) PLS-TV outputs restored using a subsampled acquisition (i.e., accelerated by a factor three ( $3$ x)). PSNR and SSIM were calculated over signal-only regions. Plot (c) exhibits a high PSNR and SSIM values, and visually it looks as good as plot (a). Still, plot (c) is contaminated with hallucinations, as shown in fig. 10 .

Method	No. of patches	sFRC hallucination
	detected as hallucinations	rate (in $[0,1]$ )
iFFT ( $1$ x)	$2$	$0.010$
iFFT ( $2$ x)	$22$	$0.112$
iFFT ( $3$ x)	$60$	$0.306$
U-Net ( $3$ x)	$43$	$0.219$
PLS-TV ( $3$ x)	$42$	$0.214$

TABLE III: sFRC analysis-based hallucinations relative to different subsampling rates and restoration methods from the four MRI test scans (totaling

196

patches)

Fig. 12: The reference CT image in (a) and its 36 view–based reconstruction from a state-of-the-art (SOTA) model in (c) are processed using sFRC to detect potentially hallucinated local regions, shown as red bounding boxes in (b). Display window is (W: $400$ L: $50$ )

Restoration	Training	Test	No. of Testing	PSNR	SSIM	Hellinger	sFRC
Method	set	set	[Images $\|\|$ Patches]	(std)	(std)	Distance (std)	hallucination rate
CT Super-resolution problem
Bicubic (x $4$ )	-	smooth	$[188\ \|\|\ 12032]$	$33.85\ (0.87)$	$0.904\ (0.012)$	$\mathbf{0.216\ (0.012)}$	$0.368$
SRGAN (x $4$ )	smooth	smooth	$[188\ \|\|\ 12032]$	$36.74\ (2.73)$	$0.945\ (0.010)$	$0.328\ (0.096)$	$\mathbf{0.037}$
SR-WGAN (x $4$ )	smooth	smooth	$[188\ \|\|\ 12032]$	$\mathbf{39.93\ (1.45)}$	$\mathbf{0.970\ (0.005)}$	$0.259\ (0.054)$	$0.042$
Bicubic (x $4$ )	-	sharp	$[188\ \|\|\ 12032]$	$32.86\ (0.76)$	$0.818\ (0.022)$	$0.277\ (0.030)$	$0.390$

TABLE IV: Variation in restoration performance across different evaluation metrics. Higher values indicate better performance for PSNR (

\in[0,\infty]

) and SSIM (

\in[0,1]

), whereas lower values indicate better performance for the Hellinger distance (

\in[0,1]

) and normalized sFRC (

\in[0,1]

Fig. 13: A visual depiction of how our patch-wise (scanned) FRC (sFRC) plot enables estimation of the hallucination rate as a function of the sFRC’s hallucination threshold ( $x_{h_{t}}$ ). Concretely, the minimum $x_{h_{t}}$ value is set based on the sampling rate for a given restoration problem, while the maximum value is determined by considering the resolving power of the imaging modality and the high-frequency bands where confirmed hallucinations have been isolated.

Fig. 14: Hallucination Operating Characteristic (HOC) curve for the SRGAN model - trained on smooth data - evaluated using smooth and sharp test sets.

Appendix figures & tables20 assets

Supplementary material from the paper’s appendix.

Appendix

No. of patients used	Testing	PSNR	SSIM
to train (training kernel)	kernel	(std)	(std)
6 patients (smooth)	smooth	$36.74\ (2.73)$	$0.945\ (0.010)$
8 patients (smooth)	smooth	$37.14\ (2.67)$	$0.943\ (0.001)$
6 patients (smooth)	sharp	$33.72\ (1.95)$	$0.860\ (0.020)$
8 patients (smooth)	sharp	$33.94\ (1.91)$	$0.859\ (0.019)$

Appendix

TABLE I: SSIM and PSNR of SRGAN trained using six and eight patient data and tested using

188

CT images

Fig. S1: An illustration of user-defined hallucinated ROIs (as bounding boxes) and markers (as arrows) annotated in the SRGAN upsampled output in (d) by comparing the image against its normal-resolution reference in (c). The comparison was assisted using the complementary components (e, f) of the two CT images (c, d) retrieved by convolving a bandpass filter ( $0.25\cdot r$ to $0.5\cdot r$ ) in (b) to (c, d), respectively. Prior information that the normal-resolution CT reference and low-resolution test data correspond to contrast-enhanced abdominal CT examinations in the portal-venous phase of enhancement was also utilized when comparing. The normal-resolution CT scan in (c) corresponds to the axial scan 000039.IMA of patient L $067$ (acquired at a $3$ mm slice thickness using a sharp kernel). This normal-resolution CT scan was provided in the LDGC repository [ 1 ] . The display window for CT images in (c, d) is (W: $700$ L: $50$ ).

Fig. S2: An illustration of user-defined hallucinated ROIs (as bounding boxes) and a marker (as an arrow) annotated in the SRGAN upsampled output in (d) by comparing the image against its normal-resolution reference in (c). The comparison was assisted using the complementary components (e, f) of the two CT images (c, d) retrieved by convolving a bandpass filter ( $0.25\cdot r$ to $0.5\cdot r$ ) in (b) to (c, d), respectively. Prior information that the normal-resolution CT reference and low-resolution test data correspond to contrast-enhanced abdominal CT examinations in the portal-venous phase of enhancement was also utilized when comparing. The normal-resolution CT scan in (c) corresponds to the axial scan 000069.IMA of patient L $067$ (acquired at a $3$ mm slice thickness using a sharp kernel). This normal-resolution CT scan was provided in the LDGC repository [ 1 ] . The display window for CT images in (c, d) is (W: $700$ L: $50$ ).

Fig. S3: An illustration of user-defined hallucinated ROIs (as bounding boxes) annotated in the SRGAN upsampled output in (d) by comparing the image against its normal-resolution reference in (c). The comparison was assisted using the complementary components (e, f) of the two CT images (c, d) retrieved by convolving a bandpass filter ( $0.25\cdot r$ to $0.5\cdot r$ ) in (b) to (c, d), respectively. Prior information that the normal-resolution CT reference and low-resolution test data correspond to contrast-enhanced abdominal CT examinations in the portal-venous phase of enhancement was also utilized when comparing. The normal-resolution CT scan in (c) corresponds to the axial scan 000149.IMA of patient L $067$ (acquired at a $3$ mm slice thickness using a sharp kernel). This normal-resolution CT scan was provided in the LDGC repository [ 1 ] . The display window for CT images in (c, d) is (W: $700$ L: $50$ ).

Fig. S4: An illustration of user-defined hallucinated ROIs (as bounding boxes) and markers (as arrows) annotated in the SRGAN upsampled output in (d) by comparing the image against its normal-resolution reference in (c). The comparison was assisted using the complementary components (e, f) of the two CT images (c, d) retrieved by convolving a bandpass filter ( $0.25\cdot r$ to $0.5\cdot r$ ) in (b) to (c, d), respectively. Prior information that the normal-resolution CT reference and low-resolution test data correspond to contrast-enhanced abdominal CT examinations in the portal-venous phase of enhancement was also utilized when comparing. The normal-resolution CT scan in (c) corresponds to the axial scan 000189.IMA of patient L $067$ (acquired at a $3$ mm slice thickness using a sharp kernel). This normal-resolution CT scan was provided in the LDGC repository [ 1 ] . The display window for CT images in (c, d) is (W: $700$ L: $50$ ).

Fig. S5: An illustration of user-defined hallucinated ROIs (as bounding boxes) and markers (as arrows) annotated in the SRGAN upsampled output in (d) by comparing the image against its normal-resolution reference in (c). The comparison was assisted using the complementary components (e, f) of the two CT images (c, d) retrieved by convolving a bandpass filter ( $0.25\cdot r$ to $0.5\cdot r$ ) in (b) to (c, d), respectively. Prior information that the normal-resolution CT reference and low-resolution test data correspond to contrast-enhanced abdominal CT examinations in the portal-venous phase of enhancement was also utilized when comparing. The normal-resolution CT scan in (c) corresponds to the axial scan 000217.IMA of patient L $067$ (acquired at a $3$ mm slice thickness using a sharp kernel). This normal-resolution CT scan was provided in the LDGC repository [ 1 ] . The display window for CT images in (c, d) is (W: $700$ L: $50$ ).

Fig. S6: Illustration of hallucination threshold ( $x_{h_{t}}$ ) getting stricter (i.e., label more patches to be hallucinated) as $x_{h_{t}}$ approaches Nyquist frequency ( $r$ ). For instance, $x_{h_{t}}$ in plot (a) is very relaxed, as sFRC will only consider very low-frequency bands (i.e., blurred image components) when performing correlation analysis between Deep learning and reference image pairs. In contrast, $x_{h_{t}}$ in plot (d) is strict, as sFRC considers image components across low- to high-frequency bands when performing the correlation analysis.

Fig. S7: A visual depiction of how the FRC threshold ( $Y$ ) targets different spatial frequencies in sFRC analysis between deep learning and reference images, with Y set in the range $[0,1]$ .

Fig. S8: Hallucinations identified in the red bounding boxes from our sFRC analysis in the SRGAN upsampled outputs. The five SRGAN upsampled CT images correspond to images used to tune the sFRC parameters. The sFRC parameters comprised of $64\times 64$ as the patch size, $0.5$ as the FRC threshold, and $\mathbf{0.25}$ as the hallucination threshold ( $x_{h_{t}}$ ). At $0.25$ $x_{h_{t}}$ , almost all the hallucinated ROIs - labeled in figs. S1 through S5 - were missed by the sFRC. The display window for all images is (W: $700$ L: $50$ ).

Fig. S9: Hallucinations identified in the red bounding boxes from our sFRC analysis in the SRGAN upsampled outputs. The five SRGAN upsampled CT images correspond to images used to tune the sFRC parameters. The sFRC parameters comprised of $64\times 64$ as the patch size, $0.5$ as the FRC threshold, and $\mathbf{0.35}$ as the hallucination threshold ( $x_{h_{t}}$ ). At $0.35$ $x_{h_{t}}$ , almost all the hallucinated ROIs - labeled in figs. S1 through S5 - were captured by the sFRC. The display window for all images is (W: $700$ L: $50$ ).

Fig. S10: Hallucinations identified in the red bounding boxes from our sFRC analysis in the SRGAN upsampled outputs. The five SRGAN upsampled CT images correspond to images used to tune the sFRC parameters. The sFRC parameters comprised of $64\times 64$ as the patch size, $0.5$ as the FRC threshold, and $\mathbf{0.34}$ as the hallucination threshold ( $x_{h_{t}}$ ). At $0.34$ $x_{h_{t}}$ , most of the hallucinated ROIs - labeled in figs. S1 through S5 - were captured by the sFRC. The display window for all images is (W: $700$ L: $50$ ).

Fig. S11: Hallucinations identified in the red bounding boxes from our sFRC analysis in the SRGAN upsampled outputs. The five SRGAN upsampled CT images correspond to images used to tune the sFRC parameters. The sFRC parameters comprised of $64\times 64$ as the patch size, $0.5$ as the FRC threshold, and $\mathbf{0.33}$ as the hallucination threshold ( $x_{h_{t}}$ ). At $0.33$ $x_{h_{t}}$ , the sFRC was still able to detect all types of hallucinations (related to plaques, perturbations, and indentations) that were annotated in figs. S1 through S5 . The display window for all images is (W: $700$ L: $50$ ).

Fig. S12: An illustration of imaging errors spanning from low- to high-frequency components of restored images plagued with Missing Wedge and Distortion artifacts. The second, third, and fourth rows indicate plots corresponding to reference, missing wedge, and distorted images. Imaging filters corresponding to each component of an image (in the first column) are depicted in the second through sixth columns in the top row. Zoomed views of the ROI corresponding to the yellow bounding box are provided for all the image components. Display windows corresponding to each column are provided in the bottom row.

Fig. S13: An illustration of a reference CT image in (a) that was used to simulate different types of conventional CT artifacts. (b-e) plots outcomes from sFRC, as a red-bounded box, on the CT images with artifacts. (b) Missing wedge is due to a limited number of projection angles ( $[30^{\circ},150^{\circ})$ at a $2^{\circ}$ stepsize). (c) Distortion is due to a mismatch in the forward projection and backprojection phases by $10^{\circ}$ . (d) Blurring artifact is due to interpolation-based pixel upsampling and downsampling by a factor of $4$ . (e) Noise is due to CT physics-based noise insertion for an acquisition corresponding to a $5\%$ dose of the reference CT image in (a). The display window for all the CT images is (W: $780$ L: $260$ ).

Fig. S14: An illustration of linear system–based improvements achieved by progressively decreasing the missing-wedge gap. These linear system–based improvements also translate into a human observer’s increased ability to discern dark lesions—shown using yellow circular regions in plots (b–d)—as well as improvements in full-image PSNR and SSIM values in plots (b–d), and Fourier Ring Correlation (FRC) curves in plot (e). The display window for all the CT images in (a-d) is (W: $780$ L: $260$ ).

Fig. S15: An illustration of the lack of efficacy of the FRC, PSNR, and SSIM determined using the entire image to evaluate SRGAN, a post-processing non-linear method, accurately. The FRC curves and PSNR and SSIM values indicate apparent improvement with the use of SRGAN for a CT super-resolution problem. However, the second column reveals hallucinations in the SRGAN-based outputs, including the appearance of two bowel loops instead of a single contiguous loop within the yellow-boxed region, as well as the addition of a plaque-like structure in the green-boxed region. The display window for all the CT images in the first two columns is (W: $700$ L: $50$ ).

Fig. S16: The purple ROIs indicate hallucinated patches in the SRWGAN outputs that were initially missed by our sFRC analysis in the leftmost scan (000077.IMA). The regions corresponding to these missed hallucinations are subsequently flagged in the center (000080.IMA) and rightmost (000082.IMA) scans within the abdominal CT volume of patient L067 [ 1 ] . This plot is similar to that obtained from the SRGAN model shown in fig. 6 of the main paper. Display window is (W: $400$ L: $50$ ).

Fig. S17: An explicit illustration of the decay in performance of the SRWGAN super-resolution model—trained using smooth data—when analyzed using our sFRC approach against (b) sharp- and (d) smooth-kernel–based normal-resolution patches. The smoothly trained SRWGAN generates hallucinated indentations in (c), which are detected by our sFRC analysis. In contrast, the same SRWGAN upsamples (x $4$ ) smooth test data more faithfully in (e) and is not flagged by sFRC. The patches in (b–e) correspond to an ROI from an abdominal CT scan shown in (a). This plot is similar to that obtained from the SRGAN model shown in fig. 7 of the main paper. Display window is (W: $400$ L: $50$ ).

Fig. S18: Red bounding boxes indicate hallucinations identified by our sFRC analysis in an SRWGAN output. A plaque-like hallucination is incorrectly added along a bordering region (indicated by arrow (1)), and an ROI with fatty attenuation is underfitted as air (indicated by arrow (2)) in the SRWGAN-based output compared with its normal-resolution FBP counterpart. Radiologists are trained to identify even small amounts of air; therefore, such air-based hallucinations can mislead clinical decision-making. This plot is similar to that obtained from the SRGAN model shown in fig. 8 of the main paper. However, compared with fig. 8, sFRC flags two additional ROIs in the SRWGAN results here, as shown in the third column. Red arrow (3) indicates a cut-like hallucination, while marking (4) appears to be a false positive. Display window is (W: $700$ L: $50$ ).

Fig. S19: Sparse view sinogram data is shown in (a). The corresponding FBP reconstruction is shown in (b), and the PAIL reconstruction is shown in (f). The fully sampled FBP reconstruction is provided in (e). sFRC is performed using the fully sampled FBP and PAIL reconstructions in (e) and (f), with $x_{h_{t}}=0.5$ and a patch size of $48\times 48$ . The subsequent hallucinations, highlighted by red bounding boxes in (d), were confirmed and labeled in coordination with a medical officer and are shown in (f). Display window in (b-f) is (W: $400$ L: $50$ ).

Hallucination Detection

Hallucination detection identifies unsupported, fabricated, or evidence-inconsistent content in language and multimodal model outputs.

See all 78 papers

Aug 6, 2026

Does FLAIR super-resolution erase or hallucinate small white-matter lesions?

White matter hyperintensities (WMH), bright regions on Fluid-attenuated Inversion Recovery (FLAIR) scans are associated with cerebrovascular pathology and neurodegeneration. FLAIR is usually acquired with thick slices in clinical settings, giving it poor through-plane resolution. Super-resolution (SR) is a widely used method for recovering an isotropic volume from an anisotropic scan. Yet whether applying it prior to WMH segmentation preserves lesion content remains unknown: a model may erase small real lesions or hallucinate absent ones. We used 1-mm isotropic high-resolution (HR) FLAIR scans from 29 individuals in the ADNI cohort, each manually segmented for WMH by an expert. Then, we degraded each to simulated 3 and 5 mm through-plane acquisitions. Multi-contrast implicit neural representation (INR), a single-contrast self-supervised model (ECLARE), and cubic interpolation were used to upsample them onto the HR grid. WMH segmentation from a simulated thick slice and the original HR FLAIR set the floor and ceiling, respectively, for the per-lesion analysis. Of four WMH segmentation methods (WMH-SynthSeg, segcsvd, MARS-WMH, TrUE-Net), we ran the analysis under the most sensitive one to small lesions on HR (MARS-WMH) with the evaluation metrics of detection sensitivity, erasure rate (HR-detected lesions lost after reconstruction), and hallucination rate (predicted components absent from both the manual and HR segmentation). The dominant effect of SR was erasure of small real lesions, not hallucination, and it increased with slice thickness, though every reconstruction still improved lesion detection over the raw thick slice. ECLARE recovered small lesion signal best at both thicknesses, while the INR was no better than cubic interpolation.

Zahra Khodakarami, Yue Li, Pulkit Khandelwal +5

University of Pennsylvania, Philadelphia, PA, USA

Medical Image Super-Resolution Magnetic Resonance Imaging Medical Image Segmentation

Ask PDF

May 13, 2026

On Hallucinations in Inverse Problems: Fundamental Limits and Provable Assessment Methods

Artificial intelligence (AI) has transformed imaging inverse problems, from medical diagnostics to Earth observation. Yet deep neural networks can produce hallucinations, realistic-looking but incorrect details, undermining their reliability, especially when ground truth data is unavailable. We develop a theoretical framework showing that such hallucinations are not merely artifacts of particular models, but can arise from the ill-posed nature of the inverse problem itself. We derive necessary and sufficient conditions for hallucinations, together with computable bounds on their magnitude that depend only on the forward model. Building on this theory, we introduce algorithms to: (1) estimate the minimum hallucination magnitude achievable by any reconstruction model for a given input; (2) assess the faithfulness of reconstructed details by a given reconstruction model. Experiments across three imaging tasks demonstrate that our approach applies broadly, including to modern generative models, and provides a principled way to quantify and evaluate AI hallucinations.

David Iagaru, Nina M. Gottschling, Anders C. Hansen +1

CMAP, Ecole polytechnique, Institut Polytechnique de Paris, 91120 Palaiseau, France. · German Aerospace Center (DLR), Remote Sensing Technology Institute, Wessling, Germany. · Computing and Computational Sciences, Oak Ridge National Laboratory, Oak Ridge, Tennessee. +2

Image Reconstruction Inverse Problems

Ask PDF

Feb 23, 2026

A Green Learning Approach to LDCT Image Restoration

This work proposes a green learning (GL) approach to restore medical images. Without loss of generality, we use low-dose computed tomography (LDCT) images as examples. LDCT images are susceptible to noise and artifacts, where the imaging process introduces distortion. LDCT image restoration is an important preprocessing step for further medical analysis. Deep learning (DL) methods have been developed to solve this problem. We examine an alternative solution using the Green Learning (GL) methodology. The new restoration method is characterized by mathematical transparency, computational and memory efficiency, and high performance. Experiments show that our GL method offers state-of-the-art restoration performance at a smaller model size and with lower inference complexity.

Wei Wang, Yixing Wu, C. -C. Jay Kuo

University of Southern California, Los Angeles, California, USA

Image Restoration LDCT Denoising

Ask PDF

Abstract

Hallucination Detection

Explore similar work

Does FLAIR super-resolution erase or hallucinate small white-matter lesions?

On Hallucinations in Inverse Problems: Fundamental Limits and Provable Assessment Methods

A Green Learning Approach to LDCT Image Restoration