VLM- and LLM-Driven Multi-Agent System for PET Image Denoising

Boxiao Yu; Savas Ozdemir; Yang Xing; Fumio Hashimoto; Jiong Wu; Yizhou Chen; Axel Rominger; Ruogu Fang

eess.IVAug 13, 2026

VLM- and LLM-Driven Multi-Agent System for PET Image Denoising

Authors: Boxiao Yu, Savas Ozdemir, Yang Xing, Fumio Hashimoto, Jiong Wu, Yizhou Chen, Axel Rominger, Ruogu Fang,

Organizations: J. Crayton Pruitt Family Department of Biomedical Engineering, University of Florida, Gainesville, FL, 32611, USA · Department of Radiology, University of Florida, Jacksonville, FL, 32209, USA · Department of Nuclear Medicine, University of Bern, Bern, Switzerland · Department of Imaging Physics, University of Texas MD Anderson Cancer Center, Houston, TX, 77230, USA

Abstract

Positron emission tomography (PET) imaging suffers from limited spatial resolution and low signal-to-noise ratio, which can compromise quantitative accuracy and lesion detectability. Deep learning-based denoising methods have demonstrated strong potential for improving PET image quality. However, their practical deployment in real-world settings remains challenging, often requiring multiple specialized models and expert interventions, such as identifying motion-induced misregistration artifacts, estimating noise levels to select an appropriate denoiser, and performing lesion-focused quantitative assessment after denoising. Recent advances in vision-language models (VLMs) for image quality understanding and large language models (LLMs) for contextual reasoning provide new opportunities for automated, decision-driven workflows. Inspired by expert workflows for PET image quality enhancement, we propose an VLM- and LLM-driven multi-agent PET denoising framework that dynamically assesses image quality and lesion status, autonomously selects optimal denoising models and parameters, and enables closed-loop feedback with rollback mechanisms. Experiments were conducted on Siemens Biograph Vision Quadra PET/CT data with 1/20 and 1/50 low-dose settings. Individual module evaluations demonstrated the reliability of the agentic components, while the complete framework achieved higher PSNR and SSIM than UNet, GAN, and DDPM baselines at both dose levels. These preliminary results demonstrate the feasibility of using a closed-loop multi-agent framework to adapt PET denoising strategies to different image conditions.

Figures & tables

Fig. 1: Overview of the proposed closed-loop multi-agent framework for low-dose PET image denoising.

Fig. 2: Detailed architecture of the proposed multi-agent PET denoising system.

Fig. 3: Construction of the PET denoising domain knowledge base.

Fig. 4: Evaluation of dose-aware perception and lesion-aware segmentation. (A) Violin plots of the dose-level confidence predicted by the fine-tuned BiomedCLIP. A 50% threshold is used to distinguish the $1/20$ - and $1/50$ -dose predictions. Representative subjects with relatively high and low confidence scores within each dose group are illustrated, with corresponding PET images and acquisition-related information shown on the right. (B) Lesion segmentation performance of the lesion-aware agent on the AutoPET-IV FDG test subsets (LMU and UKT), compared with the challenge baseline, reported with foreground DSC, FNV, and FPV.

Fig. 5: Sagittal views of a Siemens Biograph Vision Quadra 18 F-FDG test data with $1/20$ and $1/50$ low-dose PET images denoised using different methods, alongside the corresponding normal-dose PET image.

Fig. 6: Quantitative evaluation of overall PET denoising performance (PSNR and SSIM) on 20 test cases for each of the $1/20$ - and $1/50$ low-dose settings. Statistical significance is indicated by * ( $p\textless 0.05$ ) and *** ( $p\textless 0.001$ ).

Fig. 7: Zoomed-in lesion region and L 2 norm distance images between the predicted and GT images of a Siemens Biograph Vision Quadra 18 F-FDG test data, alongside the corresponding normal-dose PET image.

Image Denoising

Image denoising removes unwanted noise while preserving the edges, textures, and colors needed for faithful images.

See all 157 papers

U-TTT: Towards Generalizable PET Image Denoising via Test-Time Training
Jun 9, 2026Zhiwen Yang, Jiayin Li, Hao Lu +3OOD Generalization PET/CT Imaging
School of Biological Science and Medical Engineering, Beihang University, Beijing 100191, China · Department of Biomedical Engineering, Tsinghua University, Beijing 100084, China · School of Aerospace Engineering, Tsinghua University, Beijing 100084, China
UniPET: a universal network for high-quality PET image denoising across varied dose reduction factors
Jun 9, 2026Zhiwen Yang, Yang Zhou, Haowei Chen +4PET/CT Imaging Image Denoising
School of Biological Science and Medical Engineering, State Key Laboratory of Software Development Environment, Key Laboratory of Biomechanics, Mechanobiology of Ministry of Education and Beijing Advanced Innovation Centre for Biomedical Engineering, Beihang University, Beijing 100191, China · Department of Biomedical Engineering, Tsinghua University, Beijing 100084, China · Department of Gynecology Oncology, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing 100021, China +2
Rethinking Cross-Dose PET Denoising: Mitigating Averaging Effects via Residual Noise Learning
Apr 18, 2026Yichao Liu, Zongru Shao, Yueyang Teng +1Residual Learning PET/CT Imaging
IWR, Heidelberg University, 69120, Heidelberg, Germany · Silicon Austria Labs, 4040, Linz, Austria · College of Medicine and Biological Information Engineering, Key Laboratory of Intelligent Computing in Medical Image, Ministry of Education, Northeastern University, 110169, Shenyang, China +1

Jun 9, 2026J/K move · Enter open · S save

U-TTT: Towards Generalizable PET Image Denoising via Test-Time Training

Zhiwen Yang, Jiayin Li, Hao Lu, Hui Zhang+2

School of Biological Science and Medical Engineering, Beihang University, Beijing 100191, China · Department of Biomedical Engineering, Tsinghua University, Beijing 100084, China · School of Aerospace Engineering, Tsinghua University, Beijing 100084, China

OOD Generalization PET/CT Imaging Test-Time Training Image Denoising

Open paper AskPDF arXiv

Existing deep learning models for Positron Emission Tomography (PET) image denoising often suffer from severe performance degradation under distribution shifts, fundamentally restricting their robust clinical deployment. This lack of generalization stems from the conventional paradigm of fixed-parameter models that cannot adapt to variations in test data (e.g., dose levels or scanner types) after training. To overcome this limitation and achieve robust generalization, we introduce U-TTT, a novel U-shaped model that integrates Test-Time Training (TTT) layers to dynamically adjust model parameters during inference through self-supervision, thereby adapting to the specific characteristics of each test instance. Furthermore, to comprehensively capture the complex degradations of 3D PET data, U-TTT features a dual-domain adaptation mechanism comprising a Spatial Test-Time Training (S-TTT) layer and a Frequency Test-Time Training (F-TTT) layer. The S-TTT layer captures and corrects spatial structural degradations, while the F-TTT layer suppresses global noise spectra and restores delicate high-frequency details. Extensive experiments demonstrate that U-TTT achieves state-of-the-art PET denoising performance and exhibits superior generalization under challenging distribution shifts, including both unseen dose levels and unseen scanners. Our code will be available at https://github.com/Yaziwel/U-TTT.

Abstract

Image Denoising

Explore similar work

U-TTT: Towards Generalizable PET Image Denoising via Test-Time Training

UniPET: a universal network for high-quality PET image denoising across varied dose reduction factors

Rethinking Cross-Dose PET Denoising: Mitigating Averaging Effects via Residual Noise Learning