VLM- and LLM-Driven Multi-Agent System for PET Image Denoising
Authors: Boxiao Yu, Savas Ozdemir, Yang Xing, Fumio Hashimoto, Jiong Wu, Yizhou Chen, Axel Rominger, Ruogu Fang, Kuangyu Shi, Tinsu Pan, Kuang Gong
Organizations: J. Crayton Pruitt Family Department of Biomedical Engineering, University of Florida, Gainesville, FL, 32611, USA · Department of Radiology, University of Florida, Jacksonville, FL, 32209, USA · Department of Nuclear Medicine, University of Bern, Bern, Switzerland · Department of Imaging Physics, University of Texas MD Anderson Cancer Center, Houston, TX, 77230, USA
Positron emission tomography (PET) imaging suffers from limited spatial resolution and low signal-to-noise ratio, which can compromise quantitative accuracy and lesion detectability. Deep learning-based denoising methods have demonstrated strong potential for improving PET image quality. However, their practical deployment in real-world settings remains challenging, often requiring multiple specialized models and expert interventions, such as identifying motion-induced misregistration artifacts, estimating noise levels to select an appropriate denoiser, and performing lesion-focused quantitative assessment after denoising. Recent advances in vision-language models (VLMs) for image quality understanding and large language models (LLMs) for contextual reasoning provide new opportunities for automated, decision-driven workflows. Inspired by expert workflows for PET image quality enhancement, we propose an VLM- and LLM-driven multi-agent PET denoising framework that dynamically assesses image quality and lesion status, autonomously selects optimal denoising models and parameters, and enables closed-loop feedback with rollback mechanisms. Experiments were conducted on Siemens Biograph Vision Quadra PET/CT data with 1/20 and 1/50 low-dose settings. Individual module evaluations demonstrated the reliability of the agentic components, while the complete framework achieved higher PSNR and SSIM than UNet, GAN, and DDPM baselines at both dose levels. These preliminary results demonstrate the feasibility of using a closed-loop multi-agent framework to adapt PET denoising strategies to different image conditions.
Figures & tables
Fig. 1: Overview of the proposed closed-loop multi-agent framework for low-dose PET image denoising.
Fig. 2: Detailed architecture of the proposed multi-agent PET denoising system.
Fig. 3: Construction of the PET denoising domain knowledge base.
Fig. 4: Evaluation of dose-aware perception and lesion-aware segmentation. (A) Violin plots of the dose-level confidence predicted by the fine-tuned BiomedCLIP. A 50% threshold is used to distinguish the 1/20 - and 1/50 -dose predictions. Representative subjects with relatively high and low confidence scores within each dose group are illustrated, with corresponding PET images and acquisition-related information shown on the right. (B) Lesion segmentation performance of the lesion-aware agent on the AutoPET-IV FDG test subsets (LMU and UKT), compared with the challenge baseline, reported with foreground DSC, FNV, and FPV.
Fig. 5: Sagittal views of a Siemens Biograph Vision Quadra 18 F-FDG test data with 1/20 and 1/50 low-dose PET images denoised using different methods, alongside the corresponding normal-dose PET image.
Fig. 6: Quantitative evaluation of overall PET denoising performance (PSNR and SSIM) on 20 test cases for each of the 1/20 - and 1/50 low-dose settings. Statistical significance is indicated by * ( p\textless0.05 ) and *** ( p\textless0.001 ).
Fig. 7: Zoomed-in lesion region and L 2 norm distance images between the predicted and GT images of a Siemens Biograph Vision Quadra 18 F-FDG test data, alongside the corresponding normal-dose PET image.
Existing deep learning models for Positron Emission Tomography (PET) image denoising often suffer from severe performance degradation under distribution shifts, fundamentally restricting their robust clinical deployment. This lack of generalization stems from the conventional paradigm of fixed-parameter models that cannot adapt to variations in test data (e.g., dose levels or scanner types) after training. To overcome this limitation and achieve robust generalization, we introduce U-TTT, a novel U-shaped model that integrates Test-Time Training (TTT) layers to dynamically adjust model parameters during inference through self-supervision, thereby adapting to the specific characteristics of each test instance. Furthermore, to comprehensively capture the complex degradations of 3D PET data, U-TTT features a dual-domain adaptation mechanism comprising a Spatial Test-Time Training (S-TTT) layer and a Frequency Test-Time Training (F-TTT) layer. The S-TTT layer captures and corrects spatial structural degradations, while the F-TTT layer suppresses global noise spectra and restores delicate high-frequency details. Extensive experiments demonstrate that U-TTT achieves state-of-the-art PET denoising performance and exhibits superior generalization under challenging distribution shifts, including both unseen dose levels and unseen scanners. Our code will be available at https://github.com/Yaziwel/U-TTT.
Zhiwen Yang, Jiayin Li, Hao Lu +3
School of Biological Science and Medical Engineering, Beihang University, Beijing 100191, China · Department of Biomedical Engineering, Tsinghua University, Beijing 100084, China · School of Aerospace Engineering, Tsinghua University, Beijing 100084, China
Most existing deep learning-based PET image denoising methods assume a fixed and known dose reduction factor (DRF) for low-dose PET images. However, these methods encounter significant performance degradation when the DRF varies beyond the assumed one in practical applications. To address the challenge posed by varied DRFs, several preliminary studies focus on the task of universal PET image denoising, aiming to train a universal model over low-dose data across DRFs. Nonetheless, these vanilla universal models often struggle with misaligned styles present in different DRF data, leading to the style elimination issue with a significant over-smoothing effect. To deal with this issue, we innovatively introduce domain generalization to PET image denoising and propose a universal PET image denoising network (UniPET) to achieve high-quality PET image denoising across diverse DRFs. UniPET comprises two primary innovations: a style alignment network (SAN) and a region-aware learning strategy (RALS). Specifically, SAN utilizes style alignment techniques derived from domain generalization to align and recover styles across different DRFs, ensuring the model's generalizability across various DRFs while effectively preserving styles. Furthermore, to enhance style recovery, RALS distinguishes between flat and stylized regions, exclusively conducting adversarial learning on the latter, thereby more effectively guiding the model's focus towards learning stylized regions. It is demonstrated that our proposed UniPET can adaptively recover different DRF styles and achieve high-quality PET image denoising across DRFs. Comprehensive experiments show that UniPET exhibits comparable performance to individual DRF-specific models at specific DRFs and realizes state-of-the-art performance in universal PET image denoising quantitatively, perceptually, and clinically.
Zhiwen Yang, Yang Zhou, Haowei Chen +4
School of Biological Science and Medical Engineering, State Key Laboratory of Software Development Environment, Key Laboratory of Biomechanics, Mechanobiology of Ministry of Education and Beijing Advanced Innovation Centre for Biomedical Engineering, Beihang University, Beijing 100191, China · Department of Biomedical Engineering, Tsinghua University, Beijing 100084, China · Department of Gynecology Oncology, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing 100021, China +2
Cross-dose denoising for low-dose positron emission tomography (LDPET) has been proposed to address the limited generalization of models trained at a single noise level. However, neural networks trained on a specific dose level often fail to generalize to other dose conditions due to variations in noise magnitude and statistical properties. Conventional "one-size-for-all" models attempt to mitigate this variability but tend to learn averaged representations across noise levels, resulting in degraded performance. In this work, we analyze this limitation and show that standard training formulations implicitly optimize an expectation over heterogeneous noise distributions, causing the network to learn an averaged denoising mapping that cannot accurately model dose-specific noise characteristics. We propose a unified residual noise learning framework that estimates noise directly from low-dose PET images rather than predicting full-dose images. Experiments on large-scale multi-dose PET datasets from two medical centers demonstrate that the proposed method outperforms the "one-size-for-all" model, individual dose-specific U-Net models, and dose-conditioned approaches, achieving improved denoising performance. These results indicate that residual noise learning effectively mitigates the averaging effect and enhances generalization for cross-dose PET denoising.
Yichao Liu, Zongru Shao, Yueyang Teng +1
IWR, Heidelberg University, 69120, Heidelberg, Germany · Silicon Austria Labs, 4040, Linz, Austria · College of Medicine and Biological Information Engineering, Key Laboratory of Intelligent Computing in Medical Image, Ministry of Education, Northeastern University, 110169, Shenyang, China +1