cs.CYMar 25, 2026

Secure On-Premise Deployment of Open-Weights Large Language Models in Radiology: An Isolation-First Architecture with Prospective Pilot Evaluation

Authors: Sebastian Nowak, Jann-Frederick Laß, Narine Mesropyan, Babak Salam, Nico Piel, Mohammed Bahaaeldin, Wolfgang Block, Alois Martin Sprinkart, 

Organizations: Department of Diagnostic and Interventional Radiology, University Hospital Bonn · Department of Research-IT, University Hospital Bonn

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Purpose: To design, implement, evaluate, and report on the regulatory requirements of a self-hosted LLM infrastructure for radiology adhering to the principle of least privilege, emphasizing technical feasibility, network isolation, and clinical utility. Materials and Methods: The isolation-first, containerized LLM inference stack relies on strict network segmentation, host-enforced egress filtering, and active isolation monitoring preventing unauthorized external connectivity. An accompanying deployment package provides automated isolation and hardening tests. The system served the open-weights DeepSeek-R1 model via vLLM. In a one-week pilot phase, 22 residents and radiologists were free to use 10 predefined prompt-templates whenever they considered them useful in daily work. Afterward, they rated clinical utility and system stability on an 0-10 Likert scale and reported observed critical errors in model output. Results: The applied institutional governance pathway achieved approval from clinic management, compliance, data protection and information security officers for processing unanonymized PHI. The system was rated stable and user friendly during the pilot. Source text-anchored tasks, such as report corrections or simplifications, and radiology guideline recommendations received the highest utility ratings, whereas open-ended conclusion generation based on findings resulted in the highest frequency of critical errors, such as clinically relevant hallucinations or omissions. Conclusion: The proposed isolation-first on-premise architecture enabled overcoming regulatory borders, showed promising clinical utility in text-anchored tasks and is the current base to serve open-weights LLMs as an official service of a German University Hospital with over 10,000 employees. The deployment package were made publicly available (https://github.com/ukbonn/ukb-gpt).

Figures & tables

Threat ScenarioPrimary Mitigation
Malicious Model Weights: Weights or model-loading code contain hidden logic with arbitrary code execution intended to exfiltrate data [20].Internal network isolation, Host-enforced egress filtering
Supply Chain Attack: Compromised OSS dependencies or Docker images contain malware designed to compromise sensitive data [10].Host-enforced firewalling, Active isolation monitoring
Auth-Bridge Compromise: An attacker attempts to tunnel data through the LDAP airlock.Dedicated LDAP egress proxy, Strict IP pinning, Upstream TLS certificate verification
Web Application Compromise: XSS, CSRF, or RCE enables command execution in the chat frontend [21]; attacker then attempts privilege escalation or host compromise [22].Reverse-proxy hardening, Container capability/privilege drop, Host-enforced egress restrictions
Credential Theft: Sensitive TLS keys or LDAP credentials are recovered from persistent storage or writable runtime paths.In-memory secret injection, Selective read-only mounts, Restricted writable paths and tmpfs
Table 1: Selected Threat Scenarios and Corresponding Primary Technical Mitigations.
TaskDescription
Report: Correction & ImprovementSuggests improvements for correctness, clarity, completeness, and professional tone without changing the intended meaning of the report.
Report: Conclusion GenerationDrafts a suggested conclusion section based on the provided indication and findings text.
Report: Follow-up ComparisonCreates a follow-up report based on a previous report and notes on changes between examinations.
Report: CT Report StructuringConverts a free-text CT chest/abdomen report into a structured format aligned with departmental templates.
Report: ICD-10 CodingSuggests ICD-10 codes based on the given report text, using a list of frequently used codes.
Report: SimplificationProduces a layperson-friendly version of the report text for patients.
Table 2: Overview of the ten standardized, template-based workflows provided in the chat interface during the one-week pilot study.
Perceived Utility
TaskPotentialQualityEase of EditingTime Savings
Report: Correction & Improvement8.0 [7.2-9.0]7.5 [7.0-8.8]8.0 [5.5-8.8]2.5 [1.2-5.0]
Report: Conclusion Generation7.0 [5.0-8.0]6.0 [4.8-7.0]5.0 [4.0-7.2]3.0 [2.0-5.0]
Report: Follow-up Generation7.5 [5.2-9.0]8.0 [6.5-8.0]7.0 [5.0-8.0]5.5 [3.5-7.2]
Report: CT Report Structuring5.5 [5.0-8.0]8.0 [8.0-9.0]8.0 [7.0-9.0]3.5 [2.0-5.8]
Report: ICD-10 Coding9.0 [7.2-10]8.0 [7.0-9.0]9.0 [9.0-10]3.5 [2.2-5.0]
Table 3: Clinical Utility and Critical Error Profiles by Task. Utility metrics are presented as Median [Interquartile Range] on a 0-10 scale. Critical Errors are presented as absolute counts and percentages (n (%)). A "Critical Error" denotes an output containing hallucinations, omissions, or incorrect medical advice requiring manual intervention.
Figure 1: System architecture diagram : The core evaluated stack comprised browser access via the hospital intranet or VPN, an Nginx ingress proxy, an OpenWebUI frontend, and a vLLM inference backend. Security was enforced through layers of redundant isolation. Note: The custom clinical application block illustrates potential extensions of the same isolated backbone. These applications were not part of the present radiologist pilot.
Figure 2: Perceived Clinical Utility by Task : Bar chart showing mean ratings for "Perceived Potential," "Output Quality," "Ease of Editing," and "Time Savings" across ten radiological tasks. Tasks are grouped by category: Report Transformation, Structuring, Coding, and General Utility.
Figure 3: Frequency of Critical Errors by Task : Stacked bar chart showing the system’s safety profile. We defined a "Critical Error" as an output containing hallucinations, omissions, or incorrect medical advice requiring intervention. "Report: Conclusion Generation" was the only task with multiple critical errors per response, which indicates that automated summarization carries higher risks than linguistic transformation.
Figure 4: Radiologist Ratings of Technical Performance : Histograms show the distribution of user ratings (0-10 scale) for system stability, speed, and usability (N=22). Stability ratings were the most consistent; most responses clustered between 9 and 10. Speed ratings varied more (range 5-9), likely due to differences in prompt complexity and output length.
More in this topic

LLM Serving

Large language model serving studies how to deliver model responses with low latency, high throughput, and efficient use of compute and memory.

See all 207 papers

Explore similar work

CardsList
May 1, 2026

RadLite: Multi-Task LoRA Fine-Tuning of Small Language Models for CPU-Deployable Radiology AI

Large language models (LLMs) show promise in radiology but their deployment is limited by computational requirements that preclude use in resource-constrained clinical environments. We investigate whether small language models (SLMs) of 3-4 billion parameters can achieve strong multi-task radiology performance through LoRA fine-tuning, enabling deployment on consumer-grade CPUs. We train Qwen2.5-3B-Instruct and Qwen3-4B on 162K samples spanning 9 radiology tasks - RADS classification across 10 systems, impression generation, temporal comparison, radiology NLI, NER, abnormality detection, N/M staging, and radiology Q&A - compiled from 12 public datasets. Both models are evaluated on up to 500 held-out test samples per task with standardized metrics. Our key findings are: (1) LoRA fine-tuning dramatically improves performance over zero-shot baselines (RADS accuracy +53%, NLI +60%, N-staging +89%); (2) the two models exhibit complementary strengths - Qwen2.5 excels at structured generation tasks while Qwen3 dominates extractive tasks; (3) a task-outed oracle ensemble combining both models achieves the best performance across all tasks; (4) few-shot prompting with fine-tuned models hurts performance, demonstrating that LoRA adaptation is more effective than in-context learning for specialized domains; and (5) models can be quantized to GGUF format (~1.8-2.4GB) for CPU deployment at 4-8 tokens/second on consumer hardware. Our work demonstrates that small, efficiently fine-tuned models - which we collectively call RadLite - can serve as practical multi-task radiology AI assistants deployable entirely on consumer hardware without GPU requirements. Code and models are available at https://github.com/RadioX-Labs/RadLite

Pankaj Gupta, Kartik Bose

Postgraduate Institute of Medical Education and Research, Chandigarh, India ∗Corresponding author: Additional Professor, Department of Radiodiagnosis and Imaging, Postgraduate Institute of Medical Education and Research, Chandigarh, India · Postgraduate Institute of Medical Education and Research, Chandigarh, India

LLM Fine-TuningSmall Language ModelsClinical NLPMedical LLMs
AskPDF
Oct 3, 2026

Autonomous Structuring of Radiology Reports Across Modalities at Archive Scale Using an Open-Weight Large Language Model

Purpose: To develop and evaluate an open-weight large language model (LLM) pipeline that converts an entire archive of free-text radiology reports into structured reports without human oversight. Materials and Methods: In this retrospective study, a pipeline with 150 hierarchically organized templates was developed at one center and tested at a second center on reports from 2010 to 2025. The open-weight model gpt-oss-120B selects the template in three constrained-decoding steps and fills it on one local graphics processing unit. Template selection was scored against expert labels on 914 randomly sampled reports of five modalities, structuring quality on 920 radiography and CT reports corrected field by field by five residents. The pipeline then processed the complete archive of the second center. Proportions are reported with Wilson 95% confidence intervals (CIs). Results: An optimal template set was selected for 74.4% of reports (680 of 914; 95% CI: 71.5%, 77.1%) and an appropriate set for 82.3% (752 of 914; 95% CI: 79.7%, 84.6%), 87.7% for single-region and 54.1% for multi-region reports. Macro semantic textual similarity between output and corrected reference was 0.95 for radiography and 0.97 for CT, residents left 88.7% of 24,638 fields unchanged, and unsupported content was flagged in 1.0% and 1.5% of reports. Of 2,186,982 archive reports, 96.5% received structured output, 2,401,544 structured reports, at 1,258 reports per hour on one graphics processing unit. Conclusion: An open-weight LLM pipeline structured a complete multimodality report archive without human oversight with high content fidelity. Multi-region reports remained the main source of template errors.

Friedrich Puttkammer, Fabian Drexel, Marlene Fritzsche +19

Department of Radiology, Charité – Berlin University of Medicine, corporate member of Freie Universität Berlin and Humboldt Universität zu Berlin, Berlin, Germany · Department of Diagnostic and Interventional Radiology, School of Medicine and Health, TUM Klinikum Rechts der Isar, Technical University of Munich, Munich, Germany · Chair for AI in Healthcare and Medicine, Technical University of Munich (TUM) and TUM University Hospital, Munich, Germany +9

Medical Report GenerationRadiology Report GenerationOpen-Weight Language ModelsRadiology Report Evaluation
AskPDF
Feb 4, 2025

Open Foundation Models in Healthcare: Challenges, Paradoxes, and Opportunities with GenAI Driven Personalized Prescription

In response to the success of proprietary Large Language Models (LLMs), there is a growing interest in developing open, non-proprietary LLMs and AI foundation models (AIFMs) for transparent use in academic, scientific, and non-commercial applications. Open models have substantial potential to improve healthcare applications. In this paper, we provide a comprehensive survey and a taxonomy of state-of-the-art open LLMs and AIFMs for clinical tasks. In addition, we empirically evaluate open-weight LLMs on a personalized-prescription task and compare them with a proprietary model with and without Retrieval-Augmented Generation (RAG). Across 100 synthetic patient profiles, the open-weight models matched or exceeded the proprietary baseline on overall suitability accuracy without retrieval augmentation (0.810--0.850 versus 0.780), although the confidence intervals overlap; with retrieval augmentation, every model improved, and the ranking closed to within 0.04. Gains from RAG were field-dependent rather than uniform. To highlight the clinical relevance of LLM-driven personalized prescriptions, we also conduct a subjective evaluation in which two raters, blinded to model and condition, scored responses independently of each other. Their ratings did not follow the accuracy gains: retrieval raised overall suitability accuracy for every model while lowering the two-rater means for two of the four on all five dimensions. Finally, we outline the ethical implications and potential risks associated with the misuse of advanced LLMs and AIFMs, highlighting the importance of responsible and transparent deployment in healthcare contexts. \textit{Data and code used in the paper will be released upon publication.

Mahdi Alkaeed, Sofiat Abioye, Adnan Qayyum +6

Department of Computer Science and Engineering, College of Engineering, Qatar University, Doha, Qatar · Birmingham City University, Birmingham, United Kingdom · College of Science and Engineering, Hamad Bin Khalifa University (HBKU), Doha, Qatar +2

Medical LLMsOpen-Weight Language ModelsClinical Language Model EvaluationClinical Decision Support
AskPDF