Overview

Medical report generation studies how to create diagnostic reports and clinical notes from images, patient records, or doctor–patient conversations. Approaches include vision-language models for radiology, pathology, and other imaging; speech-to-text pipelines for SOAP notes; and systems that use templates, retrieval, structured findings, or evidence-linked reasoning to guide generation. Evaluation increasingly uses clinical measures and expert review alongside language metrics on settings such as MIMIC-CXR and IU X-Ray. A central challenge is ensuring that reports are complete and factually grounded across hospitals and modalities, since fluent text and lexical similarity do not guarantee clinical accuracy or safety.

Latest papers

91 papers11 papers in the last four weeks, up 38% on the four weeks before. 0.1% of all new papers.

Jun 8, 2025

Lingshu: A Generalist Foundation Model for Unified Multimodal Medical Understanding and Reasoning

Multimodal Large Language Models (MLLMs) have demonstrated impressive capabilities in understanding common visual elements, largely due to their large-scale datasets and advanced training strategies. However, their effectiveness in medical applications remains limited due to the inherent discrepancies between data and tasks in medical scenarios and those in the general domain. Concretely, existing medical MLLMs face the following critical limitations: (1) limited coverage of medical knowledge beyond imaging, (2) heightened susceptibility to hallucinations due to suboptimal data curation processes, (3) lack of reasoning capabilities tailored for complex medical scenarios. To address these challenges, we first propose a comprehensive data curation procedure that (1) efficiently acquires rich medical knowledge data not only from medical imaging but also from extensive medical texts and general-domain data; and (2) synthesizes accurate medical captions, visual question answering (VQA), and reasoning samples. As a result, we build a multimodal dataset enriched with extensive medical knowledge. Building on the curated data, we introduce our medical-specialized MLLM: Lingshu. Lingshu undergoes multi-stage training to embed medical expertise and enhance its task-solving capabilities progressively. Besides, we preliminarily explore the potential of applying reinforcement learning with verifiable rewards paradigm to enhance Lingshu's medical reasoning ability. Additionally, we develop MedEvalKit, a unified evaluation framework that consolidates leading multimodal and textual medical benchmarks for standardized, fair, and efficient model assessment. We evaluate the performance of Lingshu on three fundamental medical tasks, multimodal QA, text-based QA, and medical report generation. The results show that Lingshu consistently outperforms the existing open-source multimodal models on most tasks ...