Organizations: Department of Computer Science and Engineering, The Chinese University of Hong Kong, Hong Kong, China · Institute of Medical Intelligence and XR, The Chinese University of Hong Kong, Hong Kong, China
Medical Multimodal Large Language Models (MLLMs) have demonstrated remarkable capabilities across diverse healthcare tasks. However, current post-training strategies, such as supervised fine-tuning and reinforcement learning, heavily depend on substantial annotated data while overlooking the potential of unlabeled test data for model enhancement. This limitation becomes particularly pronounced in medical domains, where acquiring extensive labeled medical data is difficult due to the strict data sensitivity and annotation complexity. Moreover, leveraging test data poses challenges in generating reliable supervision signals from unlabeled samples and maintaining stable self-evolution. To address these limitations, we propose Med-Evo, the first self-evolution framework for medical MLLMs that utilizes label-free reinforcement learning to promote model performance without requiring additional labeled data. Our framework introduces two key innovations: 1) Feature-driven Pseudo Labeling (FPL) that identifies semantic centroids from all heterogeneous candidate responses to select pseudo labels in each rollout, and 2) Hard-Soft Reward (HSR) that combines exact match with token-level assessment and semantic similarity to provide hierarchical reward. Experiments on three medical VQA benchmarks and two base MLLMs show clear advantages of our approach over SOTA methods, with significant improvements of 10.43% accuracy and 4.68% recall on the SLAKE dataset using Qwen2.5-VL, showing the effectiveness of our method.
Figures & tables
Figure 1: Overview of medical MLLM adaptation strategies. (a) Direct inference leads to poor generalization on down-stream tasks; (b) Supervised learning requires extensive labeled data; (c) Our test-time self-evolution strategy using unlabeled data.
Figure 2: Overview of our proposed Med-Evo framework for test-time self-evolution with four stages: 1) we prepare unlabeled data and the policy model πθold ; 2) we employ Feature-driven Pseudo Labeling (FPL) to generate reliable supervision signals from unlabeled data (Sec. 2.1 ); 3) we utilize the Hard-soft Reward (HSR) mechanism to provide hierarchical reward signals by incorporating finer-grained evaluation strategies (Sec. 2.2 ); 4) we leverage Group Relative Policy Optimization (GRPO) to achieve iterative unsupervised self-evolution through relative advantage computation (Sec. 2.3 ).
SLAKE
VQA-Rad
VQA-Med
Method
Acc
Recall
Rouge
Acc
Recall
Rouge
Acc
Recall
Rouge
Base Model
68.73
34.70
43.97
68.53
24.00
32.03
56.88
12.57
27.45
EN-INF [ 2 ]
65.92
35.55
45.23
67.73
24.00
32.42
55.96
10.18
25.16
TTRV [ 17 ]
63.66
33.00
43.48
62.55
19.50
29.68
55.04
10.78
25.75
TTRL [ 31 ]
72.68
35.41
48.00
68.13
24.00
33.17
55.96
9.58
25.28
Ours
78.87
39.38
51.01
69.32
25.00
33.72
57.79
14.97
27.96
Table 1: Performance comparison on three medical VQA datasets using Qwen2.5-VL-3B-Instruct as the base model. Bold indicates the best performance for each metric.
SLAKE
VQA-Rad
VQA-Med
Method
Acc
Recall
Rouge
Acc
Recall
Rouge
Acc
Recall
Rouge
Base Model
61.41
17.99
26.82
49.00
13.00
19.33
44.04
9.58
23.02
EN-INF [ 2 ]
68.73
25.21
33.50
58.57
17.00
24.58
46.79
10.18
23.02
TTRV [ 17 ]
68.45
25.64
33.74
59.36
17.50
25.02
44.95
10.18
21.83
TTRL [ 31 ]
71.27
26.20
36.72
57.37
17.00
25.08
38.53
6.59
20.23
Ours
71.55
30.03
39.66
61.75
18.00
25.37
48.62
10.18
23.42
Table 2: Performance comparison on three medical VQA datasets using MedVLM-R1 as the base model. Bold indicates the best performance for each metric.
Figure 3: Illustrative experiments based on Qwen2.5-VL-3B-Instruct on SLAKE dataset. (a) ablation study; (b) averaged reward score and model performance during the whole evolution process; (c) the hit rate comparison using base model.
Self-evolution of multimodal large language models (MLLMs) remains a critical challenge: pseudo-label-based methods suffer from progressive quality degradation as model predictions drift, while template-based methods are confined to a static set of transformations that cannot adapt in difficulty or diversity. We contend that robust, continuous self-improvement requires not only deterministic external feedback independent of the model's internal certainty, but also a mechanism to perpetually diversify the training distribution. To this end, we introduce EVE (Executable Visual transformation-based self-Evolution), a novel framework that entirely bypasses pseudo-labels by harnessing executable visual transformations continuously enriched in both variety and complexity. EVE adopts a Challenger-Solver dual-policy architecture. The Challenger maintains and progressively expands a queue of visual transformation code examples, from which it synthesizes novel Python scripts to perform dynamic visual transformations. Executing these scripts yields VQA problems with absolute, execution-verified ground-truth answers, eliminating any reliance on model-generated supervision. A multi-dimensional reward system integrating semantic diversity and dynamic difficulty calibration steers the Challenger to enrich its code example queue while posing progressively more challenging tasks, preventing mode collapse and fostering reciprocal co-evolution between the two policies. Extensive experiments demonstrate that EVE consistently surpasses existing self-evolution methods, establishing a robust and scalable paradigm for verifiable MLLM self-evolution. The code is available at https://github.com/0001Henry/EVE .
Yongrui Heng, Chaoya Jiang, Han Yang +2
School of Control Science and Engineering, Shandong University Jinan, China · Zeekr, Geely Auto Shanghai, China
Multimodal Large Language Models (MLLMs) have demonstrated impressive capabilities in understanding common visual elements, largely due to their large-scale datasets and advanced training strategies. However, their effectiveness in medical applications remains limited due to the inherent discrepancies between data and tasks in medical scenarios and those in the general domain. Concretely, existing medical MLLMs face the following critical limitations: (1) limited coverage of medical knowledge beyond imaging, (2) heightened susceptibility to hallucinations due to suboptimal data curation processes, (3) lack of reasoning capabilities tailored for complex medical scenarios. To address these challenges, we first propose a comprehensive data curation procedure that (1) efficiently acquires rich medical knowledge data not only from medical imaging but also from extensive medical texts and general-domain data; and (2) synthesizes accurate medical captions, visual question answering (VQA), and reasoning samples. As a result, we build a multimodal dataset enriched with extensive medical knowledge. Building on the curated data, we introduce our medical-specialized MLLM: Lingshu. Lingshu undergoes multi-stage training to embed medical expertise and enhance its task-solving capabilities progressively. Besides, we preliminarily explore the potential of applying reinforcement learning with verifiable rewards paradigm to enhance Lingshu's medical reasoning ability. Additionally, we develop MedEvalKit, a unified evaluation framework that consolidates leading multimodal and textual medical benchmarks for standardized, fair, and efficient model assessment. We evaluate the performance of Lingshu on three fundamental medical tasks, multimodal QA, text-based QA, and medical report generation. The results show that Lingshu consistently outperforms the existing open-source multimodal models on most tasks ...
Medical multimodal reasoning models are typically trained on teacher-generated rationales, discarding the valid reasoning in students' own failed trajectories, while general-domain correction methods offer little guidance for errors in image reading, anatomical context, or differential diagnosis. We propose MedEvo, which uses medical knowledge, namely a tag taxonomy and error-type diagnosis, to navigate both trajectory repair and data selection. For each failed rollout, a gold-blind auditor locates the first consequential error and provides a minimal, answer-free hint; the same student retains the valid prefix and regenerates the erroneous suffix. After each round, MedEvo identifies capability gaps across medical tags on a held-out feedback set and prioritizes related questions for new rollouts and repairs. With supervised fine-tuning (SFT) alone, MedEvo raises Qwen3.5-9B mean accuracy across seven benchmarks from 65.57% to 69.12% (47.40% to 54.50% on the multimodal track of MedXpertQA (MedXpertQA-MM)), surpassing teacher-distillation SFT (66.93%). It also raises Qwen3.5-35B and Qwen3.5-397B mean accuracy from 67.91% to 73.04% and from 71.83% to 74.95%, respectively. Our analyses show that failed trajectories contain reasoning worth preserving, that repair success does not imply learning value, and that independent feedback is key to sustained improvement.
Xiaoliang Wang, Zhuo Chen, Chuanyang Zheng +11
Zhejiang University · Ant Group · Shanghai Artificial Intelligence Laboratory +1