Large language models demonstrate strong medical question-answering capabilities, yet reasoning performance remains uneven across languages. Process reward models (PRMs) provide step-level feedback for improving reasoning, but a fixed PRM may become increasingly misaligned with the trajectory distribution of an evolving policy during reinforcement learning, thereby reducing the reliability of its supervision. We propose a policy--reward alternating training framework that jointly adapts a medical PRM and optimizes a multilingual reasoning policy. Following supervised initialization, the current policy generates multilingual trajectories to update the PRM, which then provides process-level rewards for the next policy update. Medical step supervision maintains the PRM's medical foundation, while process, outcome, and language-related rewards guide policy learning. We develop multilingual medical PRMs (PAIR-PRM) and reasoning models (PAIR-MED) at 8B and 9B scales across 12 languages. On the medical subsets of Global-MMLU and MMLU-ProX, our models improve average accuracy by 3.01--7.47% over their corresponding Qwen backbones. On CUREMED-BENCH, PAIR-MED-9B improves the joint language consistency and logical accuracy metric by 12.88% and 7.45% over baselines for in-domain and out-of-domain languages. Further analyses examine PRM adaptation, multilingual reasoning quality, and the contributions of individual training components.
Figures & tables
Figure 1: Traditional Frozen-PRM Training vs. Alternating Policy–PRM Training in PAIR-Med .
Figure 2: Overview of PAIR-Med: Multilingual Policy and PRM Initialization (Stage One), followed by Alternating PRM Adaptation and Policy Optimization (Stage Two).
Global-MMLU
MMLU-ProX
Model
High
Medium
Low
Avg.
High
Medium
Low
Avg.
Qwen3-8B Yang et al. (2025a)
61.95
62.27
32.19
52.11
43.81
39.32
18.81
34.35
Apollo2-7B Zheng et al. (2025b)
62.38
51.70
31.67
49.47
38.86
28.57
11.75
27.25
BioMistral-7B Labrak et al. (2024)
41.19
39.38
28.27
36.43
18.54
10.24
13.32
14.72
CURE-Med-7B Onyame et al. (2026)
62.62
54.32
33.18
50.73
33.82
27.17
14.96
25.87
HuatuoGPT-3-8B Chen et al. (2026a)
70.23
63.19
34.05
56.41
46.81
37.84
19.39
35.43
Table 1: Accuracy (%) on Global-MMLU-medical and MMLU-ProX-health. High, Medium, and Low denote language resource groups. Within each model group, the highest and second-highest values per column are bolded and underlined, respectively. PAIR-Med accuracy is averaged over N=10 samples, while w/ PAIR-Prm selects the highest-scoring response among N=10 candidates. Per-language results are provided in the Appendix. B .
In-Domain
Out-of-Domain
Model
es
fr
ja
ko
th
sw
yo
Avg.
vi
hi
tr
am
bn
ha
Avg.
Qwen3-8B
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
HuatuoGPT-3-8B
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
Qwen3.5-9B
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
CURE-Med-7B
11.43
8.57
4.76
4.00
2.04
0.00
0.00
4.40
6.67
0.75
1.79
0.00
0.00
0.00
1.53
PAIR-Med -8B
22.86
25.71
15.24
13.33
9.18
2.86
0.00
12.74
14.29
6.72
8.04
0.00
0.71
0.00
4.96
Table 2: Zero-shot prompting results on CUREMED-BENCH: LC&LA. Reasoning baselines use thinking mode.
Figure 3: Accuracy changes relative to each base model across high-, medium-, and low-resource languages.
Figure 4: Stage-One and Stage-Two PRM best-of- N answer selection on Global-MMLU. The vertical zero line denotes self-consistency accuracy.
Table 5: Generation and optimization settings for the alternating-training runs. Each PRM and policy update uses one epoch.
SFT Cold Start
Multilingual PRM Initialization
Backbone
Training hours
GPU-hours
Training hours
GPU-hours
Qwen3-8B
0.73
5.86
13.39
107.10
Qwen3.5-9B
2.38
19.02
45.88
367.07
Appendix
Table 6: Stage-One training cost on eight GPUs.
Backbone
Alternating Training
Qwen3-8B
7.52
Qwen3.5-9B
25.22
Appendix
Table 7: GPU-hours for PRM and policy updates over all rounds of Stage Two training.
High
Medium
Low
Model
en
es
fr
zh
ja
ko
id
th
sw
wo
yo
zu
Avg.
Qwen3-8B
68.04
64.78
60.79
62.65
53.48
54.46
65.54
66.80
37.54
34.60
25.90
30.71
52.11
Apollo2-7B
69.90
57.81
57.34
66.84
60.01
52.62
54.41
48.07
38.67
29.32
29.36
29.31
49.47
BioMistral-7B
55.08
43.78
45.91
37.67
23.52
31.56
37.24
49.33
30.36
29.79
28.84
24.09
36.43
CURE-Med-7B
69.23
60.53
61.46
64.25
57.61
53.28
57.94
51.73
39.13
30.72
32.82
30.04
50.73
HuatuoGPT-3-8B
75.81
70.09
69.30
70.23
65.71
57.34
68.62
63.60
38.40
35.53
30.96
31.30
56.41
Appendix
Table 8: Accuracy (%) for each language on Global-MMLU. Within each of the two model groups, the highest value in each column is bold, and the second highest is underlined.
High
Medium
Low
Model
en
es
fr
zh
ja
ko
id
th
sw
wo
yo
zu
Avg.
Qwen3-8B
48.61
45.71
42.50
41.92
40.32
33.53
48.18
36.24
21.68
19.35
16.45
17.75
34.35
Apollo2-7B
47.89
37.41
37.99
38.86
32.17
32.89
31.44
21.39
13.10
12.22
11.93
9.75
27.25
BioMistral-7B
25.90
22.12
22.27
13.39
9.02
8.15
15.43
7.13
13.10
14.41
12.22
13.53
14.72
CURE-Med-7B
40.17
31.44
32.02
37.55
27.94
26.49
31.44
23.58
14.56
16.30
14.84
14.12
25.87
HuatuoGPT-3-8B
56.33
48.18
48.18
42.06
39.30
33.92
45.12
34.49
20.81
18.05
18.77
19.94
35.43
Appendix
Table 9: Accuracy (%) for each language on MMLU-ProX. Within each of the two model groups, the highest value in each column is bold, and the second highest is underlined.
In-Domain
Out-of-Domain
Model
es
fr
ja
ko
th
sw
yo
Avg.
vi
hi
tr
am
bn
ha
Avg.
Logical Accuracy
CURE-Med-7B
80.95
72.53
67.35
50.27
62.14
42.83
35.74
58.83
67.71
47.16
32.61
23.96
51.48
49.94
45.48
Qwen3-8B
72.98
74.15
73.79
64.43
59.79
25.65
5.30
53.73
66.05
67.97
61.16
26.14
56.60
6.87
47.47
HuatuoGPT-3-8B
35.37
34.27
31.50
28.17
31.03
14.56
5.60
25.79
34.65
36.38
20.80
15.91
30.97
5.24
23.99
PAIR-Med -8B
67.33
71.47
64.81
50.43
53.45
22.30
15.11
49.27
61.89
37.73
41.19
5.47
27.87
3.20
29.56
Appendix
Table 10: Detailed CUREMED-BENCH results (%) under zero-shot prompting.
High-resource
Medium-resource
Low-resource
Training method
en
es
fr
zh
ja
ko
id
th
sw
wo
yo
zu
Avg.
Qwen3-8B
Base model
68.04
64.78
60.79
62.65
53.48
54.46
65.54
66.80
37.54
34.60
25.90
30.71
52.11
Stage-one SFT
75.34
66.71
66.91
68.82
57.47
56.40
53.26
56.06
31.68
23.54
23.47
29.15
50.73
Native SFT
72.82
66.82
66.71
68.49
61.42
57.32
65.75
60.87
39.35
40.24
34.86
35.87
55.88
Native GRPO
74.68
70.23
69.10
71.09
64.65
60.46
69.69
63.60
36.74
27.78
10.89
23.36
53.52
Appendix
Table 11: Accuracy (%) for each training method and language on Global-MMLU. Bold marks the highest value within each backbone block; ties share the same marking.
High-resource
Medium-resource
Low-resource
Training method
en
es
fr
zh
ja
ko
id
th
sw
wo
yo
zu
Avg.
Qwen3-8B
Base model
48.61
45.71
42.50
41.92
40.32
33.53
48.18
36.24
21.68
19.35
16.45
17.75
34.35
Stage-one SFT
57.73
44.91
44.89
46.03
38.05
34.11
34.59
32.69
17.94
10.36
11.11
12.37
32.07
Native SFT
54.10
45.40
45.74
47.52
39.57
34.64
43.25
34.78
20.61
21.92
16.89
18.05
35.21
Native GRPO
56.62
47.88
47.16
49.78
40.32
37.26
47.30
36.53
17.75
13.10
4.30
10.77
34.06
Appendix
Table 12: Accuracy (%) for each training method and language on MMLU-ProX. Avg. is the unweighted mean over all 12 languages. Bold marks the highest value within each backbone block; ties share the same marking.
Inference strategy
High
Medium
Low
Avg.
Native GRPO-8B
Policy alone
69.95
64.58
24.69
53.52
Self-consistency
73.54
68.33
38.45
60.54
Stage-One PRM
75.55
71.20
39.48
62.44
Stage-Two PRM
75.27
71.49
41.08
62.93
PAIR-Med -8B
Appendix
Table 13: Different inference accuracy (%) on Global-MMLU by resource group. Bold marks the highest value among inference strategies for the same policy.
PAIR-Med -8B
PAIR-Med -9B
Language
GPT
PRM-8B
ρ
GPT
PRM-9B
ρ
en
0.855
0.742
0.244
0.871
0.736
0.462
es
0.727
0.695
0.433
0.795
0.735
0.411
fr
0.707
0.668
0.511
0.777
0.684
0.485
zh
0.780
0.718
0.412
0.824
0.716
0.340
ja
0.669
0.703
0.334
0.738
0.723
0.315
Appendix
Table 14: Reasoning quality and PAIR-Prm agreement for each language.
High-resource
Medium-
Low
Inference strategy
en
es
fr
zh
ja
ko
id
th
sw
wo
yo
zu
Avg.
Native GRPO-8B
Policy alone
74.68
70.23
69.10
71.09
64.65
60.46
69.69
63.60
36.74
27.78
10.89
23.36
53.52
Self-consistency
78.47
73.55
73.22
73.88
68.57
63.98
73.09
67.93
42.52
38.41
34.08
38.79
60.54
Stage-One PRM
79.66
74.42
75.38
77.60
70.69
67.71
75.70
70.20
45.58
39.61
33.55
39.18
62.44
Stage-Two PRM
79.26
74.42
74.55
77.47
70.63
68.11
76.03
70.33
46.77
42.35
35.02
40.18
62.93
Appendix
Table 15: Different inference accuracy (%) for each language on Global-MMLU using 8B policies.
High
Medium
Low
Inference strategy
en
es
fr
zh
ja
ko
id
th
sw
wo
yo
zu
Avg.
Native GRPO-9B
Policy alone
79.33
77.40
76.27
75.28
70.56
69.90
76.30
61.86
58.27
49.36
50.09
53.00
66.47
Self-consistency
82.25
78.73
78.67
78.00
73.88
71.82
78.70
67.06
62.65
55.24
55.68
58.74
70.12
Stage-One PRM
83.12
79.73
80.73
79.47
76.27
74.48
79.71
70.93
65.78
57.98
57.07
60.21
72.12
Stage-Two PRM
82.45
80.53
80.53
78.87
76.41
74.61
79.77
70.47
66.51
59.58
55.61
62.21
72.30
Appendix
Table 16: Differnt inference accuracy (%) for each language on Global-MMLU using 9B policies.