cs.CLNov 19, 2026

PAIR-MED: Policy-reward Alternation for Iterative Refinement in Multilingual Medical Reasoning

Authors: Fan Gao, Xiaohang Xu, Lang Huang, Nan Lu, Irene Li

Organizations: The University of Tokyo · National Institute of Informatics · University of Bristol

Abstract

Large language models demonstrate strong medical question-answering capabilities, yet reasoning performance remains uneven across languages. Process reward models (PRMs) provide step-level feedback for improving reasoning, but a fixed PRM may become increasingly misaligned with the trajectory distribution of an evolving policy during reinforcement learning, thereby reducing the reliability of its supervision. We propose a policy--reward alternating training framework that jointly adapts a medical PRM and optimizes a multilingual reasoning policy. Following supervised initialization, the current policy generates multilingual trajectories to update the PRM, which then provides process-level rewards for the next policy update. Medical step supervision maintains the PRM's medical foundation, while process, outcome, and language-related rewards guide policy learning. We develop multilingual medical PRMs (PAIR-PRM) and reasoning models (PAIR-MED) at 8B and 9B scales across 12 languages. On the medical subsets of Global-MMLU and MMLU-ProX, our models improve average accuracy by 3.01--7.47% over their corresponding Qwen backbones. On CUREMED-BENCH, PAIR-MED-9B improves the joint language consistency and logical accuracy metric by 12.88% and 7.45% over baselines for in-domain and out-of-domain languages. Further analyses examine PRM adaptation, multilingual reasoning quality, and the contributions of individual training components.

Figures & tables

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Multilingual Medical Reasoning for Question Answering with Large Language Models

    Dec 5, 2025Pietro Ferrazzi, Aitor Soroa, Rodrigo AgerriKnowledge Augmentation for Language ModelsHealthcare

  2. MAPLE: Elevating Medical Reasoning from Statistical Consensus to Process-Led Alignment

    Mar 9, 2026Kailong Fan, Anqi Pu, Yichen Wu +7Medical LLMsRL for Language Models