As large language models enter professional domains, they must satisfy domain constraints, include critical evidence, and provide complete reasoning rather than merely produce fluent responses. Existing post-training methods often rely on holistic preferences or outcome-level verification, while recent rubric-based methods usually generate rubrics independently for each query. In specialized domains, such unconstrained rubrics may omit critical requirements and vary across samples, hindering the diagnosis and targeted repair of persistent capability deficiencies. We propose APTER (Adaptive Post-Training with Expert-Grounded Rubrics), a framework that integrates structured domain knowledge into fine-grained evaluation, optimization, and diagnosis for specialized complex reasoning. First, expert-grounded rubric construction starts from an expert criteria framework built by domain experts, where each criterion represents a stable professional capability. For each query, APTER selects relevant criteria and instantiates them into query-level rubrics linked to their source criteria, turning reusable expert criteria into executable query-level supervision without reference answers. Second, adaptive post-training uses rubric verdicts as both optimization and criterion-level diagnostic signals. Aggregating low-scoring verdicts by criterion ID reveals persistent deficiencies and triggers targeted supervised fine-tuning updates during reinforcement learning. Experiments on mathematical reasoning and medical question answering show consistent gains across both domains. Across three model generations, APTER improves the mathematics and medical averages over the corresponding base models by up to 15.86 and 8.04 points, respectively. Code and rubric datasets are available at https://github.com/AntDT-APTER/APTER.
Figures & tables
Figure 1: Comparison of post-training supervision signals. RLHF provides coarse response-level preferences and RLVR provides outcome verification, while existing rubric-based methods often generate query-level rubrics without explicit grounding in expert-defined criteria. APTER integrates structured domain knowledge into rubric construction and uses rubric verdicts for fine-grained optimization, capability diagnosis, and targeted repair.
Figure 2: The APTER pipeline. Expert-grounded rubric construction first builds an expert criteria framework and instantiates relevant criteria into query-level rubrics through routing, multi-role generation and refinement, and expert-in-the-loop calibration. The resulting rubric verdicts preserve criterion back-mapping and are used for Rubric RL, Rubric-based SFT, and Ada-IFT.
Figure 3: Ada-IFT training flow. Rubric verdicts provide both a scalar reward for the RL update and criterion-level diagnostic signals. Persistent criterion failures trigger targeted SFT before training returns to RL.
(a) Mathematical Reasoning
Model Family
Variant
AIME24
AIME25
MATH500
Olympiad
Avg.
Qwen2.5-7B-Instruct
Base
13.33
4.61
68.84
38.71
31.37
APTER
34.27 (+20.94)
12.74 (+8.13)
86.10 (+17.26)
52.34 (+13.63)
46.36 (+14.99)
Qwen3-8B
Base
23.33
18.33
84.26
55.89
45.45
APTER
49.15 (+25.82)
35.00 (+16.67)
92.60 (+8.34)
68.50 (+12.61)
61.31 (+15.86)
Qwen3.5-9B
Base
62.00
53.13
97.20
78.64
72.74
Table 1: Main results across the Qwen2.5, Qwen3, and Qwen3.5 model generations. Each APTER row reports the final score followed by its absolute improvement over the corresponding unmodified model in green parentheses . Mathematical reasoning and medical question answering averages use four and three benchmarks, respectively. All metrics are higher-is-better.
Variant
Ablation Configuration
Downstream Performance
Rubric Configuration
Reward Type
Training Strategy
MATH500
AIME24
Olymp.
Health
(A) Post-training Components Rubric data, rubric reward, and Ada-IFT
Qwen3-8B
—
—
Base
84.26
23.33
55.89
45.43
SFT
Plain CoT data
—
SFT
84.30
26.79
55.71
44.62
RiC-SFT
Rubric-augmented CoT data
—
SFT
88.86
28.37
57.05
47.10
Rubric RL
Query-level rubric data
Rubric Reward
GRPO
90.92
41.63
58.97
50.54
Table 2: Unified summary of the five performance ablations on Qwen3-8B. Light-blue section rows identify the factor isolated in each block. The best result among fully reported comparable variants within each block is highlighted in bold.
Figure 4: Criterion-level repair dynamics with and without Ada-IFT. Curves show exponential moving averages of criterion failure rates for three randomly selected criteria; lower is better. Percentages report the relative reduction in normalized failure-rate AUC.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Source
Queries
LiveMedBench
4,865
SpeechMedDataset
4,933
II-Medical (via RubricHub)
4,915
Main medical training set
14,713
Medical ablation set (complete LiveMedBench)
4,865
Appendix
Table 3: Medical training data composition.
Mathematics sub-domain
Capabilities
Leaf criteria
Analysis
4
16
Number theory
4
16
Discrete mathematics
4
16
Arithmetic
3
13
Applied mathematics
3
14
Trigonometry
3
14
Appendix
Table 4: Composition of the mathematics expert criteria framework.
Capability
Criteria
Criterion definitions
Medical-knowledge accuracy
7
Factual correctness; medical-terminology accuracy; guideline and consensus adherence; avoidance of harmful advice; examination and test interpretation; medication and dosage accuracy; differential-diagnosis plausibility.
Coverage of key information; provision of alternatives; treatment-plan completeness; follow-up and monitoring guidance; prevention and health education; risk warnings.
Uncertainty handling
5
Identification and communication of uncertainty; questions that reduce uncertainty; transparent disclosure of irreducible uncertainty; balance between a direct answer and appropriate qualification; avoidance of overconfidence.
Individualized advice; proactive follow-up questions; disclosure of assumptions and premises; multi-turn coherence; use of existing context; avoidance of redundant questions.
Appendix
Table 5: Composition of the medical expert criteria framework.
Large language model post-training adapts pretrained models into more useful, reliable, and specialized assistants through supervised and preference-based learning.
Evaluations should do more than measure a models current performance. They should tell us what to fix for the next model iteration and provide a way to generate targeted post training data. Most evaluation pipelines identify weak examples, topics, or categories, but they leave the underlying capability failure implicit: they say where a model fails, not why. We introduce CRAFT, a method that converts any rubric based evaluation dataset into a model specific diagnosis of weak capabilities. CRAFT treats each grading criterion as a capability probe: it extracts a capability description from every prompt rubric pair, clusters these descriptions into a hierarchical capability tree, scores the target model at every node, and selects low performing nodes dynamically across tree levels, at the granularity where each failure is clearest. The selected weak capabilities then direct the generation of targeted supervised finetuning data. Holding the data generation, finetuning, and evaluation setup fixed, we compare CRAFT against prompt level EvalTree clustering and untargeted random generation on four open source models, two professional domains (finance and legal), and 13 held out benchmarks disjoint from the diagnostic data. CRAFT achieves the strongest finance domain average for all four models under repeated temperature decoding; on legal domain, it is strongest for three of four models and remains within the decoding variance bands of the best baseline on the fourth. Diagnosing weaknesses at the level of rubric criteria, rather than prompts or categories, thus yields both a sharper picture of what a model cannot do and measurably better models after finetuning on that diagnosis.
Vipul Gupta, Zihao Wang, Razvan-Gabriel Dumitru +3
Reliable rubric grading requires more than accurate score prediction. Each judgement must be grounded in the mark scheme and evidence from the student answer. Existing credit-assignment and intervention methods, primarily designed for self-contained reasoning tasks such as mathematics reasoning, struggle in this setting because they do not identify where grading reasoning goes wrong or how the model's belief about the final mark changes during reasoning. We propose Evidence-Diagnosed Intervention Training (EDIT), a two-phase framework for training more rubric-faithful LLM graders. First, EDIT-SFT locates problematic reasoning steps using internal model signals: posterior belief over the final mark and input-grounding scores. It then revises only these local steps with help from a rubric checklist. Second, EDIT-RL calibrates the grader with belief-guided reward shaping, penalising large harmful belief drifts while still allowing helpful exploration. Experiments on two real-world, multi-subject grading benchmarks demonstrate that EDIT consistently outperforms strong supervised fine-tuning and reinforcement learning baselines on both in-domain and out-of-domain splits, with ablation studies confirming that internal-state diagnostics drive these gains. Under deterministic rubric-edit interventions, EDIT-SFT is the most rule-responsive of all evaluated systems, and EDIT-RL largely retains this responsiveness while improving accuracy.
Zhihao Wu, Linhai Zhang, Taiyi Wang +4
1King’s College London · University of Cambridge · 3AQA +1
Reinforcement learning with verifiable rewards improves language model reasoning, but its reliance on domain-specific verifiers, sparse outcome rewards, and coarse-grained credit assignment limits its applicability. We introduce GeneralThinker, an on-policy framework that reformulates reasoning supervision as dense answer-conditioned optimization, enabling response-level evaluation and token-level credit assignment without domain-specific verifiers. GeneralThinker evaluates generated reasoning trajectories using the likelihood of the ground-truth answer and derives token-wise compatibility signals for fine-grained credit assignment. To stabilize optimization, it constrains token-level updates through clipping and direction-preserving modulation. Across 11 benchmarks spanning mathematics, STEM, and general reasoning, GeneralThinker achieves the best average performance. Further analyses show that uncontrolled token-level modulation can destabilize training, whereas controlled modulation makes fine-grained credit assignment consistently effective.