Multimodal reasoning in vision-language models (VLMs) typically relies on a two-stage process: supervised fine-tuning (SFT) and reinforcement learning (RL). In standard SFT, all tokens contribute equally to the loss, even though reasoning data are inherently token-imbalanced. Long <think> traces overshadow short but task-critical <answer> segments, leading to verbose reasoning and inaccurate answers. We propose SCALe (Scheduled Curriculum Adaptive Loss), which explicitly separates supervision over reasoning and answer segments using dynamic, length-independent weighting. Unlike vanilla SFT, which overweights the <think> segment, SCALe-SFT gradually shifts the focus from <think> to <answer> throughout training via a cosine scheduling policy, encouraging concise and well-grounded reasoning. We evaluate SCALe across diverse benchmarks and architectures. Results show that SCALe consistently improves accuracy over vanilla SFT and matches the performance of the full two-phase SFT + GRPO pipeline while requiring only about one-seventh of the training time, making it a lightweight yet effective alternative. When combined with GRPO, SCALe achieves the best overall performance, highlighting its value both as a standalone method and as a strong foundation for reinforcement refinement.
Figures & tables
Figure 1: Scheduled Curriculum Adaptive Loss progression over training time vs vanilla SFT. In standard SFT, the <think> portion dominates the output compared to short answer segments (upper panel). SFT applies uniform token weights, making the loss skewed towards the extended reasoning segment (lower left panel). SCALe addresses this token imbalance and introduces a time-dependent weighting schedule that progressively amplifies the influence of answer tokens. As training advances, this shifts the loss toward the final answer, guiding the model to prioritize correct outcomes rather than lengthy reasoning (lower right panel).
Model
Dataset
Base
SFT
SCALe-SFT
GRPO
SCALe-GRPO
LLaVA-v1.6-Mistral-7B
ScienceQA
72.58
80.17
82.6
80.86
83.39
IconQA
37.39
67.47
68.25
68.21
69.17
Gemma-3-4b
ScienceQA
77.68
85.77
85.92
85.18
86.37
IconQA
36.73
64.8
65.83
64.33
67.8
Qwen2.5-VL-3B
ScienceQA
80.6
85
85.52
85.08
85.82
IconQA
69.04
71.38
71.91
74.26
75.58
Table 1: Performance comparison for 3 models, on ScienceQA and IconQA benchmarks. Base is the original model without reasoning. SFT denotes vanilla supervised fine-tuning on a reasoning dataset; SCALe denotes our scheduled weighted loss; GRPO and SCALe+GRPO refer to reinforcement stages with initialization from vanilla and weighted-loss SFT, respectively. The reported metric is an exact match accuracy. We emphasize the best score in bold. SCALe outperforms vanilla SFT across all configurations (up to 3%) and 5% after GRPO.
Figure 2: Reasoning output from Qwen2.5, from HF test set lmms-lab/ICON-QA dataset. Blue (left) box is reasoning from vanilla-SFT. Orange (right) box is SCALe-SFT reasoning. Vanilla SFT counts the number of cars wrong.
Model
Dataset
SFT
SCALe-SFT
GRPO
SCALe-GRPO
LLaVA-v1.6-Mistral-7B
ScienceQA
4.76
2.78
2.93
1.78
IconQA
2.54
0.32
0.09
0.05
Gemma-3-4b-it
ScienceQA
2.93
2.08
2.68
1.54
IconQA
2.20
1.68
0.05
0.05
Qwen2.5-VL-3B
ScienceQA
2.33
1.54
2.58
0.94
IconQA
0.58
0.53
0.01
0
Table 2: Percentage % of generations that fail to produce <answer> tags. Lower values indicate better reasoning termination and structural consistency. SCALe and its GRPO substantially reduces invalid outputs compared to standard SFT and GRPO baselines.
Dataset
SFT
SFT-FW
SCALe-SFT
GRPO-FW
SCALe-GRPO
ScienceQA
85.0
85.18
85.52
84.23
85.82
IconQA
71.38
71.55
71.91
74.13
75.58
Table 3: Ablation study on the effect of the scheduled weighting mechanism for Qwen2.5-VL-3B. FW (fixed weights) denotes a configuration where all weights remain constant throughout training, effectively disabling the scheduling effect. Metrics are Exact Match.
Figure 3: Reasoning output from Qwen2.5, from HF test set lmms-lab/ScienceQA-IMG dataset. Blue bottom right box is reasoning from vanilla-SFT. Orange bottom left is SCALe-SFT reasoning. Vanilla SFT correctly counts two gas giants among eight planets yet concludes that 50% are gas, revealing a logical inconsistency between reasoning and answer, while Scheduled Weighted SFT maintains logical consistency and yields the correct conclusion.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 4: Reasoning output from Gemma3, from HF test set lmms-lab/ScienceQA-IMG dataset. Blue bottom right box is reasoning from vanilla-SFT. Orange bottom left is SCALe-SFT reasoning. Vanilla SFT goes into an endless loop and cannot reach a final answer (shortened with …) while SCALe-SFT has consice reasoning steps and accurate final answer.
Figure 5: Reasoning output from Llava-next, from HF test set lmms-lab/ScienceQA-IMG dataset. Blue bottom right box is reasoning from vanilla-SFT. Orange bottom left is SCALe-SFT reasoning. In vanilla SFT the token imbalance can lead to well structured reasoning traces that fail to align with the final answer supervision- the model “thinks correctly” but concludes incorrectly.
Vision-language models (VLMs) aim to reason by jointly leveraging visual and textual modalities. While allocating additional inference-time computation has proven effective for large language models (LLMs), achieving similar scaling in VLMs remains challenging. A key obstacle is that visual inputs are typically provided only once at the start of generation, while textual reasoning (e.g., early visual summaries) is generated autoregressively, causing reasoning to become increasingly text-dominated and allowing early visual grounding errors to accumulate. Moreover, vanilla guidance for visual grounding during inference is often coarse and noisy, making it difficult to steer reasoning over long texts. To address these challenges, we propose Saliency-Aware Principle (SAP) selection. SAP operates on high-level reasoning principles rather than token-level trajectories, which enable stable control over discrete generation under noisy feedback while allowing later reasoning steps to re-consult visual evidence when renewed grounding is required. In addition, SAP supports multi-route inference, enabling parallel exploration of diverse reasoning behaviors. SAP is model-agnostic and data-free, requiring no additional training. Empirical results show that SAP achieves competitive performance, especially in reducing object hallucination, under comparable token-generation budgets while yielding more stable reasoning and lower response latency than CoT-style long sequential reasoning.
Recent advances in Vision-Language Models (VLMs) have benefited from Reinforcement Learning (RL) for enhanced reasoning. However, existing methods still face critical limitations, including the lack of low-level visual information and effective visual feedback. To address these problems, this paper proposes a unified multimodal interleaved reasoning framework ForeSight, which enables VLMs to See Further with low-level visual cues and Think Deeper with effective visual feedback. First, it introduces a set of low-level visual tools to integrate essential visual information into the reasoning chain, mitigating the neglect of fine-grained visual features. Second, a mask-based visual feedback mechanism is elaborated to incorporate visual reflection into the thinking process, enabling the model to dynamically re-examine and update its answers. Driven by RL, ForeSight learns to autonomously decide on tool invocation and answer verification, with the final answer accuracy as the reward signal. To evaluate the performance of the proposed framework, we construct a new dataset, Character and Grounding SalBench (CG-SalBench), based on the SalBench dataset. Experimental results demonstrate that the ForeSight-7B model significantly outperforms other models with the same parameter scale, and even surpasses the current SOTA closed-source models on certain metrics.
Zhiheng Wu, Tong Wang, Shuning Wang +2
1Baidu Inc. · †Independent researcher. · 2Zhejiang University +1
Reinforcement Learning (RL) is an important paradigm for improving the reasoning capabilities of Vision-Language Models (VLMs). However, directly applying RL to rollout multimodal reasoning can lead to instability, due to the exploitation of language priors, the neglect of visual evidence, and the generation of reasoning traces that are fluent yet not visually grounded. The question arises: Can initially steer the policy toward visually faithful reasoning regime before applying reinforcement learning? To this end, we propose a Faithful Warm-Start (FWS) strategy that first curates samples with explicit vision-language causal relationships from six general VQA benchmarks to construct the FaithfulQA dataset, where each of the image-question pairs gains a certain degree of visual observations, question requirements, commonsense knowledge, domain knowledge, and the final answer. Subsequently, a VLM-based judge is employed to further purify the dataset, ensuring strong causal consistency and visual faithfulness. This warm-start stage equips the model with the capability to understand causally grounded vision-language patterns before subsequent RL optimization under sparse answer-level rewards. Experimental results show that such faithful supervision improves answer accuracy, stabilizes RL training, and reduces visually unsupported reasoning.
Peng, Lee, Yin Zhang +9
GMLab · Hong Kong Polytechnic University · XPENG Robotics