Balanced Thinking: Improving Chain of Thought Training in Vision Language Models

Shaked Perek; Ben Wiesel; Avihu Dekel; Nimrod Shabtay; Eli Schwartz

cs.AIMar 19, 2026

Balanced Thinking: Improving Chain of Thought Training in Vision Language Models

Authors: Shaked Perek, Ben Wiesel, Avihu Dekel, Nimrod Shabtay, Eli Schwartz

Organizations: IBM Research

Abstract

Multimodal reasoning in vision-language models (VLMs) typically relies on a two-stage process: supervised fine-tuning (SFT) and reinforcement learning (RL). In standard SFT, all tokens contribute equally to the loss, even though reasoning data are inherently token-imbalanced. Long <think> traces overshadow short but task-critical <answer> segments, leading to verbose reasoning and inaccurate answers. We propose SCALe (Scheduled Curriculum Adaptive Loss), which explicitly separates supervision over reasoning and answer segments using dynamic, length-independent weighting. Unlike vanilla SFT, which overweights the <think> segment, SCALe-SFT gradually shifts the focus from <think> to <answer> throughout training via a cosine scheduling policy, encouraging concise and well-grounded reasoning. We evaluate SCALe across diverse benchmarks and architectures. Results show that SCALe consistently improves accuracy over vanilla SFT and matches the performance of the full two-phase SFT + GRPO pipeline while requiring only about one-seventh of the training time, making it a lightweight yet effective alternative. When combined with GRPO, SCALe achieves the best overall performance, highlighting its value both as a standalone method and as a strong foundation for reinforcement refinement.

Figures & tables

Figure 1: Scheduled Curriculum Adaptive Loss progression over training time vs vanilla SFT. In standard SFT, the <think> portion dominates the output compared to short answer segments (upper panel). SFT applies uniform token weights, making the loss skewed towards the extended reasoning segment (lower left panel). SCALe addresses this token imbalance and introduces a time-dependent weighting schedule that progressively amplifies the influence of answer tokens. As training advances, this shifts the loss toward the final answer, guiding the model to prioritize correct outcomes rather than lengthy reasoning (lower right panel).

Model	Dataset	Base	SFT	SCALe-SFT	GRPO	SCALe-GRPO
LLaVA-v1.6-Mistral-7B	ScienceQA	72.58	80.17	82.6	80.86	83.39
LLaVA-v1.6-Mistral-7B	IconQA	37.39	67.47	68.25	68.21	69.17
Gemma-3-4b	ScienceQA	77.68	85.77	85.92	85.18	86.37
Gemma-3-4b	IconQA	36.73	64.8	65.83	64.33	67.8
Qwen2.5-VL-3B	ScienceQA	80.6	85	85.52	85.08	85.82
Qwen2.5-VL-3B	IconQA	69.04	71.38	71.91	74.26	75.58

Table 1: Performance comparison for 3 models, on ScienceQA and IconQA benchmarks. Base is the original model without reasoning. SFT denotes vanilla supervised fine-tuning on a reasoning dataset; SCALe denotes our scheduled weighted loss; GRPO and SCALe+GRPO refer to reinforcement stages with initialization from vanilla and weighted-loss SFT, respectively. The reported metric is an exact match accuracy. We emphasize the best score in bold. SCALe outperforms vanilla SFT across all configurations (up to 3%) and 5% after GRPO.

Figure 2: Reasoning output from Qwen2.5, from HF test set lmms-lab/ICON-QA dataset. Blue (left) box is reasoning from vanilla-SFT. Orange (right) box is SCALe-SFT reasoning. Vanilla SFT counts the number of cars wrong.

Model	Dataset	SFT	SCALe-SFT	GRPO	SCALe-GRPO
LLaVA-v1.6-Mistral-7B	ScienceQA	4.76	2.78	2.93	1.78
LLaVA-v1.6-Mistral-7B	IconQA	2.54	0.32	0.09	0.05
Gemma-3-4b-it	ScienceQA	2.93	2.08	2.68	1.54
Gemma-3-4b-it	IconQA	2.20	1.68	0.05	0.05
Qwen2.5-VL-3B	ScienceQA	2.33	1.54	2.58	0.94
Qwen2.5-VL-3B	IconQA	0.58	0.53	0.01	0

Table 2: Percentage

\%

of generations that fail to produce <answer> tags. Lower values indicate better reasoning termination and structural consistency. SCALe and its GRPO substantially reduces invalid outputs compared to standard SFT and GRPO baselines.

Dataset	SFT	SFT-FW	SCALe-SFT	GRPO-FW	SCALe-GRPO
ScienceQA	85.0	85.18	85.52	84.23	85.82
IconQA	71.38	71.55	71.91	74.13	75.58

Table 3: Ablation study on the effect of the scheduled weighting mechanism for Qwen2.5-VL-3B. FW (fixed weights) denotes a configuration where all weights remain constant throughout training, effectively disabling the scheduling effect. Metrics are Exact Match.

Figure 3: Reasoning output from Qwen2.5, from HF test set lmms-lab/ScienceQA-IMG dataset. Blue bottom right box is reasoning from vanilla-SFT. Orange bottom left is SCALe-SFT reasoning. Vanilla SFT correctly counts two gas giants among eight planets yet concludes that 50% are gas, revealing a logical inconsistency between reasoning and answer, while Scheduled Weighted SFT maintains logical consistency and yields the correct conclusion.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4: Reasoning output from Gemma3, from HF test set lmms-lab/ScienceQA-IMG dataset. Blue bottom right box is reasoning from vanilla-SFT. Orange bottom left is SCALe-SFT reasoning. Vanilla SFT goes into an endless loop and cannot reach a final answer (shortened with …) while SCALe-SFT has consice reasoning steps and accurate final answer.

Figure 5: Reasoning output from Llava-next, from HF test set lmms-lab/ScienceQA-IMG dataset. Blue bottom right box is reasoning from vanilla-SFT. Orange bottom left is SCALe-SFT reasoning. In vanilla SFT the token imbalance can lead to well structured reasoning traces that fail to align with the final answer supervision- the model “thinks correctly” but concludes incorrectly.

Explore similar work

Saliency-Aware Multi-Route Thinking: Revisiting Vision-Language Reasoning
Feb 18, 2026Mingjia Shi, Yinhan He, Yaochen Zhu +1VLM Reasoning Test-Time Scaling for VLMs
University of Virginia.
See Further, Think Deeper: Advancing VLM's Reasoning Ability with Low-level Visual Cues and Reflection
Apr 27, 2026Zhiheng Wu, Tong Wang, Shuning Wang +2Tool Use in VLMs VLM Reasoning
1Baidu Inc. · †Independent researcher. · 2Zhejiang University +1
Be Faithful When Response: Returning Fluent and Grounded Answers for Vision-Language Models Reinforcement Learning
Jun 29, 2026Peng, Lee, Yin Zhang +9Vision-Language Grounding VLM Reasoning
GMLab · Hong Kong Polytechnic University · XPENG Robotics

Feb 18, 2026J/K move · Enter open · S save

Saliency-Aware Multi-Route Thinking: Revisiting Vision-Language Reasoning

Mingjia Shi, Yinhan He, Yaochen Zhu, Jundong Li

University of Virginia.

VLM Reasoning Test-Time Scaling for VLMs Vision-Language Grounding VLM Hallucination Mitigation

Open paper AskPDF arXiv

Vision-language models (VLMs) aim to reason by jointly leveraging visual and textual modalities. While allocating additional inference-time computation has proven effective for large language models (LLMs), achieving similar scaling in VLMs remains challenging. A key obstacle is that visual inputs are typically provided only once at the start of generation, while textual reasoning (e.g., early visual summaries) is generated autoregressively, causing reasoning to become increasingly text-dominated and allowing early visual grounding errors to accumulate. Moreover, vanilla guidance for visual grounding during inference is often coarse and noisy, making it difficult to steer reasoning over long texts. To address these challenges, we propose Saliency-Aware Principle (SAP) selection. SAP operates on high-level reasoning principles rather than token-level trajectories, which enable stable control over discrete generation under noisy feedback while allowing later reasoning steps to re-consult visual evidence when renewed grounding is required. In addition, SAP supports multi-route inference, enabling parallel exploration of diverse reasoning behaviors. SAP is model-agnostic and data-free, requiring no additional training. Empirical results show that SAP achieves competitive performance, especially in reducing object hallucination, under comparable token-generation budgets while yielding more stable reasoning and lower response latency than CoT-style long sequential reasoning.

Abstract

Explore similar work

Saliency-Aware Multi-Route Thinking: Revisiting Vision-Language Reasoning

See Further, Think Deeper: Advancing VLM's Reasoning Ability with Low-level Visual Cues and Reflection

Be Faithful When Response: Returning Fluent and Grounded Answers for Vision-Language Models Reinforcement Learning