Robust perception and reasoning require consistency across sensory modalities. Yet current multimodal models often violate this principle, yielding contradictory predictions for visual and textual representations of the same concept. Rather than masking these failures with standard voting mechanisms, which can amplify systematic biases, we show that cross-modal inconsistency provides a rich and natural signal for learning. We introduce RC2, a reinforcement learning framework that resolves internal conflicts by enforcing cross-modal cycle consistency. By requiring a model to perform backward inference, switch modalities, and reliably reconstruct the answer through forward inference, we obtain a dense, label-free reward. This cyclic constraint encourages the model to align its internal representations autonomously. Optimizing for this structure mitigates modality-specific errors and improves reasoning accuracy by up to 7.6 points. Our results suggest that advanced reasoning emerges not only from scaling data, but also from enforcing a structurally consistent understanding of the world.
Figures & tables
Figure 1: Gap in Multimodal Reasoning. Multimodal large language models (MLLMs) frequently fail the test of modal-invariance. For example, they produce conflicting answers for the same webpage when presented as a screenshot versus its raw HTML source. We introduce a cycle-consistency framework that directly targets this modality gap, leveraging the inconsistency itself as a signal to jointly improve reasoning and alignment.
Figure 2: Failure of multimodal voting. Left: Consistent Conflict — both text and image modalities produce self-consistent predictions ( mode-stable ) but disagree with each other, and only one modality aligns with the ground truth. Right: Unstable Recovery — within a single modality, some rollouts yield the correct answer, but the majority vote remains wrong, reflecting intra-modal instability. Using multimodal voting can amplify biases or lose correct signals.
Figure 3: Overview of multimodal cycle consistency. Starting from a potential answer candidate aorig , the model performs backward inference to reconstruct two latent queries, q^T from the text view xT and q^I from the image view xI . Each reconstructed query is then used for forward inference across both modalities, resulting in four reconstructed answers {att,ati,ait,aii} generated via the paths T→T , T→I , I→T , and I→I . Cycle consistency is measured by whether the reconstructed answers remain consistent with the original aorig , forming a full 4-way cross-modal reasoning cycle.
Figure 4: Examples of the backward-inference (Answer → Query) step . Given a Candidate Answer, the model generates distinct, semantically-grounded queries for both the text and image modalities. This demonstrates the viability of the first step of our cycle-consistency reward, enabling the model to check its answer in the alternate modality.
Qwen2.5-VL-3B-Instruct
Dataset
Base Model
+ Voting (Text)
+ Voting (Image+Text)
+ R-C 2 (Ours)
Text Acc
Vision Acc
Text Acc
Vision Acc
Text Acc
Vision Acc
Text Acc
Vision Acc
ScienceQA
68.9
76.0
70.7 (+1.8)
76.2 (+0.2)
73.1 (+4.2)
78.0 (+2.0)
76.7 (+7.8)
83.3 (+7.3)
ChartQA
71.1
82.8
76.0 (+4.9)
83.6 (+0.8)
76.2 (+5.1)
83.5 (+0.7)
77.2 (+6.1)
84.8 (+2.0)
MathVista
49.8
64.8
50.6 (+0.8)
65.0 (+0.2)
52.1 (+2.3)
65.7 (+0.9)
55.8 (+6.0)
67.6 (+2.8)
VWA
69.0
62.9
74.5 (+5.5)
63.1 (+0.2)
73.3 (+4.3)
64.5 (+1.6)
74.5 (+5.5)
67.1 (+4.2)
Table 1: Multimodal Reasoning Accuracy. We compare the Base Model, Voting (Text-only), Voting (Image+Text), and our R-C 2 -RL method on Qwen2.5-VL-3B-Instruct (top) and Qwen3-VL-8B-Instruct (bottom). Each cell reports Text Accuracy (input content in text format) and Vision Accuracy (input content in image format), with green numbers indicating absolute improvements of R-C 2 over the base model. We bold the best accuracy for each dataset. Multimodal voting consistently outperforms text-only voting and the base model, and R-C 2 achieves the largest gains across nearly all benchmarks.
Dataset
Qwen2.5-VL-3B
Qwen3-VL-8B
Base
Voting (Text)
Voting (Image+Text)
R-C 2 (Ours)
Base
Voting (Text)
Voting (Image+Text)
R-C 2 (Ours)
ScienceQA
74.9
79.7 (+4.8)
81.7 (+6.8)
84.9 (+10.0)
74.7
75.3 (+0.6)
77.3 (+2.6)
78.3 (+3.6)
ChartQA
73.4
79.2 (+5.8)
78.9 (+5.5)
79.5 (+6.1)
81.4
82.7 (+1.3)
83.2 (+1.8)
83.5 (+2.1)
MathVista
57.3
58.8 (+1.5)
59.2 (+1.9)
61.4 (+4.1)
64.7
64.8 (+0.1)
69.0 (+4.3)
69.2 (+4.5)
VWA
63.9
66.9 (+3.0)
70.0 (+6.1)
67.1 (+3.2)
79.2
80.0 (+0.8)
79.8 (+0.6)
81.2 (+2.0)
A-OKVQA
67.8
80.1 (+12.3)
80.2 (+12.4)
80.3 (+12.5)
70.2
79.3 (+9.1)
80.3 (+10.1)
79.6 (+9.4)
Table 2: Multimodal Consistency Ratio. We compare both Qwen2.5-VL-3B and Qwen3-VL-8B. Each dataset is evaluated under four supervision settings: Base, +Voting (Text), +Voting (Image+Text), and + R-C 2 . Values in (green) indicate absolute improvements of R-C 2 over the base model. Across most datasets, R-C 2 achieves the highest cross-modal agreement.
Cycle Type
ScienceQA
ChartQA
Text
Vision
Cons.
Text
Vision
Cons.
Single
74.0
81.7
81.2
76.2
83.7
77.3
Cross
75.8
80.1
83.1
76.5
83.8
78.4
Mixed
76.7
83.3
84.9
77.2
84.8
79.5
Table 3: Ablation of cycle path configurations in R-C 2 on ScienceQA and ChartQA using Qwen2.5-VL-3B-Instruct. Single uses intra-modal cycles (I→I, T→T); Cross uses cross-modal cycles (I→T, T→I); Mixed combines all four paths. The mixed configuration yields the highest accuracy and consistency.
Figure 5: Visual comparison among the base model, voting baseline, and R-C 2 (ours). For each example, we display the original multimodal observation, the query, and the predictions from text and image modalities under different methods. The base model and voting baseline often yield either conflicting answers across modalities or spurious agreements on incorrect predictions, reflecting the modality gap and unreliable consensus confidence. In contrast, R-C 2 enforces cycle-based alignment, producing answers that are both correct and consistent across modalities.
Figure 6: Importance of cross modal in cycle consistency path. We show ScienceQA results under R-C 2 training with nine different forward–backward cycle paths. Backward denotes answer → query and forward denotes query → answer The heatmaps show (a) average accuracy and (b) cross-modal consistency across all modality configurations. Using both image and text modalities for backward query generation consistently leads to higher accuracy and stronger cross-modal agreement, suggesting that multimodal grounding provides a more reliable signal for cycle-consistent reasoning.
Source of Candidate
A-OKVQA
VWA
Vision Acc
Text Acc
Cons.
Vision Acc
Text Acc
Cons.
Base model
86.1
70.3
67.8
69.0
62.9
63.9
Self-generated answer
87.3
71.6
77.6
74.9
65.3
68.2
Training set answer
88.8
75.2
80.3
74.5
67.1
67.1
Table 4: Effect of candidate answer source for initiating the cycle . We compare using the model’s own predicted answer ( Self-generated ) versus using the reference answer from the dataset ( Training-set answer ). Results show comparable or better performance without requiring annotated QA pairs, highlighting the self-bootstrapping capability of R-C 2 .
Figure 7: Effect of cross-modal inconsistency ratio on ScienceQA performance. We fine-tune Qwen2.5-VL-3B-Instruct on ScienceQA with varying fractions of cross-modal inconsistent training samples. As the inconsistency ratio increases from 0% to 50%, both answer accuracy and cross-modal consistency systematically improve. This suggests that exposing the model to more modality conflicts provides a useful structural signal for the RL objective, enabling it to better resolve cross-modal disagreements and yielding more stable multimodal reasoning.
Figure 8: Examples from the reproposed VWA multiple-choice dataset constructed on shopping webpages. For each webpage, we generate 10 questions with 6 answer choices, among which 1 is correct. The questions cover diverse reasoning types, including product category recognition, price comparison, customer rating analysis, and attribute identification.
Figure 9: Examples from the reproposed VWA multiple-choice dataset constructed on shopping webpages.
Parameter
Value
Global Batch Size
(256, 1024)
Learning Rate
1×10−6
Weight Decay
1×10−2
KL Penalty Coefficient ( λKL )
1×10−2
Max Steps
100
Number of Rollouts
4
Table A1 : Key global hyperparameters used throughout our experiments. Larger batch sizes are adopted when GPU memory permits.
Dataset
Prompt Template
DocVQA & InfoVQA
Backward Query Generation System Message: You are a question generator. Given an observation of a document (infographic) and the correct Answer, produce a single, concise, unambiguous question whose answer is exactly the given Answer and should be asking about the information provided by the observation; when grounded ONLY on the observation provided. Rules: (1) Return ONLY the question text. (2) Avoid ambiguous wording; ensure a 1-to-1 mapping, this means the query should not have answers to it other than the correct Answer provided to you. (3) No extra commentary, quotes, or prefixes. (4) Make sure that the question is asking about the observed document (infographic), don’t propose random answers to fit the answer. User Message: Base rule: You need to generate a question. Here is the structure of the examples: few shot examples: Example i OBS: [TEXT_OBS] or [IMAGE_OBS] Answer: [ANS] Given the observation of the document (infographic), I will come up with a document (infographic) facts based Query that can be answered by the given answer. Query is: [QUESTION] The remaining examples follow the same structure. Visual focus: In this task, the OBS is an image version of the document (infographic). You must generate the question based solely on visible information in the image, without assuming or inferring any unseen text or external knowledge. Textual focus: In this task, the OBS is ocr and captioning of an document (infographic) image. OBS: [TEXT_OBS] or [IMAGE_OBS] Answer: [ANS] Given the observation of the document (infographic), I will come up with a document (infographic) facts based Query that can be answered by the given answer. Query is: Assistant Message: (Generated response)
Cycle Verification User Message: You are a helpful assistant for document VQA. Answer with the exact final answer only. OBS: [TEXT_OBS] or [IMAGE_OBS] Query: [QUESTION] Answer concisely with only the final answer. Assistant Message: (Generated response)
Table A2 : Prompt templates for each dataset and prompt type. The Backward Query Generation prompt infers plausible multimodal queries conditioned on each candidate answer; and the Cycle Verification prompt assesses whether forward inference from these queries reconstructs the original answer, providing the cycle-consistency supervision signal.
Dataset
Prompt Template
ChartQA
Backward Query Generation Image-specific part: [IMAGE_OBS] You are writing ONE short, clear, self-contained question about a chart image such that the answer equals a GIVEN TARGET ANSWER. Text-specific part: You are writing ONE short, clear, self-contained question about a chart based ONLY on the following caption text. The question must have the GIVEN TARGET ANSWER. [TEXT_OBS] : Shared rules and ending: Rules: - Must be answerable using chart visual information only. - Must not reveal or restate the answer. - Avoid yes/no, multiple choice, multi-part questions. - Include necessary qualifiers (series, category, unit, timestamp). - Length: 1–2 sentences, no explanation. Target Answer: [ANSWER] . Write the question now. Return only the question string.
Cycle Verification Image-based QA Prompt: [IMAGE_OBS] You are given a chart image and a question. Use ONLY the information that is explicitly visible in the chart (titles, labels, legends, tick marks, data labels, notes). Question: [QUESTION] Answer concisely with plain or numeric text only (no reasoning, no steps, no formatting). Text-based QA Prompt: You are given a chart description in JSON extracted from an image. Answer the user’s question using ONLY the information contained in the JSON. JSON: [TEXT_OBS] Question: [QUESTION] Answer concisely with plain or numeric text only (no reasoning, no steps, no formatting).
Table A3 : Prompt templates for each dataset and prompt type. The Backward Query Generation prompt infers plausible multimodal queries conditioned on each candidate answer; and the Cycle Verification prompt assesses whether forward inference from these queries reconstructs the original answer, providing the cycle-consistency supervision signal.
Reinforcement Learning with Verifiable Rewards (RLVR) improves multimodal reasoning by rewarding verifiable final answers. Yet answer-correct trajectories may still rely on incomplete derivations, weak evidence, or statements that contradict their conclusions. This gap between answer correctness and reasoning validity, which we call reasoning-answer inconsistency, motivates trajectory supervision in multimodal RL. We compare two main approaches: reward models (RMs), and Generative Rewards (GRs). RMs are efficient and help early in training, but their gains weaken as the policy distribution shifts; GRs improve performance, but may give unstable rewards and computationally expensive. We therefore propose Groupwise Ranking Reward, which ranks verifier-passed trajectories for the same prompt in one pass and redistributes reward accordingly. Groupwise comparison better separates stronger and weaker correct trajectories with lower judge overhead than GRs. Experiments show that RLVR aggravates reasoning-answer inconsistency, while trajectory supervision alleviates it. Groupwise Ranking Reward performs best overall, improving reliability-conditioned accuracy from 47.4% to 54.7% over RLVR.
Inductive biases steer learning toward generalizable solutions by encoding task structure. In this work, we identify a crucial missing bias in MLLMs: cross-view consistency, i.e., semantically invariant views of the same instance should lead to the same answer. Standard reinforcement learning with verifiable rewards (RLVR) objectives do not impose this constraint, but instead assign pointwise rewards to each visual input. Even with data augmentation (DA), transformed views are typically rewarded independently, providing little signal once within-view rewards saturate. We propose ConsistRoll, a simple but effective method that injects cross-view consistency into RLVR training by reusing the group-sampling mechanism of GRPO. Specifically, ConsistRoll places original and semantically invariant transformed views in the same generation group, and assigns a joint reward only when paired completions are both correct and consistent. In this way, ConsistRoll turns consistency into an online credit-assignment signal, without extra generation overhead and annotations. Theoretically, we show that cross-view consistency is a valid inductive bias, and ConsistRoll introduces a cross-view correction term absent from DA, penalizing view dependence and alleviating advantage collapse. Comprehensive benchmarks across math, general-purpose, hallucination domains confirm that ConsistRoll achieves robust improvements in multimodal reasoning.
Reinforcement learning with verifiable rewards (RLVR) has substantially enhanced the reasoning capabilities of multimodal large language models (MLLMs). However, existing RLVR approaches typically rely on outcome-driven optimization that updates both perception and reasoning using a shared reward based solely on the final answer. This shared reward blurs credit assignment, frequently improving reasoning patterns while failing to reliably enhance the accuracy of upstream visual evidence extraction. To address this perception bottleneck, we introduce PRCO (Perception-Reasoning Coevolution), a dual-role RLVR framework with a shared policy. PRCO consists of two cooperative roles: an Observer that generates an evidence caption tailored to the question and a Solver that predicts the final answer based on this caption. Crucially, PRCO employs role-specific reward signals: the Solver is optimized using verifiable outcome rewards on the final answer, while the Observer receives a utility reward derived from the Solver's downstream success. Extensive experiments across eight challenging multimodal reasoning benchmarks demonstrate that PRCO yields consistent improvements across model scales by over 7 points on average accuracy compared to the base model, outperforming prior open-source RL-tuned baselines.