cs.AIMar 26, 2026
SaveR-C2: Cycle-Consistent Reinforcement Learning Improves Multimodal Reasoning
Organizations: Rutgers University · Columbia University · University of Chicago
Abstract
Robust perception and reasoning require consistency across sensory modalities. Yet current multimodal models often violate this principle, yielding contradictory predictions for visual and textual representations of the same concept. Rather than masking these failures with standard voting mechanisms, which can amplify systematic biases, we show that cross-modal inconsistency provides a rich and natural signal for learning. We introduce RC2, a reinforcement learning framework that resolves internal conflicts by enforcing cross-modal cycle consistency. By requiring a model to perform backward inference, switch modalities, and reliably reconstruct the answer through forward inference, we obtain a dense, label-free reward. This cyclic constraint encourages the model to align its internal representations autonomously. Optimizing for this structure mitigates modality-specific errors and improves reasoning accuracy by up to 7.6 points. Our results suggest that advanced reasoning emerges not only from scaling data, but also from enforcing a structurally consistent understanding of the world.
Figures & tables
Figure 1: Gap in Multimodal Reasoning. Multimodal large language models (MLLMs) frequently fail the test of modal-invariance. For example, they produce conflicting answers for the same webpage when presented as a screenshot versus its raw HTML source. We introduce a cycle-consistency framework that directly targets this modality gap, leveraging the inconsistency itself as a signal to jointly improve reasoning and alignment.
Figure 2: Failure of multimodal voting. Left: Consistent Conflict — both text and image modalities produce self-consistent predictions ( mode-stable ) but disagree with each other, and only one modality aligns with the ground truth. Right: Unstable Recovery — within a single modality, some rollouts yield the correct answer, but the majority vote remains wrong, reflecting intra-modal instability. Using multimodal voting can amplify biases or lose correct signals.
Figure 3: Overview of multimodal cycle consistency. Starting from a potential answer candidate , the model performs backward inference to reconstruct two latent queries, from the text view and from the image view . Each reconstructed query is then used for forward inference across both modalities, resulting in four reconstructed answers generated via the paths , , , and . Cycle consistency is measured by whether the reconstructed answers remain consistent with the original , forming a full 4-way cross-modal reasoning cycle.
Figure 4: Examples of the backward-inference (Answer Query) step . Given a Candidate Answer, the model generates distinct, semantically-grounded queries for both the text and image modalities. This demonstrates the viability of the first step of our cycle-consistency reward, enabling the model to check its answer in the alternate modality.
| Qwen2.5-VL-3B-Instruct | ||||||||
| Dataset | Base Model | + Voting (Text) | + Voting (Image+Text) | + R-C 2 (Ours) | ||||
| Text Acc | Vision Acc | Text Acc | Vision Acc | Text Acc | Vision Acc | Text Acc | Vision Acc | |
| ScienceQA | 68.9 | 76.0 | 70.7 (+1.8) | 76.2 (+0.2) | 73.1 (+4.2) | 78.0 (+2.0) | 76.7 (+7.8) | 83.3 (+7.3) |
| ChartQA | 71.1 | 82.8 | 76.0 (+4.9) | 83.6 (+0.8) | 76.2 (+5.1) | 83.5 (+0.7) | 77.2 (+6.1) | 84.8 (+2.0) |
| MathVista | 49.8 | 64.8 | 50.6 (+0.8) | 65.0 (+0.2) | 52.1 (+2.3) | 65.7 (+0.9) | 55.8 (+6.0) | 67.6 (+2.8) |
| VWA | 69.0 | 62.9 | 74.5 (+5.5) | 63.1 (+0.2) | 73.3 (+4.3) | 64.5 (+1.6) | 74.5 (+5.5) | 67.1 (+4.2) |
Table 1: Multimodal Reasoning Accuracy. We compare the Base Model, Voting (Text-only), Voting (Image+Text), and our R-C 2 -RL method on Qwen2.5-VL-3B-Instruct (top) and Qwen3-VL-8B-Instruct (bottom). Each cell reports Text Accuracy (input content in text format) and Vision Accuracy (input content in image format), with green numbers indicating absolute improvements of R-C 2 over the base model. We bold the best accuracy for each dataset. Multimodal voting consistently outperforms text-only voting and the base model, and R-C 2 achieves the largest gains across nearly all benchmarks.
| Dataset | Qwen2.5-VL-3B | Qwen3-VL-8B | ||||||
| Base | Voting (Text) | Voting (Image+Text) | R-C 2 (Ours) | Base | Voting (Text) | Voting (Image+Text) | R-C 2 (Ours) | |
| ScienceQA | 74.9 | 79.7 (+4.8) | 81.7 (+6.8) | 84.9 (+10.0) | 74.7 | 75.3 (+0.6) | 77.3 (+2.6) | 78.3 (+3.6) |
| ChartQA | 73.4 | 79.2 (+5.8) | 78.9 (+5.5) | 79.5 (+6.1) | 81.4 | 82.7 (+1.3) | 83.2 (+1.8) | 83.5 (+2.1) |
| MathVista | 57.3 | 58.8 (+1.5) | 59.2 (+1.9) | 61.4 (+4.1) | 64.7 | 64.8 (+0.1) | 69.0 (+4.3) | 69.2 (+4.5) |
| VWA | 63.9 | 66.9 (+3.0) | 70.0 (+6.1) | 67.1 (+3.2) | 79.2 | 80.0 (+0.8) | 79.8 (+0.6) | 81.2 (+2.0) |
| A-OKVQA | 67.8 | 80.1 (+12.3) | 80.2 (+12.4) | 80.3 (+12.5) | 70.2 | 79.3 (+9.1) | 80.3 (+10.1) | 79.6 (+9.4) |
Table 2: Multimodal Consistency Ratio. We compare both Qwen2.5-VL-3B and Qwen3-VL-8B. Each dataset is evaluated under four supervision settings: Base, +Voting (Text), +Voting (Image+Text), and + R-C 2 . Values in (green) indicate absolute improvements of R-C 2 over the base model. Across most datasets, R-C 2 achieves the highest cross-modal agreement.
| Cycle Type | ScienceQA | ChartQA | ||||
| Text | Vision | Cons. | Text | Vision | Cons. | |
| Single | 74.0 | 81.7 | 81.2 | 76.2 | 83.7 | 77.3 |
| Cross | 75.8 | 80.1 | 83.1 | 76.5 | 83.8 | 78.4 |
| Mixed | 76.7 | 83.3 | 84.9 | 77.2 | 84.8 | 79.5 |
Table 3: Ablation of cycle path configurations in R-C 2 on ScienceQA and ChartQA using Qwen2.5-VL-3B-Instruct. Single uses intra-modal cycles (I→I, T→T); Cross uses cross-modal cycles (I→T, T→I); Mixed combines all four paths. The mixed configuration yields the highest accuracy and consistency.
Figure 5: Visual comparison among the base model, voting baseline, and R-C 2 (ours). For each example, we display the original multimodal observation, the query, and the predictions from text and image modalities under different methods. The base model and voting baseline often yield either conflicting answers across modalities or spurious agreements on incorrect predictions, reflecting the modality gap and unreliable consensus confidence. In contrast, R-C 2 enforces cycle-based alignment, producing answers that are both correct and consistent across modalities.
Figure 6: Importance of cross modal in cycle consistency path. We show ScienceQA results under R-C 2 training with nine different forward–backward cycle paths. Backward denotes answer → query and forward denotes query → answer The heatmaps show (a) average accuracy and (b) cross-modal consistency across all modality configurations. Using both image and text modalities for backward query generation consistently leads to higher accuracy and stronger cross-modal agreement, suggesting that multimodal grounding provides a more reliable signal for cycle-consistent reasoning.
| Source of Candidate | A-OKVQA | VWA | ||||
| Vision Acc | Text Acc | Cons. | Vision Acc | Text Acc | Cons. | |
| Base model | 86.1 | 70.3 | 67.8 | 69.0 | 62.9 | 63.9 |
| Self-generated answer | 87.3 | 71.6 | 77.6 | 74.9 | 65.3 | 68.2 |
| Training set answer | 88.8 | 75.2 | 80.3 | 74.5 | 67.1 | 67.1 |
Table 4: Effect of candidate answer source for initiating the cycle . We compare using the model’s own predicted answer ( Self-generated ) versus using the reference answer from the dataset ( Training-set answer ). Results show comparable or better performance without requiring annotated QA pairs, highlighting the self-bootstrapping capability of R-C 2 .
Figure 7: Effect of cross-modal inconsistency ratio on ScienceQA performance. We fine-tune Qwen2.5-VL-3B-Instruct on ScienceQA with varying fractions of cross-modal inconsistent training samples. As the inconsistency ratio increases from 0% to 50%, both answer accuracy and cross-modal consistency systematically improve. This suggests that exposing the model to more modality conflicts provides a useful structural signal for the RL objective, enabling it to better resolve cross-modal disagreements and yielding more stable multimodal reasoning.
Figure 8: Examples from the reproposed VWA multiple-choice dataset constructed on shopping webpages. For each webpage, we generate 10 questions with 6 answer choices, among which 1 is correct. The questions cover diverse reasoning types, including product category recognition, price comparison, customer rating analysis, and attribute identification.
Figure 9: Examples from the reproposed VWA multiple-choice dataset constructed on shopping webpages.
| Parameter | Value |
| Global Batch Size | (256, 1024) |
| Learning Rate | |
| Weight Decay | |
| KL Penalty Coefficient ( ) | |
| Max Steps | 100 |
| Number of Rollouts | 4 |
Table A1 : Key global hyperparameters used throughout our experiments. Larger batch sizes are adopted when GPU memory permits.
| Dataset | Prompt Template |
| DocVQA & InfoVQA | Backward Query Generation System Message: You are a question generator. Given an observation of a document (infographic) and the correct Answer, produce a single, concise, unambiguous question whose answer is exactly the given Answer and should be asking about the information provided by the observation; when grounded ONLY on the observation provided. Rules: (1) Return ONLY the question text. (2) Avoid ambiguous wording; ensure a 1-to-1 mapping, this means the query should not have answers to it other than the correct Answer provided to you. (3) No extra commentary, quotes, or prefixes. (4) Make sure that the question is asking about the observed document (infographic), don’t propose random answers to fit the answer. User Message: Base rule: You need to generate a question. Here is the structure of the examples: few shot examples: Example i OBS: [TEXT_OBS] or [IMAGE_OBS] Answer: [ANS] Given the observation of the document (infographic), I will come up with a document (infographic) facts based Query that can be answered by the given answer. Query is: [QUESTION] The remaining examples follow the same structure. Visual focus: In this task, the OBS is an image version of the document (infographic). You must generate the question based solely on visible information in the image, without assuming or inferring any unseen text or external knowledge. Textual focus: In this task, the OBS is ocr and captioning of an document (infographic) image. OBS: [TEXT_OBS] or [IMAGE_OBS] Answer: [ANS] Given the observation of the document (infographic), I will come up with a document (infographic) facts based Query that can be answered by the given answer. Query is: Assistant Message: (Generated response) |
| Cycle Verification User Message: You are a helpful assistant for document VQA. Answer with the exact final answer only. OBS: [TEXT_OBS] or [IMAGE_OBS] Query: [QUESTION] Answer concisely with only the final answer. Assistant Message: (Generated response) |
Table A2 : Prompt templates for each dataset and prompt type. The Backward Query Generation prompt infers plausible multimodal queries conditioned on each candidate answer; and the Cycle Verification prompt assesses whether forward inference from these queries reconstructs the original answer, providing the cycle-consistency supervision signal.
| Dataset | Prompt Template |
| ChartQA | Backward Query Generation Image-specific part: [IMAGE_OBS] You are writing ONE short, clear, self-contained question about a chart image such that the answer equals a GIVEN TARGET ANSWER. Text-specific part: You are writing ONE short, clear, self-contained question about a chart based ONLY on the following caption text. The question must have the GIVEN TARGET ANSWER. [TEXT_OBS] : Shared rules and ending: Rules: - Must be answerable using chart visual information only. - Must not reveal or restate the answer. - Avoid yes/no, multiple choice, multi-part questions. - Include necessary qualifiers (series, category, unit, timestamp). - Length: 1–2 sentences, no explanation. Target Answer: [ANSWER] . Write the question now. Return only the question string. |
| Cycle Verification Image-based QA Prompt: [IMAGE_OBS] You are given a chart image and a question. Use ONLY the information that is explicitly visible in the chart (titles, labels, legends, tick marks, data labels, notes). Question: [QUESTION] Answer concisely with plain or numeric text only (no reasoning, no steps, no formatting). Text-based QA Prompt: You are given a chart description in JSON extracted from an image. Answer the user’s question using ONLY the information contained in the JSON. JSON: [TEXT_OBS] Question: [QUESTION] Answer concisely with plain or numeric text only (no reasoning, no steps, no formatting). |
Table A3 : Prompt templates for each dataset and prompt type. The Backward Query Generation prompt infers plausible multimodal queries conditioned on each candidate answer; and the Cycle Verification prompt assesses whether forward inference from these queries reconstructs the original answer, providing the cycle-consistency supervision signal.
More in this topic
Multimodal Reasoning
Multimodal reasoning studies how AI systems combine text with images, audio, video, and other signals to solve evidence-dependent tasks.
See all 508 papers