Visual-Language Models (VLMs) have achieved remarkable progress in image captioning, visual question answering, and visual reasoning. Yet they remain prone to vision-language misalignment, often producing overly generic or hallucinated descriptions. Existing approaches address this via instruction tuning-requiring costly, large-scale annotated datasets or via complex test-time frameworks for caption refinement. In this work, we revisit image-text alignment through the lens of cycle consistency: given an image and a caption generated by an image-to-text model, the backward mapping through a text-to-image model should reconstruct an image that closely matches the original. In our setup, a VLM serves as the image-to-text component, while a pre-trained text-to-image model closes the loop by reconstructing the image from the generated caption. Building on this, we introduce CycleCap, a fine-tuning scheme to improve image captioning using Group Relative Policy Optimization (GRPO) with a reward based on the similarity between the original and reconstructed images, computed on-the-fly. Unlike previous work that uses cycle consistency loss for preference dataset construction, our method leverages cycle consistency directly as a self-supervised training signal. This enables the use of raw images alone, eliminating the need for curated image-text datasets, while steering the VLM to produce more accurate and grounded text descriptions. Applied to four VLMs ranging from 1B to 7B parameters, CycleCap yields consistent improvements across captioning and hallucination benchmarks, surpassing state-of-the-art methods that rely on supervised cycle consistency training.
Figures & tables
Figure 1: Overview of the CycleCap framework. The Visual-Language Model generates multiple captions {yi}i=1n for an image x . Each caption is used by a frozen Image Generation Model to reconstruct an image xi′=G(yi) , whilst the similarity between x and G(yi) is measured to obtain the cycle consistency reward Ri , i=1,…,n . These rewards guide fine-tuning of the Visual–Language Model via GRPO to encourage captions that better reflect visual content in a self-supervised manner.
Figure 2: Win-rates (%) of (a) CycleCap fine-tuned models versus the corresponding baseline and (b) Qwen2-VL-7B [ 57 ] fine-tuned with different methods versus the baseline for CompreCap [ 41 ] benchmark.
Method
T2I Model
CompreCap
CAPability
CapsBench
MMHal
Baseline (Qwen2-VL-7B)
—
61.73
70.47
74.17
3.85
CyclePref [ 3 ]
SD3
62.03
70.59
74.27
3.95
RICO-Flash [ 58 ]
FLUX.1-dev
62.93
68.83
75.30
3.92
CycleCap (Ours)
SD3
63.06
72.95
76.38
4.02
CycleCap (Ours)
FLUX.1-dev
63.64
73.73
77.25
4.02
Table 2: Comparison of CycleCap with state-of-the-art cycle-consistency-based methods on CompreCap [ 41 ] , CAPability [ 40 ] , CapsBench [ 36 ] , and MMHal [ 52 ] benchmarks, using Qwen2-VL-7B [ 57 ] as baseline. For CompreCap we report here only Unified Score, the analytic table is included in the Appendix.
Metric
Baseline (Qwen2-VL-7B [ 57 ] )
CyclePref [ 3 ] (SD3)
RICO-Flash [ 58 ] (FLUX.1-dev)
CycleCap (Ours) (SD3)
CycleCap (Ours) (FLUX.1-dev)
Obj. Category
73.52
72.61
70.97
74.17
77.57
Obj. Number
60.67
60.00
57.85
65.12
65.74
Obj. Color
89.67
89.33
88.64
90.36
89.94
Spatial Relation
69.11
67.50
68.24
71.29
71.08
Scene
77.07
76.36
75.56
78.60
79.06
Camera Angle
38.00
39.24
36.90
40.20
38.42
Table 3: Comparison of CycleCap with state-of-the-art methods on the CAPability [ 40 ] benchmark, all using Qwen2-VL-7B [ 57 ] as baseline. Bold and underlined values indicate the best and second best results, respectively.
CompreCap
CAPability
CapsBench
MMHal
Method
Obj
Attr
Rel
Uni
Baseline (Qwen2-VL-7B)
77.86
2.84
2.80
61.73
70.47
74.17
3.85
RICO-Flash [ 58 ]
79.09
2.93
2.83
62.93
68.83
75.30
3.92
CycleCap (Ours)
79.32
2.91
2.86
63.06
73.73
76.38
4.02
RICO-Flash [ 58 ] + CycleCap (Ours)
80.59
2.95
2.88
63.85
73.49
77.72
4.01
Table 4: Performance of CycleCap applied on top of RICO-Flash [ 58 ] across captioning and hallucination benchmarks, with Qwen2-VL-7B [ 57 ] as the baseline.
CompreCap
CapsBench
MMHal
Method
Obj
Attr
Rel
Uni
Baseline (Qwen2-VL-2B)
73.10
2.72
2.76
59.35
69.70
3.63
CycleCap w/ LPIPS
74.21
2.68
2.80
59.70
64.05
3.49
CycleCap w/ CLIP
73.96
2.78
2.83
60.58
71.16
3.94
CycleCap w/ DreamSim
76.94
2.85
2.86
62.09
72.11
3.71
Table 5: Effect of different similarity metrics (LPIPS [ 64 ] , CLIP [ 48 ] , DreamSim [ 20 ] ) on CycleCap performance for fine-tuning the baseline Qwen2-VL-2B [ 57 ] .
CompreCap
CapsBench
MMHal
Method
Obj
Attr
Rel
Uni
Baseline (Qwen2-VL-2B)
73.10
2.72
2.76
59.35
69.70
3.63
CycleCap w/ SD3
76.94
2.85
2.86
62.09
72.11
3.71
CycleCap w/ FLUX.1-schnell
75.96
2.87
2.89
62.14
72.18
3.65
Table 6: Effect of different text-to-image models (SD3 [ 18 ] and FLUX.1-schnell [ 29 ] ) on the proposed CycleCap performance for fine-tuning baseline Qwen2-VL-2B [ 57 ] .
Figure 3: Qualitative comparison of captions generated by the baseline Qwen2-VL-7B [ 57 ] and the proposed CycleCap on samples from the CapsBench [ 36 ] dataset. Newly added information is indicated in bold . Our method produces more detailed, organized, and accurate descriptions compared to the baseline outputs.
Figure 4: Qualitative comparison of image reconstructions with captions generated by SOTA and our method (CycleCap) deployed for Qwen2-VL-7B. The visualization shows that the model fine-tuned with CycleCap captures more structural details and object attributes, leading to reconstructions closer to the original image. The generated captions are provided in the Appendix.
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Category
Parameters
Training
Batch size: 64
Learning rate: 10−5
Scheduler: Linear
Epochs: 1
LoRA
Rank r : 64
Dropout: 0.05
Target modules: all linear projection layers
Vision tower: frozen
GRPO
KL weight β : 0.04
Clip value ε : 0.02
Generations n : 8
Appendix
Table A1: CycleCap training parameters.
Figure A1: The designed prompt of the CycleCap framework used to generate image captions during training.
InternVL3-1B [ 66 ]
Qwen2-VL-2B [ 57 ]
Qwen2.5-VL-3B [ 5 ]
Qwen2-VL-7B [ 57 ]
Baseline
CycleCap
Baseline
CycleCap
Baseline
CycleCap
Baseline
CycleCap
Avg. Length
152.64
179.80
209.39
202.73
176.06
198.73
384.56
385.23
Appendix
Table A3: Average caption length (in tokens) on the CompreCap [ 41 ] benchmark.
Method
Generations n
CompreCap
CAPability
CapsBench
Baseline (Qwen2-VL-7B)
—
61.73
70.47
74.17
CycleCap
2
62.57
72.40
74.24
CycleCap
4
62.58
72.17
76.22
CycleCap
8
63.06
72.95
76.38
Appendix
Table A4: Effect of the number of GRPO generations n (captions generated per image) on CycleCap performance on captioning benchmarks CompreCap (Unified Score) [ 41 ] , CAPability [ 40 ] , and CapsBench [ 36 ] .
Figure A2: Effect of the number of GRPO generations n on CycleCap performance vs. relative training cost. We report performance on CompreCap (Unified Score) [ 41 ] , CAPability [ 40 ] , and CapsBench [ 36 ] benchmarks. Relative training cost is normalized with respect to n=2 ( 1× ).
Method
CompreCap
CAPability
CapsBench
MMHal
Baseline (Qwen2-VL-7B)
61.73
70.47
74.17
3.85
DPO
61.78
71.11
74.41
3.95
CycleCap (Ours)
62.33
71.89
74.47
4.01
Appendix
Table A5: Controlled comparison between DPO and CycleCap using the sub-sampled CyclePrefDB-I2T [ 3 ] training set with Qwen2-VL-7B [ 57 ] as the base model. Results are reported on CompreCap (Unified Score) [ 41 ] , CAPability [ 40 ] , CapsBench [ 36 ] , and MMHal [ 52 ] benchmarks.
CompreCap
CAPability
CapsBench
MMHal
Method
T2I Model
Obj
Attr
Rel
Uni
Baseline (Qwen2-VL-7B)
—
77.86
2.84
2.80
61.73
70.47
74.17
3.85
CyclePref [ 3 ]
SD3
77.29
2.85
2.85
62.03
70.59
74.27
3.95
RICO-Flash [ 58 ]
FLUX.1-dev
79.09
2.93
2.83
62.93
68.83
75.30
3.92
CycleCap (Ours)
SD3
79.32
2.91
2.86
63.06
72.95
76.38
4.02
CycleCap (Ours)
FLUX.1-dev
79.67
2.93
2.90
63.64
73.73
77.25
4.02
Appendix
Table A6: Detailed results corresponding to Tab. 2 in the main paper, comparing the proposed CycleCap with SOTA cycle-consistency-based approaches.
Figure A3: Additional qualitative comparisons between captions generated by the baseline Qwen2-VL-7B [ 57 ] model and the CycleCap fine-tuned version, for image samples from the CapsBench [ 36 ] and CompreCap [ 41 ] benchmarks. Newly added information is shown in bold , content present in the baseline but not preserved in the CycleCap caption is underlined , and hallucinations are highlighted in pink. Overall, CycleCap tends to produce more structured and grounded descriptions, while it adds details related to object attributes, spatial layout, and broader scene context.
Figure A4: Generated captions corresponding to the qualitative reconstruction comparison shown in the first row of Fig. 4 in the main paper.
Figure A5: Generated captions corresponding to the qualitative reconstruction comparison shown in the second row of Fig. 4 in the main paper.
Reinforcement Learning (RL) has demonstrated significant gains in image captioning, yet it is still limited in encouraging Large Vision-Language Models (LVLMs) to explore novel reasoning strategies. This limitation leads to a performance gap between RL and Supervised Fine-Tuning (SFT). In this paper, we argue that multi-modal retrieval can serve as an effective reasoning signal for caption refinement. Based on this insight, we present the Retrieval-Guided Refinement for Image Captioning (Re3Cap), a retrieval-guided reasoning strategy that enhances image captioning without requiring additional annotations. Instantiated by Caption Refinement Suggester (CRS) and Caption Quality Assessor (CQA), this strategy identifies hallucinations and omissions in image captions, leading to more accurate and detailed descriptions. Extensive experiments demonstrate the superiority of our method in image captioning, even compared with Supervised Fine-Tuning. Especially, Re3Cap outperforms GRPO with an average improvement of 8.64% in relation reasoning on the COCO-LN500 benchmark.
Haonan Jia, Shichao Dong, Zenghui Sun +7
Taobao & Tmall Group of Alibaba · The Hong Kong University of Science and Technology (Guangzhou) · Shanghai Artificial Intelligence Laboratory
Large Vision-Language Models (LVLMs) often omit or misrepresent critical visual content in generated image captions. Minimizing such information loss will force LVLMs to focus on image details to generate precise descriptions. However, measuring information loss during modality conversion is inherently challenging due to the modal gap between visual content and text output. In this paper, we argue that the quality of an image caption is positively correlated with the similarity between images retrieved via text search using that caption. Based on this insight, we further propose Cross-modal Identity Mapping (CIM), a reinforcement learning framework that enhances image captioning without requiring additional annotations. Specifically, the method quantitatively evaluates the information loss from two perspectives: Gallery Representation Consistency and Query-gallery Image Relevance. Supervised under these metrics, LVLM minimizes information loss and aims to achieve identity mapping from images to captions. The experimental results demonstrate the superior performance of our method in image captioning, even when compared with Supervised Fine-Tuning. Particularly, on the COCO-LN500 benchmark, CIM achieves a 20% improvement in relation reasoning on Qwen2.5-VL-7B.
Haonan Jia, Shichao Dong, Xin Dong +6
Taobao & Tmall Group of Alibaba · The University of Hong Kong
Image and video captioning are fundamental tasks that bridge the visual and linguistic domains, playing a critical role in pre-training Large Vision-Language Models (LVLMs). Current state-of-the-art captioning models are typically trained with Supervised Fine-Tuning (SFT), a paradigm that relies on expensive, non-scalable annotations and often causes models to memorize specific ground-truth answers, limiting their generality and ability to generate diverse, creative descriptions. To overcome these limitations, we propose applying Reinforcement Learning with Verifiable Rewards (RLVR) to the open-ended task of multimodal captioning. We introduce Captioning Reinforcement Learning++ (CapRL++), a novel reference-free training framework that redefines caption quality through its utility: a high-quality caption should enable a non-visual language model to accurately answer questions about the corresponding visual content. CapRL++ employs a decoupled two-stage pipeline where an LVLM generates a caption, and the objective reward is derived from the accuracy of a separate, vision-free LLM answering Multiple-Choice Questions based solely on that caption. Evaluations on more than 20 image and video benchmarks show that CapRL++ improves dense caption quality and strengthens caption-based pretraining across tasks such as spatial and temporal understanding. Pretraining on scalable image and video caption datasets annotated by CapRL++ yields substantial downstream gains. Furthermore, within the Prism Framework for caption quality evaluation, compact models trained with CapRL++ achieve dense captioning performance comparable to substantially larger models such as Qwen2.5-VL-72B and Qwen3-VL-235B-A22B. These results validate that CapRL++ effectively trains models to produce generalizable, high-fidelity descriptions, establishing a robust foundation beyond the limitations of traditional SFT.
Penghui Yang, Long Xing, Xiaoyi Dong +10
Tsinghua University · University of Science and Technology of China · Microsoft +4