Where Do Long Captions Fail? Position-Aware Diagnosis and Reinforcement Learning for Detailed Image Captioning

Yuanze Hu; Zhichao Yang; Junwei Jing; Xin Yang; Wenxuan Liu; Tinghai Zhang; Jia Fu; Hongzhi Zhang

cs.CVOct 22, 2026

Where Do Long Captions Fail? Position-Aware Diagnosis and Reinforcement Learning for Detailed Image Captioning

Authors: Yuanze Hu, Zhichao Yang, Junwei Jing, Xin Yang, Wenxuan Liu, Tinghai Zhang, Jia Fu, Hongzhi Zhang,

Organizations: Beihang University · Kuaishou Technology · Fudan University · Zhejiang University

Abstract

Detailed image captioning is usually framed as an amount problem: a model should mention more visual facts while avoiding hallucination. However, autoregressive VLMs generate captions as ordered sequences, and later generations may be produced under weaker or noisier visual grounding. We therefore argue that long-form captioning should be studied not only by what facts are mentioned, but also by where they appear. We introduce a position-aware rubric representation that decomposes dense human references into atomic visual claims, verifies generated captions against those claims, and induces a quartile-wise composition of supported, hallucinated, and generic content. This lens reveals two qualitatively different failures in long captions: models tend to produce unsupported specificity in the middle, while the final portion often escapes into vague, non-evidential language rather than simply accumulating more hallucination. A same-reference human control does not exhibit these patterns; additional human audits examine the visual validity of the induced labels. Motivated by this diagnosis, we propose PAGER (Position-Aware Grounded Evidence Reinforcement), a GRPO training objective with separate signals for supported fine-grained middle evidence, evidence-bearing tail continuation, claim coverage, and content-aware length control. The result is a closed loop: the same ordered claim representation identifies where long captions fail and supplies the position-specific reward needed to repair those failures. Experiments on detailed-caption benchmarks show that PAGER improves position-wise grounding and pairwise caption preference while preserving competitive global caption quality.

Figures & tables

Figure 1: Position-dependent failures in detailed captioning. Top: hallucination rate

H_{q}

and generic rate

G_{q}

across caption quartiles. Six colored lines are evaluated captioners; the dashed human line is a same-reference control. Statistics use rubric-dimension weights. Shaded regions show 95% image-level paired bootstrap CIs. Bottom: forest plots for

D_{\mathrm{mid}}

and

D_{\mathrm{tail}}

. Every model has positive effects with CIs excluding zero; human references do not.

Metric	Generated captions, mean of 6 models	Human references
$S_{Q_{1}}$	0.527	0.933
$S_{Q_{4}}$	0.333	0.945
$G_{Q_{1}}$	0.103	0.066
$G_{Q_{4}}$	0.281	0.053
$D_{\mathrm{mid}}$ (CI95)	$+0.101$	$-0.001$ $[-0.003,+0.000]$
$D_{\mathrm{tail}}$ (CI95)	$+0.178$	$-0.013$ $[-0.028,+0.004]$

Table 1: Human-reference control. Applying the same pipeline to dense human references yields near-zero mid-caption hallucination and no tail generic escape. This is a same-reference consistency check rather than an independent completeness test.

Requirement	Diagnostic reason	Reward implication
Sub-caption granularity	Global scores hide where the composition changes.	Reward must operate on claims and sentence positions.
Mid-caption specificity	$H_{Q_{2}}/H_{Q_{3}}$ rises when models add fine details.	Reward supported fine-grained evidence in the middle, not arbitrary specificity.
Tail evidence density	$G_{Q_{4}}$ rises even when $H_{Q_{4}}$ does not.	Penalize non-evidential tail sentences without rewarding fabricated detail.
Length control	Coverage rewards can be hacked by verbosity.	Budget length according to image/reference complexity.

Table 2: The diagnostic findings translate into concrete constraints on reward design.

Figure 2: PAGER training pipeline. Reference-derived rubrics and generated captions provide claim judgments and sentence positions. Four reward terms guide GRPO; the tail term uses a separate sentence-level genericity judge. DeepSeek and human audits are used only for evaluation. At inference, the policy takes an image without a reference or rubric.

Method	$D_{\mathrm{mid}}\downarrow$	$S_{Q_{4}}\uparrow$	$H_{Q_{4}}\downarrow$	$G_{Q_{4}}\downarrow$	$\Delta S\uparrow$	$\Delta H\downarrow$	$D_{\mathrm{tail}}\downarrow$
Base	+.096	.290	.298	.412	-.241	-.028	+.269
SFT	+.091	.296	.267	.437	-.203	-.053	+.256
CapRL	+.125	.395	.283	.322	-.151	+.028	+.123
PAGER	+.036	.436	.235	.329	-.094	-.083	+.177

Table 3: Unweighted position metrics on the mixed IIW-400 + DOCCI diagnostic subset (

n=198

\Delta S=S_{Q_{4}}-S_{Q_{1}}

\Delta H=H_{Q_{4}}-H_{Q_{1}}

, and

D_{\mathrm{tail}}=G_{Q_{4}}-G_{Q_{1}}

. Best/second-best values are bold/underlined. Methods are defined in the setup.

Dataset	Method	CAPTURE $\uparrow$	DCScore-F1 $\uparrow$	Facts/100tok $\uparrow$	Length (tok)	$\mathrm{CapArena}_{4}\uparrow$
IIW-400	Base	.638	.507	5.46	257	4.32%
	SFT	.622	.402	6.27	131	1.03%
	CapRL	.640	.552	3.20	475	8.25%
	PAGER	.642	.526	3.43	412	96.28%
DOCCI	Base	.593	.482	5.32	229	5.93%
	SFT	.601	.398	6.27	128	2.06%

Table 4: Global benchmark summaries. CAPTURE and DCScore-F1 are quality metrics; facts per 100 tokens and mean token length describe density and verbosity.

\mathrm{CapArena}_{4}

excludes ties: each baseline faces PAGER , whose row pools those three comparisons. Scores are descriptive; no significance of small quality-score differences is claimed.

Variant	$H_{Q_{2}}\downarrow$	$H_{Q_{3}}\downarrow$	$S_{Q_{4}}\uparrow$	$H_{Q_{4}}\downarrow$	$G_{Q_{4}}\downarrow$	$D_{\mathrm{mid}}\downarrow$	$\Delta S\uparrow$	$\Delta H\downarrow$
PAGER full	.364	.344	.436	.235	.329	+.036	-.094	-.083
w/o $R_{\mathrm{mid}}$	.421	.408	.418	.241	.341	+.103	-.112	-.071
w/o $R_{\mathrm{tail}}$	.371	.350	.374	.248	.378	+.039	-.156	-.074
w/o $R_{\mathrm{len}}$	.378	.356	.408	.252	.340	+.047	-.118	-.068
w/o filtering	.393	.375	.401	.258	.341	+.064	-.127	-.062
Holistic checklist	.388	.367	.421	.244	.335	+.058	-.103	-.076

Table 5: Component ablations under the position-aware diagnostic. Holistic checklist uses a caption-level score; monotone weight replaces separate middle/tail rewards with increasing late-position weight. Removing each reward term tests its contribution to the target region.

Appendix figures & tables18 assets

Supplementary material from the paper’s appendix.

Appendix

Model	$Q_{1}$ S/H/G	$Q_{2}$ S/H/G	$Q_{3}$ S/H/G	$Q_{4}$ S/H/G
Qwen-Plus	.483/.444/.073	.447/.520/.033	.418/.517/.065	.372/.417/.211
Qwen3-VL-8B	.521/.377/.102	.481/.463/.055	.488/.420/.091	.365/.411/.224
Qwen3-VL-4B	.542/.364/.094	.494/.453/.053	.495/.449/.056	.330/.346/.324
InternVL3.5-8B	.547/.339/.114	.515/.403/.082	.480/.406/.114	.413/.383/.204
InternVL3.5-4B	.573/.330/.097	.480/.446/.075	.508/.403/.089	.369/.424/.207
LLaVA-v1.5-7B	.498/.364/.138	.302/.551/.147	.212/.610/.178	.146/.337/.517

Appendix

Table 6: Weighted per-model rubric-based position-wise composition on the mixed IIW-400 + DOCCI set (

n=198

). S/H/G are weighted sentence-level rates. Effect sizes use image-level paired bootstrap (

B=2000

Model	$Q_{1}$ S/H/G	$Q_{2}$ S/H/G	$Q_{3}$ S/H/G	$Q_{4}$ S/H/G
InternVL3.5-4B	.562/.289/.149	.477/.407/.116	.504/.366/.131	.351/.362/.287
InternVL3.5-8B	.529/.297/.174	.498/.374/.128	.459/.370/.171	.367/.343/.291
Qwen3-VL-4B	.531/.326/.142	.496/.422/.082	.497/.421/.082	.290/.298/.412
Qwen3-VL-8B	.515/.329/.156	.479/.432/.089	.474/.389/.137	.335/.347/.318
LLaVA-v1.5-7B	.486/.336/.178	.285/.511/.204	.200/.573/.227	.120/.293/.586
Qwen-Plus	.487/.398/.115	.462/.485/.053	.426/.476/.098	.359/.349/.293

Appendix

Table 7: Unweighted position-wise diagnostic statistics on the mixed IIW-400 + DOCCI subset (

n=198

). This table reports the same diagnostic without rubric-dimension weighting, supporting that the two position effects are not artifacts of the weighting scheme.

\Delta_{\mathrm{halluc}}=H_{Q_{4}}-H_{Q_{1}}

and

\Delta_{\mathrm{supported}}=S_{Q_{1}}-S_{Q_{4}}

Source	$\kappa$ vs. human	F1-Supp.	F1-Contr.	F1-Miss.
Primary judge (Qwen3-8B)	0.89	0.92	0.87	0.91
Secondary judge (Qwen3-14B)	0.90	0.92	0.88	0.91
Independent judge (DeepSeek-V3.2)	0.92	0.94	0.90	0.93
Inter-annotator ceiling	0.94	0.96	0.92	0.95

Appendix

Table 8: Verifier reliability against human annotations. All LLM judges are within

0.05

of the human inter-annotator ceiling in Cohen’s

\kappa

; the contradicted class remains the hardest.

Setting	Value
Base model	Qwen3-VL-4B-Instruct
Training data	filtered PixMo-derived long-caption parquet
Training validation file	DCI densely-captioned-images test parquet
Prompt / answer / image keys	problem / answer / images
Prompt template	caption chat template
Shuffle / seed	true / 1

Appendix

Table 9: Main GRPO training hyperparameters.

Setting	Value
Advantage estimator	GRPO
KL loss / penalty	enabled; low-variance KL
KL coefficient	$1.0\times 10^{-2}$
Optimizer	AdamW
Learning rate	$1.0\times 10^{-6}$
Weight decay	$1.0\times 10^{-2}$

Appendix

Table 10: Optimization and systems configuration for the main GRPO run.

Parameter	Value	Role
$\alpha$	1.0	Reward for a supported checklist item in $R_{\mathrm{cov}}$
$\beta$	0.5	Penalty weight for a contradicted checklist item in $R_{\mathrm{cov}}$
$\lambda_{\mathrm{mid}}$	0.3	Weight for supported fine-grained evidence in $Q_{2}/Q_{3}$
$\lambda_{\mathrm{tail}}$	0.2	Penalty weight for generic final-quartile sentences
$L_{0}$	512	Base caption length budget
$c$	100	Additional length budget per rubric item

Appendix

Table 11: Reward hyperparameters for the main PAGER run. The reward is

R_{\textsc{PAGER}}=R_{\mathrm{cov}}+R_{\mathrm{mid}}+R_{\mathrm{tail}}+R_{\mathrm{len}}

Asset	Role in this paper	License / terms identified	Use in this work
Qwen3-VL-4B-Instruct [ 11 ]	Base captioning model and main GRPO initialization	Apache-2.0	Used for research training and evaluation; no third-party weights are redistributed by this paper.
Qwen3-VL-8B / Qwen3-8B [ 11 ]	Evaluated captioner and checklist / reward judge	Apache-2.0	Used for deterministic judging and evaluation; no third-party weights are redistributed by this paper.
InternVL3.5-8B / InternVL3.5-4B [ 12 ]	Evaluated captioners in the diagnostic study	Apache-2.0	Used for research evaluation; no third-party weights are redistributed by this paper.
LLaVA-v1.5-7B [ 13 ]	Evaluated captioner in the diagnostic study	Apache-2.0 for the project; use is also subject to the licenses and terms of its underlying datasets and base models	Used for research evaluation only; no LLaVA weights, datasets, or checkpoints are redistributed.
DeepSeek-V3.2 [ 34 ]	Independent verifier in the reliability study	MIT	Used for verifier reliability and independent diagnostic comparison; no model weights are redistributed.
PixMo-Cap / PixMo-derived captions [ 17 ]	Source of long-caption training data for the final GRPO run	ODC-BY-1.0; intended for research and educational use under Ai2 Responsible Use Guidelines; generated text may also be subject to provider terms	Used to construct filtered training prompts/rubrics; derived training data are not redistributed unless allowed by the upstream terms.

Appendix

Table 12: Existing assets used in this work and their licenses or terms of use. “No redistribution” means that the paper reports aggregate results or uses the resource internally, but does not repackage or release the third-party asset.

Figure 3: Case-study image aar_test_04938 . The image shows an old Cruis’n World arcade racing cabinet standing next to a black change machine against a horizontal wooden plank wall.

Method	Supported evidence	Hallucinated / unsupported detail	Generic or non-evidential content
Qwen3-VL-4B-Instruct	Correctly identifies a Cruis’n World racing cabinet, active racing screen, steering wheel, change machine, wooden wall, and concrete-like floor.	Introduces a “dragon” in the marquee and misreads the change machine as accepting “$1.50”.	Ends with broad nostalgic framing such as “ready for play” and “retro atmosphere”.
CapRL-Qwen3VL-4B	Covers the arcade cabinet, screen, steering wheel, coin area, foot pedal, change machine, wooden wall, and left-edge red lights.	Misreads the title as Crashin’ World and speculates that the game is a version of Crash Bandicoot .	Uses verbose sectioning and late explanatory language that is not directly visual evidence.
PAGER	Correctly identifies the Cruis’n World cabinet, screen text, control layout, coin hardware, change machine, wooden wall, floor, and left-edge red LEDs.	Minor risk remains in over-specific decorative marquee interpretation, but the caption avoids the major title error and unrelated franchise speculation.	Contains a short meta-style ending, but most tail content remains grounded in visible layout, text, materials, and colors.

Appendix

Table 13: Summary of local caption behavior for Figure 3 . The base model is mostly correct at the global level but ends with generic interpretation. CapRL is long and structured, but introduces a salient title error and speculative game identification. PAGER keeps more of the caption anchored to visible text, hardware, layout, and environmental evidence.

Audit	Outcome	Count	Share
Supported claims	Confirmed true	142/150	94.7%
( PAGER )	Not confirmed / ambiguous	8/150	5.3%
Contradicted sentences	Genuine image conflict	139/150	92.7%
(six models)	Correct but conflicts with reference	3/150	2.0%
	Ambiguous / partly occluded	8/150	5.3%

Appendix

Table 14: Image-based audits of supported claims from PAGER and contradicted sentences from the six diagnosed models. Results are descriptive.

Method	Votes	Share	Per-rater share range
PAGER	389	77.8%	72–84%
CapRL	91	18.2%	14–22%
SFT	20	4.0%	2–6%

Appendix

Table 15: Blind preference study with five unpaid volunteers: 100 images and 500 first-place votes.

Category	Base (%)	PAGER (%)	Difference [95% CI]
Non-evidential filler	88	81	$-7.0\;[-17.4,3.6]$
Correct, absent from reference	8	14	$+6.0\;[-2.8,15.1]$
Visually incorrect	4	5	$+1.0\;[-5.2,7.3]$
Inter-annotator $\kappa$	.84	.85	—

Appendix

Table 16: Post-RL residual-label percentages and differences (percentage points), with 95% bootstrap intervals. The comparison is descriptive.

Metric	Base	PAGER
Distinct-1	.71	.68
Distinct-2	.58	.55
Self-BLEU-4	.19	.24

Appendix

Table 17: Tail lexical diversity with two sentences per caption.

Method	$D_{\mathrm{mid}}$	$S_{Q_{4}}$	$H_{Q_{4}}$	$G_{Q_{4}}$
Base	.096 / .089	.290 / .301	.298 / .289	.412 / .410
CapRL	.125 / .118	.395 / .388	.283 / .276	.322 / .336
PAGER	.036 / .041	.436 / .429	.235 / .241	.329 / .330

Appendix

Table 18: Position metrics under Qwen3-8B / DeepSeek-V3.2 on the same mixed IIW-400 + DOCCI diagnostic subset (

n=198

Model / summary	$D_{\mathrm{tail}}$	Reported 95% CI
Six-model mean (original)	.178	—
Qwen-Plus (original)	.138	[.110,.169]
GPT-5.5	.132	[.101,.159]
Gemini-3.1-Pro	.124	[.091,.154]

Appendix

Table 19: Additional foundation-model tail diagnostics.

Method	Backbone	$D_{\mathrm{mid}}$	$S_{Q_{4}}$	$H_{Q_{4}}$	$G_{Q_{4}}$
Pretrained	4B	.096	.290	.298	.412
Pretrained	8B	.101	.312	.293	.395
Pretrained	32B	.112	.326	.288	.386
CapRL	4B	.125	.395	.283	.322
CapRL + PAGER	4B	.121	.401	.277	.322
PAGER (main)	4B	.036	.436	.235	.329

Appendix

Table 20: Short-budget transfer results. New trained variants use 100 steps. The two blocks contain the same rows;

\Delta S=S_{Q_{4}}-S_{Q_{1}}

and

\Delta H=H_{Q_{4}}-H_{Q_{1}}

Backbone	$D_{\mathrm{mid}}$	$S_{Q_{4}}$	$H_{Q_{4}}$
8B	.041 [.039,.043]	.457 [.423,.491]	.216 [.214,.219]
32B	.043 [.041,.045]	.467 [.455,.480]	.214 [.214,.215]

Appendix

Table 21: Three-seed, 100-step update: mean [95%

t

CI],

n=3

Stage / model	Input	Output	Frequency
Rubric extraction / Qwen3-30B-A3B-Thinking	1186	423	Once per image
Itemwise verification / Qwen3-8B	548	596	Per caption

Appendix

Table 22: Mean input/output token counts per call.

Image Captioning

Image captioning generates natural-language descriptions of visual content, with research focused on accuracy, coverage, grounding, and evaluation.

See all 96 papers

ClaimDiff-RL: Fine-Grained Caption Reinforcement Learning through Visual Claim Comparison
May 19, 2026Tianle Li, Xuyang Shen, Yan Ma +7VLM Evaluation Reward Modeling
1The Chinese University of Hong Kong · 2MiniMax
ReflectCAP: Detailed Image Captioning with Reflective Memory
Apr 14, 2026Kyungmin Min, Minbeom Kim, Kang-il Lee +2Image Captioning VLM Hallucination Mitigation
IPAI, Seoul National University · Dept. of ECE, Seoul National University · Adobe Research
DetailVerifyBench: A Benchmark for Dense Hallucination Localization in Long Image Captions
Apr 7, 2026Xinran Wang, Yuxuan Zhang, Xiao Zhang +7VLM Hallucination VLM Evaluation
Beijing University of Posts and Telecommunications, Beijing, China

May 19, 2026J/K move · Enter open · S save

ClaimDiff-RL: Fine-Grained Caption Reinforcement Learning through Visual Claim Comparison

Tianle Li, Xuyang Shen, Yan Ma, Rongxin Guo+6

1The Chinese University of Hong Kong · 2MiniMax

VLM Evaluation Reward Modeling Image Captioning Object Hallucination in VLMs RL for VLMs

Open paper AskPDF arXiv

Long-form image captioning exposes a reward granularity problem in RL: captions are judged as whole sequences, while the important errors occur at the level of individual visual claims. A good dense caption should be both faithful and informative, avoiding hallucination without omitting salient details. Yet pairwise preferences, reference-based metrics, and holistic scalar rewards compress these local errors into a single sequence-level signal, obscuring the tradeoff between factuality and coverage. We introduce ClaimDiff-RL, a framework that uses reference-conditioned atomic claim differences as the reward unit for caption RL. Given an image, an actor caption, and a reference caption, a multimodal judge enumerates visually grounded differences, verifies each difference against the image, assigns open-vocabulary error types and severity levels, and produces per-difference statistics for reward composition. This makes hallucinated claims and omitted salient facts separately measurable and tunable. Experiments show that holistic scalar rewards can reduce hallucination by increasing missing facts, while ClaimDiff-RL exposes this faithfulness and coverage tradeoff and enables more balanced operating points. On a 160-image human-labeled diagnostic benchmark, public captioning benchmarks, and VQA benchmarks, ClaimDiff-RL improves the hallucination--missing-fact balance, preserves general capability, and even surpasses Gemini-3-Pro-Preview on several fine-grained Capability dimensions such as object counting, spatial relations, and scene recognition. These results suggest that typed, verifiable claim differences are an effective reward unit for fine-grained and diagnosable caption RL.

Abstract

Image Captioning

Explore similar work

ClaimDiff-RL: Fine-Grained Caption Reinforcement Learning through Visual Claim Comparison

ReflectCAP: Detailed Image Captioning with Reflective Memory

DetailVerifyBench: A Benchmark for Dense Hallucination Localization in Long Image Captions