Does the Model Use What It Cites? Counterfactual Tests of Grounded Reasoning Chains in Small Video-Language Models
Organizations: Independent Researchers
Abstract
A grounded reasoning chain cites a frame span and a box for every step, so a reader can check where the model looked. Whether the answer depends on the cited region is a different question. We test it by intervention: the regions a model cited in its own chain are blacked out, and the share of correct answers that flip is compared with blacking out boxes of the same size elsewhere on the same frames. The paired difference, the cite gap, needs no annotation. We apply it to 2-3B video-language models trained on exact CLEVRER chains and on pseudo-labelled NExT-QA chains. With the same weights, the cite gap is 0.089 on CLEVRER and 0.003 on NExT-QA. Pooled over models, masking the cited boxes flips 2.2 times as many correct answers as the control on CLEVRER and 1.1 times as many on NExT-QA, also among the answers that depend on the video at all, while the boxes pass a grounding check on both. Masking the cited entity on every frame, instead of the cited box, flips 1.9 times as many answers as its control. The NExT-QA chains were supervised to cite two frames of entities that stay visible on most frames, so this outcome is what the supervision predicts; the point is that the usual check on grounded chains cannot see it. A citation can be verifiable and relevant without the answer depending on it, and a test of grounded reasoning should remove what a chain names, not only where it points.
Figures & tables
| Model | Acc. | Blind gap | [email protected] | Flip (cited) | Flip (random) | Cite gap | Gold-evidence gap | |
|---|---|---|---|---|---|---|---|---|
| CLEVRER chain GRPO | 0.871 | 0.37 | 0.78 | 2527 | 0.088 | 0.036 | 0.051 [0.039,0.064] | 0.076 [0.060,0.092] |
| Mixed chain SFT | 0.739 | 0.27 | 0.78 | 2512 | 0.165 | 0.077 | 0.089 [0.071,0.106] | 0.094 [0.074,0.113] |
| Mixed chain GRPO | 0.771 | 0.29 | 0.77 | 2520 | 0.078 | 0.036 | 0.042 [0.030,0.055] | 0.067 [0.052,0.082] |
| NExT-QA chain GRPO | 0.535 | 0.08 | 0.15 | 2533 | 0.069 | 0.038 | 0.030 [0.018,0.042] | 0.062 [0.043,0.080] |
| InternVL3-2B chain GRPO | 0.803 | 0.33 | 0.76 | 2532 | 0.084 | 0.037 | 0.047 [0.035,0.059] | 0.077 [0.062,0.092] |
| Model | Per-opt. | Sel. | Blind gap | Flip (cited) | Flip (random) | Cite gap | Cite gap, correct opt. |
|---|---|---|---|---|---|---|---|
| Qwen2.5-VL-3B, letters prompt | – | 0.730 | 0.29 | – | – | – | – |
| Qwen2.5-VL-3B | 0.860 | 0.595 | 0.06 | – | – | – | – |
| Mixed plain SFT (chain-free) ‡ | 0.826 | 0.637 | 0.12 | 0.014 | 0.016 | 0.002 [ 0.005,0.001] | 0.002 [ 0.005,0.010] |
| Mixed plain SFT, seed 1 | 0.849 | 0.661 | 0.13 | – | – | – | – |
| CLEVRER chain GRPO | 0.682 | 0.470 | 0.12 | 0.085 | 0.070 | 0.015 [0.003,0.025] | 0.000 [ 0.019,0.019] |
| Mixed chain SFT | 0.845 | 0.637 | 0.22 | 0.028 | 0.025 | 0.003 [ 0.002,0.007] | 0.012 [ 0.001,0.026] |
| Model | Items | Flip | Ctrl. | Gap |
|---|---|---|---|---|
| Mixed chain SFT | All | 0.110 | 0.062 | 0.048 [0.038,0.059] |
| Correct opt. | 0.194 | 0.096 | 0.098 [0.069,0.127] | |
| Wrong opt. | 0.090 | 0.054 | 0.037 [0.026,0.048] | |
| NExT-QA chain GRPO | All | 0.095 | 0.049 | 0.046 [0.037,0.055] |
| Correct opt. | 0.167 | 0.058 | 0.109 [0.084,0.134] | |
| Wrong opt. | 0.078 | 0.046 | 0.031 [0.023,0.040] |
Appendix figures & tables25 assets
Supplementary material from the paper’s appendix.
Appendix
| Test | Flip [95% CI] | Ctrl. | Ratio | Gap [95% CI] | |
|---|---|---|---|---|---|
| CLEVRER, five chain models | |||||
| Self-citation (cited boxes), all items | 9403 | 0.097 [0.090,0.105] | 0.045 | 2.2 | 0.053 [0.045,0.060] |
| Self-citation (cited boxes), correct options | 4596 | 0.090 [0.079,0.102] | 0.040 | 2.2 | 0.050 [0.040,0.059] |
| Gold evidence boxes, all items | 9433 | 0.144 [0.133,0.155] | 0.068 | 2.1 | 0.076 [0.067,0.085] |
| Gold evidence boxes, correct options | 4613 | 0.147 [0.130,0.165] | 0.073 | 2.0 | 0.074 [0.061,0.087] |
| Object track (gold objects, all frames), all items | 8922 | 0.563 [0.549,0.578] | 0.067 | 8.3 | 0.495 [0.479,0.511] |
| Test | Blind flip | Flip | Ctrl. | Gap [95% CI] | Gap / blind flip | |
|---|---|---|---|---|---|---|
| CLEVRER, five chain models | ||||||
| Self-citation (cited boxes) | 0.48 | 4486 | 0.124 | 0.057 | 0.067 [0.056,0.078] | 0.11 |
| Gold evidence boxes | 0.48 | 4502 | 0.193 | 0.090 | 0.103 [0.089,0.117] | 0.16 |
| Object track (gold objects, all frames) | 0.49 | 4338 | 0.717 | 0.095 | 0.622 [0.602,0.642] | 1.02 |
| NExT-QA, five models trained with NExT-QA chains | ||||||
| Self-citation (cited boxes) | 0.29 | 5926 | 0.045 | 0.039 | 0.006 [0.001,0.011] | 0.01 |
| Model | Descr. / Causal | Expl. / Temporal | Pred. / Descr. | CF |
|---|---|---|---|---|
| CLEVRER chain GRPO | 0.035 [0.004,0.068] | 0.042 [0.019,0.067] | 0.082 [0.047,0.118] | 0.050 [0.032,0.068] |
| Mixed chain SFT | 0.019 [ 0.009,0.047] | 0.156 [0.112,0.200] | 0.085 [0.055,0.116] | 0.054 [0.030,0.079] |
| Mixed chain GRPO | 0.000 [ 0.034,0.032] | 0.050 [0.026,0.076] | 0.045 [0.018,0.073] | 0.047 [0.028,0.068] |
| NExT-QA chain GRPO | 0.129 [0.054,0.207] | 0.011 [ 0.009,0.031] | 0.027 [0.009,0.047] | 0.026 [0.002,0.048] |
| InternVL3-2B chain GRPO | 0.059 [0.019,0.100] | 0.025 [0.010,0.039] | 0.108 [0.074,0.142] | 0.031 [0.010,0.051] |
| CLEVRER chain GRPO | 0.017 [ 0.002,0.034] | 0.009 [ 0.009,0.027] | 0.019 [0.001,0.038] | – |
| Model | Acc. (q) | Per-option | “yes” rate | Selection | Blind sel. | Blind gap |
|---|---|---|---|---|---|---|
| Qwen2.5-VL-3B, letters prompt | 0.730 | – | – | – | 0.437 | 0.29 |
| + outcome-only GRPO (CLEVRER), letters prompt | 0.750 | – | – | – | 0.437 | 0.31 |
| Qwen2.5-VL-3B | 0.461 | 0.860 | 0.19 | 0.595 | 0.270 | 0.06 |
| + outcome-only GRPO (CLEVRER) | 0.355 | 0.804 | 0.15 | 0.522 | 0.260 | 0.00 |
| CLEVRER chain SFT | 0.122 | 0.622 | 0.38 | 0.451 | 0.259 | 0.12 |
| CLEVRER chain GRPO | 0.194 | 0.682 | 0.41 | 0.470 | 0.272 | 0.12 |
| Model | Acc. | Blind gap | Flip (cited) | Flip (random) | Cite gap [95% CI] | |
|---|---|---|---|---|---|---|
| CLEVRER chain GRPO | 0.682 | 0.12 | 4770 | 0.085 | 0.070 | 0.015 [0.003,0.025] |
| Mixed chain SFT | 0.845 | 0.22 | 4906 | 0.028 | 0.025 | 0.003 [ 0.002,0.007] |
| Mixed chain SFT, seed 1 | 0.846 | 0.16 | 4914 | 0.023 | 0.020 | 0.003 [ 0.002,0.007] |
| Mixed chain GRPO | 0.837 | 0.25 | 4912 | 0.022 | 0.018 | 0.004 [0.000,0.008] |
| NExT-QA chain SFT | 0.837 | 0.19 | 4891 | 0.029 | 0.025 | 0.003 [ 0.001,0.008] |
| NExT-QA chain GRPO | 0.852 | 0.15 | 4889 | 0.016 | 0.015 | 0.001 [ 0.003,0.004] |
| Model | Flip (tracks) | Flip (random) | Entity-track gap | Causal | Temporal | Descr. | Cite gap | Blind gap | |
|---|---|---|---|---|---|---|---|---|---|
| Mixed chain SFT | 4087 | 0.110 | 0.062 | 0.048 [0.038,0.059] | 0.046 | 0.036 | 0.061 | 0.003 [ 0.002,0.007] | 0.22 |
| NExT-QA chain GRPO | 4140 | 0.095 | 0.049 | 0.046 [0.037,0.055] | 0.043 | 0.031 | 0.065 | 0.001 [ 0.003,0.004] | 0.15 |
| Model | Det. | IoU | [email protected] | Steps |
|---|---|---|---|---|
| CLEVRER chain GRPO | 0.889 | 0.25 / 0.10 | 0.12 / 0.00 | 18 |
| Mixed chain SFT | 0.900 | 0.54 / 0.23 | 0.60 / 0.18 | 14274 |
| Mixed chain GRPO | 0.916 | 0.56 / 0.24 | 0.62 / 0.19 | 14395 |
| NExT-QA chain SFT | 0.893 | 0.54 / 0.22 | 0.60 / 0.17 | 14410 |
| NExT-QA chain GRPO | 0.908 | 0.55 / 0.23 | 0.62 / 0.18 | 14448 |
| Model | Flip | Ctrl. | Gap (correct option) | Flip | Ctrl. | Gap (wrong option) | ||
| NExT-QA, self-citation test | ||||||||
| CLEVRER chain GRPO | 787 | 0.062 | 0.062 | 0.000 [ 0.019,0.019] | 2449 | 0.092 | 0.073 | 0.019 [0.007,0.032] |
| Mixed chain SFT | 769 | 0.057 | 0.046 | 0.012 [ 0.001,0.026] | 3375 | 0.021 | 0.020 | 0.001 [ 0.004,0.006] |
| Mixed chain SFT, seed 1 | 776 | 0.044 | 0.041 | 0.003 [ 0.010,0.015] | 3384 | 0.018 | 0.015 | 0.003 [ 0.001,0.008] |
| Mixed chain GRPO | 817 | 0.027 | 0.017 | 0.010 [0.001,0.018] | 3292 | 0.020 | 0.018 | 0.002 [ 0.002,0.007] |
| NExT-QA chain SFT | 822 | 0.033 | 0.027 | 0.006 [ 0.005,0.017] | 3269 | 0.028 | 0.025 | 0.003 [ 0.003,0.009] |
| Model | Items | Cite gap | Entity-track gap | Items | Cite gap | Entity-track gap |
|---|---|---|---|---|---|---|
| Mixed chain SFT | 1612 | 0.011 [0.003,0.019] | 0.042 [0.025,0.059] | 3294 | 0.001 [ 0.007,0.004] | 0.051 [0.037,0.065] |
| Mixed chain SFT, seed 1 | 1446 | 0.005 [ 0.004,0.014] | – | 3468 | 0.002 [ 0.002,0.007] | – |
| Mixed chain GRPO | 1687 | 0.001 [ 0.009,0.007] | – | 3225 | 0.006 [0.002,0.011] | – |
| NExT-QA chain SFT | 1440 | 0.001 [ 0.010,0.008] | – | 3451 | 0.005 [ 0.001,0.011] | – |
| NExT-QA chain GRPO | 1689 | 0.000 [ 0.006,0.006] | 0.049 [0.034,0.065] | 3200 | 0.001 [ 0.003,0.005] | 0.045 [0.034,0.056] |
| Model | Gap, all boxes hit | Gap, a box misses | Ctrl. | Gap, ctrl. on object | Ctrl. | Gap, ctrl. on background | ||
|---|---|---|---|---|---|---|---|---|
| CLEVRER chain GRPO | 974 | 0.061 [0.042,0.079] | 1229 | 0.044 [0.026,0.062] | 0.050 | 0.044 [0.027,0.062] | 0.014 | 0.062 [0.046,0.079] |
| Mixed chain SFT | 878 | 0.077 [0.056,0.100] | 986 | 0.098 [0.070,0.126] | 0.105 | 0.083 [0.058,0.108] | 0.035 | 0.097 [0.072,0.122] |
| Mixed chain GRPO | 952 | 0.045 [0.029,0.062] | 993 | 0.039 [0.020,0.058] | 0.049 | 0.040 [0.022,0.060] | 0.022 | 0.044 [0.028,0.060] |
| NExT-QA chain GRPO | 11 | 0.000 [ 0.188,0.333] | 90 | 0.056 [0.000,0.112] | 0.058 | 0.007 [ 0.017,0.033] | 0.029 | 0.041 [0.026,0.056] |
| InternVL3-2B chain GRPO | 950 | 0.065 [0.047,0.086] | 1084 | 0.030 [0.016,0.045] | 0.048 | 0.034 [0.015,0.051] | 0.026 | 0.060 [0.044,0.077] |
| Model | Net acc., mask | Net acc., ctrl. | W C, mask | W C, ctrl. | Q. flip, mask | Q. flip, ctrl. |
|---|---|---|---|---|---|---|
| NExT-QA, self-citation test | ||||||
| Mixed chain SFT | 0.001 | 0.001 | 0.160 | 0.131 | 0.095 | 0.101 |
| Mixed chain SFT, seed 1 | 0.001 | 0.003 | 0.122 | 0.094 | 0.074 | 0.070 |
| Mixed chain GRPO | 0.001 | 0.002 | 0.106 | 0.082 | 0.073 | 0.057 |
| NExT-QA chain SFT | 0.004 | 0.003 | 0.124 | 0.113 | 0.097 | 0.112 |
| NExT-QA chain GRPO | 0.001 | 0.001 | 0.087 | 0.080 | 0.062 | 0.054 |
| Model | ||||
|---|---|---|---|---|
| Cite gap by share of the entity’s frames that are cited | 0.15–0.3 | |||
| Mixed chain SFT | 0.000 [ 0.006,0.006] | 0.001 [ 0.008,0.005] | 0.023 [0.010,0.037] | |
| Mixed chain SFT, seed 1 | 0.002 [ 0.004,0.009] | 0.003 [ 0.003,0.010] | 0.002 [ 0.013,0.007] | |
| Mixed chain GRPO | 0.001 [ 0.004,0.005] | 0.006 [ 0.001,0.015] | 0.004 [ 0.006,0.016] | |
| NExT-QA chain SFT | 0.002 [ 0.009,0.005] | 0.008 [0.001,0.016] | 0.002 [ 0.009,0.012] | |
| NExT-QA chain GRPO | 0.008 [ 0.017,0.000] | 0.004 [0.000,0.008] | 0.002 [ 0.006,0.008] |
| Model | Frames | Covered (%) | Untouched (%) | Flip (cited) [95% CI] | Cite gap, untouched | |
|---|---|---|---|---|---|---|
| NExT-QA | ||||||
| CLEVRER chain GRPO | 1.3 | 22 | 44 | 1431 | 0.085 [0.070,0.099] | 0.016 [0.001,0.032] |
| Mixed chain SFT | 2.0 | 40 | 18 | 743 | 0.020 [0.010,0.032] | 0.003 [ 0.008,0.014] |
| Mixed chain SFT, seed 1 | 2.0 | 39 | 20 | 798 | 0.019 [0.010,0.029] | 0.005 [ 0.004,0.015] |
| Mixed chain GRPO | 2.0 | 40 | 21 | 846 | 0.015 [0.007,0.025] | 0.005 [ 0.005,0.014] |
| NExT-QA chain SFT | 2.0 | 38 | 17 | 701 | 0.026 [0.015,0.038] | 0.000 [ 0.014,0.014] |
| Model | Items | Cite gap | E.-t. | E.-t., corr. | Items | Cite gap | E.-t. | E.-t., corr. |
|---|---|---|---|---|---|---|---|---|
| Mixed chain SFT | 1943 | 0.008 [0.000,0.015] | 0.052 [0.036,0.069] | 0.081 [0.045,0.120] | 1852 | 0.002 [ 0.004,0.009] | 0.046 [0.030,0.063] | 0.115 [0.063,0.171] |
| Mixed chain SFT, seed 1 | 1921 | 0.001 [ 0.006,0.007] | – | – | 1810 | 0.003 [ 0.004,0.010] | – | – |
| Mixed chain GRPO | 2231 | 0.006 [ 0.001,0.014] | – | – | 1703 | 0.001 [ 0.007,0.005] | – | – |
| NExT-QA chain SFT | 1957 | 0.005 [ 0.004,0.014] | – | – | 1867 | 0.002 [ 0.004,0.010] | – | – |
| NExT-QA chain GRPO | 2206 | 0.000 [ 0.006,0.006] | 0.038 [0.026,0.051] | 0.086 [0.047,0.127] | 1783 | 0.003 [ 0.002,0.008] | 0.049 [0.034,0.063] | 0.120 [0.079,0.162] |
| Model | Items | Flip | Ctrl. | Entity-track gap | Gap, correct options | |
|---|---|---|---|---|---|---|
| Mixed chain SFT | Control mostly on another entity | 749 | 0.132 | 0.095 | 0.037 [0.011,0.064] | 0.070 [0.007,0.141] |
| Mixed | 1297 | 0.108 | 0.056 | 0.052 [0.033,0.070] | 0.085 [0.033,0.134] | |
| Control on background | 2041 | 0.102 | 0.053 | 0.049 [0.035,0.065] | 0.116 [0.076,0.157] | |
| NExT-QA chain GRPO | Control mostly on another entity | 768 | 0.099 | 0.055 | 0.044 [0.024,0.067] | 0.111 [0.049,0.174] |
| Mixed | 1314 | 0.093 | 0.047 | 0.046 [0.030,0.061] | 0.124 [0.085,0.166] | |
| Control on background | 2058 | 0.095 | 0.047 | 0.048 [0.035,0.061] | 0.099 [0.061,0.137] |
| Model | Masked | Same box | Same fr. | Other fr. | Same ent. | Same box | Same fr. | Other fr. | Same ent. |
|---|---|---|---|---|---|---|---|---|---|
| NExT-QA | |||||||||
| CLEVRER chain GRPO | 1.4 | 0.49 | 0.33 | 0.18 | 0.41 | 0.57 | 0.27 | 0.16 | 0.59 |
| Mixed chain SFT | 3.9 | 0.14 | 0.70 | 0.16 | 0.70 | 0.52 | 0.35 | 0.13 | 0.75 |
| Mixed chain SFT, seed 1 | 3.5 | 0.16 | 0.70 | 0.14 | 0.69 | 0.53 | 0.35 | 0.11 | 0.74 |
| Mixed chain GRPO | 3.6 | 0.15 | 0.67 | 0.18 | 0.74 | 0.52 | 0.32 | 0.15 | 0.77 |
| NExT-QA chain SFT | 4.0 | 0.15 | 0.74 | 0.11 | 0.68 | 0.54 | 0.37 | 0.09 | 0.73 |
| Judgement | Yes | No | Unsure | |
|---|---|---|---|---|
| Step: entity visible on cited frame | 299 | 0.82 | 0.09 | 0.08 |
| Step: box on the named entity | 299 | 0.70 | 0.18 | 0.12 |
| Step: relevant to the question | 299 | 0.95 | 0.04 | 0.01 |
| Chain: steps suffice to answer | 100 | 0.89 | 0.08 | 0.03 |
| Chain: a step beyond locating | 100 | 0.08 | 0.92 | 0.00 |
| Judgement | Labels | SFT | GRPO |
|---|---|---|---|
| Step: entity visible on cited frame | 0.82 (299) | 0.82 (151) | 0.79 (150) |
| Step: box on the named entity | 0.70 (299) | 0.52 (151) | 0.53 (150) |
| where the entity is visible | 0.84 (246) | 0.63 (124) | 0.68 (118) |
| Step: relevant to the question | 0.95 (299) | 1.00 (151) | 1.00 (150) |
| Chain: steps suffice to answer | 0.89 (100) | 0.52 (50) | 0.50 (50) |
| Chain: a step beyond locating | 0.08 (100) | 0.06 (50) | 0.04 (50) |
| Model | Descr. | Expl. | Pred. | CF | All (opt.) | Box IoU | Blind gap | Track gap |
|---|---|---|---|---|---|---|---|---|
| Qwen2.5-VL-3B (zero-shot) | 0.720 | 0.702 | 0.764 | 0.712 | – | – | 0.28 | 0.56 |
| + outcome-only GRPO, balanced | 0.776 | 0.939 | 0.900 | 0.820 | – | – | 0.47 | 0.59 |
| + outcome-only GRPO, natural | 0.776 | 0.957 | 0.868 | 0.823 | – | – | 0.50 | 0.63 |
| VideoRFT-3B [ 42 ] | 0.752 | 0.542 | 0.752 | 0.626 | – | – | 0.27 | 0.49 |
| VideoThinker-R1-3B [ 44 ] | 0.748 | 0.824 | 0.836 | 0.848 | – | – | 0.49 | 0.56 |
| Video-R1-7B [ 9 ] | 0.780 | 0.823 | 0.686 | 0.718 | – | – | 0.35 | 0.58 |
| Model | Descr. | Expl. | Pred. | CF | All (opt.) | Box IoU | Blind gap | Track gap |
|---|---|---|---|---|---|---|---|---|
| CLEVRER chain SFT | 0.888 | 0.902 | 0.820 | 0.805 | 0.851 | 0.679 | 0.39 | 0.51 |
| CLEVRER chain GRPO | 0.924 | 0.947 | 0.832 | 0.801 | 0.871 | 0.654 | 0.37 | 0.50 |
| Mixed plain SFT (bare answers) | 0.892 | 0.970 | 0.900 | 0.848 | 0.906 | – | 0.39 | 0.48 |
| Mixed plain SFT, seed 1 | 0.864 | 0.973 | 0.880 | 0.856 | 0.903 | – | 0.42 | 0.45 |
| + outcome GRPO | 0.904 | 0.971 | 0.892 | 0.843 | 0.904 | – | 0.39 | 0.49 |
| Mixed chain SFT | 0.860 | 0.648 | 0.798 | 0.763 | 0.739 | 0.658 | 0.27 | 0.49 |
| Model | Flip (evidence) | Flip (random) | Gold-evidence gap | Object-track gap |
|---|---|---|---|---|
| Qwen2.5-VL-3B (zero-shot) | 0.133 | 0.069 | 0.063 [0.035,0.094] | 0.560 [0.512,0.609] |
| + outcome-only GRPO, balanced | 0.204 | 0.085 | 0.119 [0.092,0.149] | 0.589 [0.547,0.628] |
| + outcome-only GRPO, natural | 0.191 | 0.093 | 0.097 [0.069,0.126] | 0.627 [0.590,0.666] |
| Mixed plain SFT | 0.107 | 0.040 | 0.067 [0.054,0.080] | 0.482 [0.461,0.503] |
| + outcome GRPO | 0.108 | 0.037 | 0.071 [0.058,0.085] | 0.488 [0.466,0.507] |
| CLEVRER chain SFT | 0.183 | 0.090 | 0.092 [0.071,0.114] | 0.510 [0.484,0.534] |
| Model | Seeds | Acc. (opt.) | Box IoU | Format | Blind gap | Track gap |
|---|---|---|---|---|---|---|
| CLEVRER chain SFT | 1 | 0.851 | 0.679 | 0.991 | 0.39 | 0.51 |
| CLEVRER chain GRPO (outcome + format) | 3 | 0.870 0.001 | 0.660 0.006 | 0.997 | 0.38 | 0.50 |
| + CSR (outcome, format, grounding, necessity, process) | 2 | 0.867 0.004 | 0.667 0.001 | 0.998 | 0.37 | 0.49 |
| + CSR without necessity | 1 | 0.866 | 0.637 | 0.996 | 0.40 | 0.51 |
| + CSR, step-level credit 0.3 | 1 | 0.867 | 0.658 | 0.990 | 0.38 | 0.46 |
| + CSR, step-level credit 1.0 | 1 | 0.842 | 0.607 | 0.962 | 0.38 | 0.50 |
| Model | Descr. | Expl. | Pred. | CF | All (opt.) | Box IoU | Blind gap | Track gap |
|---|---|---|---|---|---|---|---|---|
| InternVL3-2B (zero-shot) | 0.836 | 0.855 | 0.826 | 0.764 | – | – | 0.42 | 0.51 |
| + outcome-only GRPO | 0.856 | 0.927 | 0.896 | 0.770 | – | – | 0.50 | 0.56 |
| Chain SFT | 0.824 | 0.694 | 0.794 | 0.773 | 0.754 | 0.741 | 0.28 | 0.55 |
| + chain GRPO, lr | 0.840 | 0.762 | 0.806 | 0.776 | 0.783 | 0.734 | 0.30 | 0.57 |
| + chain GRPO, lr (cite-test model) | 0.820 | 0.812 | 0.816 | 0.780 | 0.803 | 0.674 | 0.33 | 0.55 |
| Model | Set acc. | Paired | Per-option | Blind set acc. |
|---|---|---|---|---|
| Qwen2.5-VL-3B, letters prompt | 0.580 | 0.430 | – | 0.410 |
| + outcome-only GRPO (CLEVRER), letters prompt | 0.515 | 0.350 | – | 0.390 |
| VideoThinker-R1-3B [ 44 ] , letters prompt | 0.475 | 0.310 | – | 0.340 |
| Qwen2.5-VL-3B | 0.115 | 0.050 | 0.616 | 0.070 |
| + outcome-only GRPO (CLEVRER) | 0.125 | 0.070 | 0.615 | 0.080 |
| CLEVRER chain SFT | 0.020 | 0.010 | 0.461 | 0.020 |