Reasoning segmentation requires identifying the target referred to by a query among visually similar objects, yet trajectory-level rewards cannot directly indicate which tokens provide relevant visual evidence. We propose Hint-Informed Self-Distillation (HISD), which uses target-disambiguation evidence for token-level credit assignment. An external model generates Segment Hints offline from training images, queries, and ground-truth target annotations, organizing target identity, discriminative attributes, confusable objects, and anchoring paths. A hint-conditioned exponential moving average teacher scores the student's own trajectories, and HISD computes the teacher-to-student evidence ratio over a token support set to adjust only the magnitudes of positive group-relative advantages, leaving nonpositive advantages unchanged. This design encourages concise reasoning while retaining the information needed for target disambiguation, and the student requires neither hints nor a teacher at inference. Under a unified training protocol, HISD improves ReasonSeg test gIoU and cIoU over our reproduction of DPAD by 3.00 and 3.55 percentage points, respectively. Across the five evaluated splits of ReasonSeg and the RefCOCO family, HISD achieves higher segmentation metrics than this baseline and reduces the average token count of valid reasoning spans by 80.17 to 91.83 percent.
Figures & tables
Figure 1: For the same query, HISD selects the refrigerator, while Seg-Zero and DPAD select the black trash bin. The charts report aggregate cIoU and average reasoning-token counts on the ReasonSeg test set from local runs under the same budget.
Figure 2: Hint-Informed Self-Distillation (HISD) pipeline. Offline Segment Hints condition an exponential moving average (EMA) teacher that scores the same on-policy student responses. Terminal rewards supply trajectory advantages; teacher-derived weights modulate positive-advantage updates, while nonpositive advantages retain the original reward signal. The token sequences, probability curves, and reward curve are schematic. The hint and teacher are used only during training.
Table 2: cIoU (%) on the RefCOCO-family datasets. All three evaluation splits are shown. † : Results reproduced by us.
Stratum
N
SegZero
DPAD
HISD
Query type
Attribute
99
59.26 / 129.02
61.49 / 75.07
62.79 / 11.51
Relation
48
59.90 / 132.83
55.96 / 76.46
59.26 / 11.04
Logic
632
55.63 / 123.80
56.93 / 79.42
60.18 / 14.70
Difficulty
Easy
173
67.49 / 122.46
68.32 / 75.01
73.10 / 13.24
Table 3: ReasonSeg test-set results stratified by query type and difficulty. Each entry is gIoU (%) ↑ / #Tok ↓ . Bold indicates the best value for each metric in each stratum. #Tok denotes the mean number of tokens within successfully parsed, closed <think> spans.
Split
Method
#Tok ↓
S1
SNR
TS1
TSNR
ReasonSeg test
SegZero
125.04
NA
NA
28.92
0.93
DPAD
78.69
28.27
1.13
29.09
0.99
HISD
14.07
28.15
1.16
27.25
1.02
ReasonSeg val
SegZero
126.95
NA
NA
29.84
0.93
DPAD
77.36
29.56
1.14
29.83
1.00
HISD
15.34
29.01
1.16
27.71
1.04
Table 4: CLIP alignment and reasoning length. Shown are ROI similarities S1 and TS1 and the means of per-sample ROI/AOI ratios, SNR and TSNR; full-image means S2 and TS2 are omitted. NA: missing anchored descriptions. Bold: shortest reasoning and highest TSNR per split.
Figure 3: Comparison on horns, plant leaves, and runway queries. Predictions and verbatim output excerpts highlight HISD’s target cues. Values are per-sample IoU and full reasoning-token counts. The leaves and runway queries are abridged; ellipses mark omissions.
Variant
ReasonSeg test
ReasonSeg val
RefCOCO testA
RefCOCO+ testA
RefCOCOg test
Avg.
(a) Teacher updates
Current policy (every step)
58.52 / 53.64
61.80 / 53.87
80.33 / 78.98
77.44 / 75.39
74.79 / 72.66
70.58 / 66.91
Hard update (every 10 steps)
54.64 / 48.23
58.92 / 49.23
80.40 / 79.46
77.39 / 75.04
74.30 / 72.18
69.13 / 64.83
Frozen policy
54.51 / 49.12
60.21 / 54.79
80.79 / 79.74
77.29 / 75.14
74.40 / 72.40
69.44 / 66.24
EMA (Ours)
60.45 / 57.49
63.76 / 61.98
80.96 / 80.10
78.19 / 76.31
75.02 / 73.24
71.68 / 69.83
(b) Privileged inputs
Table 5: HISD ablations with Qwen2.5-VL-7B-Instruct. Panels vary (a) teacher updates, (b) privileged inputs, and (c) token reweighting. Entries are gIoU / cIoU (%); bold indicates the best result in each panel.