Existing vision-language models (VLMs) remain unreliable for fine-grained visual perception. In this report, we present WeVisOne-VL (WeChat Vision All-in-One), a powerful VLM that unifies object-level and pixel-level perception through structured next-token prediction. It supports multiple visual tasks including object detection, referring localization, counting, OCR, segmentation, and matting, enabling localization at any user-specific granularity, from points and boxes to masks and alpha mattes. Architecturally, we integrate a lightweight pixel decoder into the VLM via a special <OBJ> token to generate instance-level pixel outputs. We train the model with a three-stage curriculum SFT followed by mixed-task RL. Across over 30 benchmarks, WeVisOne-VL demonstrates leading performance on both object-level and pixel-level tasks, while maintaining robust rejection for object-absent queries. By releasing WeVisOne-VL to the open-source community, we hope to advance VLMs from broad visual understanding toward truly reliable perception systems for real-world deployment.
Figures & tables
Figure 2: Overview of the WeVisOne-VL Data Engine (WDE).
Figure 3: Representative input–output examples for the unified task formats used by WeVisOne-VL.
Figure 4: Overview of the lightweight pixel decoder. Qwen3-VL provides multi-level ViT features, the hidden state of each generated <OBJ> token, and its predicted box. The decoder builds shared visual features, updates a box-guided target query, and converts the two representations into a full-resolution mask or alpha matte.
Table 1: Training data composition. The SFT and RL rings show task distributions. The segmentation warm-up and matting fine-tuning rings show data-source proportions: Public vs. WDE (WeVisOne-VL Data Engine). Center numbers indicate training sample counts.
Method
COCO
LVIS
Dense200
VisDrone
0.5
0.95
Mean
0.5
0.95
Mean
0.5
0.95
Mean
0.5
0.95
Mean
Specialized Detectors
GroundingDINO-T [ 67 ]
69.8
23.0
56.6
47.7
22.7
38.8
36.9
19.7
33.1
55.2
3.9
38.5
Vision-Language Models
Qwen3-VL-2B [ 1 ]
62.9
13.1
45.2
58.5
17.8
40.8
14.7
1.8
10.0
37.0
1.5
23.1
Qwen3-VL-4B [ 1 ]
66.6
14.9
48.7
61.3
19.5
44.0
20.9
3.7
15.2
49.5
1.8
31.0
Table 2: Results on general object detection benchmarks COCO and LVIS, and dense object detection benchmarks Dense200 and VisDrone. COCO and LVIS are evaluated in a zero-shot setting. We report F1 at IoU=0.5, IoU=0.95, and averaged over IoU thresholds from 0.5 to 0.95 (Mean). Bold and underline denote the best and second-best results among VLMs.
Method
RefCOCOg
WeRef-Bench
PR-Bench
HumanRef
Ref-Adv
DVG
Ref-L4
val
test
Rej
Mean
Rej
Mean
Rej
Specialized Detectors
GroundingDINO-T [ 67 ]
45.9
46.8
0.0
21.1
14.7
25.2
0.0
25.2
19.7
24.1
23.1
Vision-Language Models
Qwen3-VL-2B [ 1 ]
70.8
71.0
1.7
30.8
22.8
41.6
4.3
63.2
34.7
35.2
67.9
Qwen3-VL-4B [ 1 ]
74.1
74.3
18.8
46.3
61.9
49.9
10.8
69.4
44.4
42.4
72.5
Table 3: Results on referring object detection. Entries are mean F1 over IoU thresholds from 0.5 to 0.95. Rej denotes the rejection rate (%) on absent-target queries; the RefCOCOg Rej is evaluated on RefCOCOg-Rej. For Ref-Adv, we use the Ref-Adv-s split. Bold and underline denote the best and second-best results among VLMs.
Method
CountQA
PixMo-Count
CLOC
KubriCount
MixCount
val
test
val
test
testA
testB
val
test
Specialized Detectors
Grounding DINO-T [ 67 ]
9.6
44.8
48.3
12.4
9.5
5.2
6.5
1.7
2.8
Vision-Language Models
Qwen3-VL-2B [ 1 ]
20.4
61.3
60.4
16.3
15.3
4.3
7.8
1.8
2.7
Qwen3-VL-4B [ 1 ]
29.2
69.6
65.6
19.7
18.6
7.1
12.1
3.5
4.3
Table 4: Object-counting exact-match accuracy (%, ↑ ). Bold and underline denote the best and second-best results among VLM-based models, respectively; the specialized detector is excluded from this ranking.
Method
PointBench
PixMo-Points
COCO
LVIS
Dense200
VisDrone
Qwen3-VL-2B [ 1 ]
61.2
62.1
72.7
61.9
29.5
35.7
Qwen3-VL-4B [ 1 ]
57.1
71.3
74.5
65.8
37.7
45.6
Qwen3-VL-8B [ 1 ]
62.5
72.2
77.5
67.7
42.2
46.4
Qwen3.5-4B [ 81 ]
61.5
59.2
72.7
62.4
7.4
38.3
Qwen3.5-9B [ 81 ]
66.4
62.6
74.4
64.8
10.5
40.6
Rex-Omni-3B [ 40 ]
69.9
72.9
80.5
70.8
82.5
58.9
Table 5: Results on object pointing. Bold and underline denote the best and second-best reported results, respectively.
Method
HierText
ICDAR2015
TotalText
SROIE
0.5
0.95
Mean
0.5
0.95
Mean
0.5
0.95
Mean
0.5
0.95
Mean
Specialist OCR Systems
PP-OCRv5 [ 12 ]
45.2
3.4
30.5
38.2
1.2
25.6
40.2
0.7
25.7
77.7
5.6
58.6
Vision-Language Models
Qwen3-VL-2B [ 1 ]
21.4
0.3
11.0
15.7
0.0
8.5
32.6
1.1
20.5
40.2
0.2
18.6
Qwen3-VL-4B [ 1 ]
39.9
0.9
22.7
52.0
0.8
30.0
48.3
3.3
33.0
69.0
1.0
40.4
Table 6: Results on OCR (text grounding and recognition). We report F1 at IoU=0.5, IoU=0.95, and averaged over IoU thresholds from 0.5 to 0.95 (Mean). Bold and underline denote the best and second-best results among VLM-based models, respectively; the specialist OCR system is excluded from this ranking.
Method
RefCOCO
RefCOCO+
RefCOCOg
ReasonSeg
Ref-ZOM
val
testA
testB
val
testA
testB
val
test
val
test
cIoU
Rej
LISA-13B (ft) [ 48 ]
76.0
78.8
72.9
65.0
70.2
58.1
69.5
70.5
65.0
61.3
-
-
GSVA-7B [ 110 ]
77.2
78.9
73.5
65.9
69.6
59.8
72.7
73.3
50.5
-
66.6
41.9
LaSagnA-7B [ 106 ]
76.8
78.7
73.8
66.4
70.6
60.1
70.6
71.9
-
-
57.4
0.7
Text4Seg-7B [ 50 ]
79.3
81.9
76.2
72.1
77.6
66.1
72.1
73.9
-
-
54.5
25.9
VRS-HQ-7B [ 27 ]
73.5
77.5
69.5
61.7
67.6
54.3
66.7
67.5
55.2
51.7
63.0
13.9
Table 7: Referring segmentation results. RefCOCO-family scores are cIoU, while ReasonSeg scores are gIoU. Ref-ZOM baseline scores are obtained using official checkpoints. A hyphen (-) denotes an unreported result. Bold and underline denote the best and second-best reported results in each column, respectively.
Dataset
Prompt
Method
MAD ↓
MSE ↓
Grad ↓
Conn ↓
SAD ↓
P3M-500-NP
Box
Matte Anything [ 117 ]
24.4
20.4
20.9
40.4
40.9
SAM2-Matting (SAM2.1-T) [ 91 ]
22.7
18.2
20.0
37.8
38.1
SAM2-Matting (SAM2.1-B+) [ 91 ]
19.0
15.0
18.3
31.0
31.4
SAM2-Matting (SAM3) [ 91 ]
15.4
11.4
17.6
25.1
25.8
WeVisOne-VL-2B
6.6
3.1
10.7
10.6
11.2
WeVisOne-VL-4B
7.3
3.8
10.5
11.7
12.6
Table 8: Image matting results under box and language prompts. MAD and MSE are multiplied by 103 . On WeRefMat, all SAM2-Matting results use the SAM3 variant. Bold and underline denote the best and second-best results, respectively.
Table 10: Data sources and sample counts for segmentation and matting (M).
Hyperparameter
SFT-S1
SFT-S2
SFT-S3
RL
Trainable modules
LLM
VLM
VLM + Dec.
LLM
Learning rate: LLM
2×10−5
5×10−6
5×10−6
1×10−6
Learning rate: vision encoder
–
5×10−7
5×10−6
–
Learning rate: projector
–
5×10−6
5×10−6
–
Learning rate: pixel decoder
–
–
1×10−5
–
Warmup ratio
5%
5%
3%
–
Appendix
Table 11: Training hyperparameters for SFT and RL. VLM comprises the vision encoder, projector, and LLM. Dec. denotes the pixel decoder. A dash in the learning-rate rows indicates a frozen or inactive module. SFT-S3 uses LoRA for the LLM ( r=128 , α=256 , dropout 0.05 ), with fully trainable token embeddings and output head. The other modules are fully trainable. For RL, the maximum prompt and response lengths are both 4,096 tokens.
Recent vision-language models (VLMs) excel at multimodal understanding and reasoning, yet their fine-grained visual perception remains underexplored. A natural extension of ``How many r are there in Strawberry?'' asks: how small a visual pattern can a VLM reliably perceive? As such, we introduce FineSightBench, a new benchmark that systematically probes this limit by separating perception tasks (pixel-level recognition of letters, shapes, objects) from reasoning tasks (spatial reasoning, counting, ordering over small targets) across controlled scales of 4--48px. Through comprehensive experiments and detailed failure mode analysis on state-of-the-art models, we reveal a sharp dissociation: perception saturates around 12px, while reasoning remains limited even at larger scales, with persistent numeracy and sequence errors. These findings expose fundamental deficiencies in VLMs' fine-scale visual reasoning that demand more rigorous evaluation.
Lujun Li, Lama Sleem, Niccolo Gentile +4
University of Luxembourg · Foyer S.A. · Université Paris-Saclay
Vision-Language Models (VLMs) still struggle on tasks requiring complex visual understanding. We argue that the core issue is not high-level reasoning, but instead failing to locate critical details in the image. Due to this shortcoming, VLMs generate often plausible but incorrect reasoning based on flawed perceptual grounding. To address this, we propose Locator-Critic (LOCI), a training-free framework that decouples visual search from evidence verification. LOCI employs a Locator agent to propose candidate visual evidence and a separate Critic agent to evaluate its relevance and sufficiency. These agents engage in an iterative refinement loop, progressively improving the evidence until it is adequate to answer the given question. This decoupled, self-correcting process yields substantial performance gains, achieving state-of-the-art results on multiple complex visual benchmarks. LOCI improves accuracy for both open-weight models like Qwen3-VL (+12.1 on V*, +5.8 on HR-Bench and +11.2 on VisualProbe-Hard) and proprietary models like Gemini 2.5 Pro (+8.9 on V*, +4.3 on HR-Bench, +4.8 on VisualProbe-Hard).
Vision-language models (VLMs) have achieved impressive performance across a wide range of multimodal tasks. However, they often fail on tasks that require fine-grained visual perception, even when the required information is still present in their internal representations. Prior work has attributed this ``hidden-in-plain-sight'' gap to the language model, but the cause remains unexplained. In this work, we demonstrate that this gap arises from the language model's lack of semantic labels for fine-grained visual details: when visual entities can be mapped to known concepts, VLMs bypass visual comparison and reason through language; when they cannot, VLMs resort to brittle and hallucinated descriptions. We verify this across semantic correspondence, synthetic shape matching, and face matching, and find that VLMs perform much better when the relevant entities are nameable than when they are unnamable. Mechanistically, Logit Lens analysis confirms that VLMs explicitly recover semantic labels for nameable entities and surface more unique tokens compared to unnameable entities. Furthermore, we show that this limitation can be addressed: teaching completely arbitrary names for unknown entities improves performance. More importantly, task-specific finetuning yields even stronger generalization without relying on language priors, i.e., through real visual perception. Our findings suggest that current VLM failures on visual tasks reflect a learned shortcut rather than a fundamental limitation of multimodal reasoning. Code and datasets are available at https://github.com/Patchwork53/VLMs-Need-Words-COLM2026.