WeVisOne-VL: Unified Visual Perception from Objects to Pixels

Tianming Liang; Junyi Pan; Yuxi Li; Canmiao Fu; Zhipeng Huang; Chong Sun; Chen Li; Jing Lyu

cs.CVOct 22, 2026

WeVisOne-VL: Unified Visual Perception from Objects to Pixels

Authors: Tianming Liang, Junyi Pan, Yuxi Li, Canmiao Fu, Zhipeng Huang, Chong Sun, Chen Li, Jing Lyu

Organizations: WeChat Vision, Tencent Inc.

Abstract

Existing vision-language models (VLMs) remain unreliable for fine-grained visual perception. In this report, we present WeVisOne-VL (WeChat Vision All-in-One), a powerful VLM that unifies object-level and pixel-level perception through structured next-token prediction. It supports multiple visual tasks including object detection, referring localization, counting, OCR, segmentation, and matting, enabling localization at any user-specific granularity, from points and boxes to masks and alpha mattes. Architecturally, we integrate a lightweight pixel decoder into the VLM via a special <OBJ> token to generate instance-level pixel outputs. We train the model with a three-stage curriculum SFT followed by mixed-task RL. Across over 30 benchmarks, WeVisOne-VL demonstrates leading performance on both object-level and pixel-level tasks, while maintaining robust rejection for object-absent queries. By releasing WeVisOne-VL to the open-source community, we hope to advance VLMs from broad visual understanding toward truly reliable perception systems for real-world deployment.

Figures & tables

Figure 2: Overview of the WeVisOne-VL Data Engine (WDE).

Figure 3: Representative input–output examples for the unified task formats used by WeVisOne-VL.

Figure 4: Overview of the lightweight pixel decoder. Qwen3-VL provides multi-level ViT features, the hidden state of each generated <OBJ> token, and its predicted box. The decoder builds shared visual features, updates a box-guided target query, and converts the two representations into a full-resolution mask or alpha matte.

Table 1: Training data composition. The SFT and RL rings show task distributions. The segmentation warm-up and matting fine-tuning rings show data-source proportions: Public vs. WDE (WeVisOne-VL Data Engine). Center numbers indicate training sample counts.

Method	COCO			LVIS			Dense200			VisDrone
Method	0.5	0.95	Mean	0.5	0.95	Mean	0.5	0.95	Mean	0.5	0.95	Mean
Specialized Detectors
GroundingDINO-T [ 67 ]	69.8	23.0	56.6	47.7	22.7	38.8	36.9	19.7	33.1	55.2	3.9	38.5
Vision-Language Models
Qwen3-VL-2B [ 1 ]	62.9	13.1	45.2	58.5	17.8	40.8	14.7	1.8	10.0	37.0	1.5	23.1
Qwen3-VL-4B [ 1 ]	66.6	14.9	48.7	61.3	19.5	44.0	20.9	3.7	15.2	49.5	1.8	31.0

Table 2: Results on general object detection benchmarks COCO and LVIS, and dense object detection benchmarks Dense200 and VisDrone. COCO and LVIS are evaluated in a zero-shot setting. We report F1 at IoU=0.5, IoU=0.95, and averaged over IoU thresholds from 0.5 to 0.95 (Mean). Bold and underline denote the best and second-best results among VLMs.

Method	RefCOCOg			WeRef-Bench		PR-Bench		HumanRef	Ref-Adv	DVG	Ref-L4
Method	val	test	Rej	Mean	Rej	Mean	Rej
Specialized Detectors
GroundingDINO-T [ 67 ]	45.9	46.8	0.0	21.1	14.7	25.2	0.0	25.2	19.7	24.1	23.1
Vision-Language Models
Qwen3-VL-2B [ 1 ]	70.8	71.0	1.7	30.8	22.8	41.6	4.3	63.2	34.7	35.2	67.9
Qwen3-VL-4B [ 1 ]	74.1	74.3	18.8	46.3	61.9	49.9	10.8	69.4	44.4	42.4	72.5

Table 3: Results on referring object detection. Entries are mean F1 over IoU thresholds from 0.5 to 0.95. Rej denotes the rejection rate (%) on absent-target queries; the RefCOCOg Rej is evaluated on RefCOCOg-Rej. For Ref-Adv, we use the Ref-Adv-s split. Bold and underline denote the best and second-best results among VLMs.

Method	CountQA	PixMo-Count		CLOC		KubriCount		MixCount
Method		val	test	val	test	testA	testB	val	test
Specialized Detectors
Grounding DINO-T [ 67 ]	9.6	44.8	48.3	12.4	9.5	5.2	6.5	1.7	2.8
Vision-Language Models
Qwen3-VL-2B [ 1 ]	20.4	61.3	60.4	16.3	15.3	4.3	7.8	1.8	2.7
Qwen3-VL-4B [ 1 ]	29.2	69.6	65.6	19.7	18.6	7.1	12.1	3.5	4.3

Table 4: Object-counting exact-match accuracy (%,

\uparrow

). Bold and underline denote the best and second-best results among VLM-based models, respectively; the specialized detector is excluded from this ranking.

Method	PointBench	PixMo-Points	COCO	LVIS	Dense200	VisDrone
Qwen3-VL-2B [ 1 ]	61.2	62.1	72.7	61.9	29.5	35.7
Qwen3-VL-4B [ 1 ]	57.1	71.3	74.5	65.8	37.7	45.6
Qwen3-VL-8B [ 1 ]	62.5	72.2	77.5	67.7	42.2	46.4
Qwen3.5-4B [ 81 ]	61.5	59.2	72.7	62.4	7.4	38.3
Qwen3.5-9B [ 81 ]	66.4	62.6	74.4	64.8	10.5	40.6
Rex-Omni-3B [ 40 ]	69.9	72.9	80.5	70.8	82.5	58.9

Table 5: Results on object pointing. Bold and underline denote the best and second-best reported results, respectively.

Method	HierText			ICDAR2015			TotalText			SROIE
Method	0.5	0.95	Mean	0.5	0.95	Mean	0.5	0.95	Mean	0.5	0.95	Mean
Specialist OCR Systems
PP-OCRv5 [ 12 ]	45.2	3.4	30.5	38.2	1.2	25.6	40.2	0.7	25.7	77.7	5.6	58.6
Vision-Language Models
Qwen3-VL-2B [ 1 ]	21.4	0.3	11.0	15.7	0.0	8.5	32.6	1.1	20.5	40.2	0.2	18.6
Qwen3-VL-4B [ 1 ]	39.9	0.9	22.7	52.0	0.8	30.0	48.3	3.3	33.0	69.0	1.0	40.4

Table 6: Results on OCR (text grounding and recognition). We report F1 at IoU=0.5, IoU=0.95, and averaged over IoU thresholds from 0.5 to 0.95 (Mean). Bold and underline denote the best and second-best results among VLM-based models, respectively; the specialist OCR system is excluded from this ranking.

Method	RefCOCO			RefCOCO+			RefCOCOg		ReasonSeg		Ref-ZOM
Method	val	testA	testB	val	testA	testB	val	test	val	test	cIoU	Rej
LISA-13B (ft) [ 48 ]	76.0	78.8	72.9	65.0	70.2	58.1	69.5	70.5	65.0	61.3	-	-
GSVA-7B [ 110 ]	77.2	78.9	73.5	65.9	69.6	59.8	72.7	73.3	50.5	-	66.6	41.9
LaSagnA-7B [ 106 ]	76.8	78.7	73.8	66.4	70.6	60.1	70.6	71.9	-	-	57.4	0.7
Text4Seg-7B [ 50 ]	79.3	81.9	76.2	72.1	77.6	66.1	72.1	73.9	-	-	54.5	25.9
VRS-HQ-7B [ 27 ]	73.5	77.5	69.5	61.7	67.6	54.3	66.7	67.5	55.2	51.7	63.0	13.9

Table 7: Referring segmentation results. RefCOCO-family scores are cIoU, while ReasonSeg scores are gIoU. Ref-ZOM baseline scores are obtained using official checkpoints. A hyphen (-) denotes an unreported result. Bold and underline denote the best and second-best reported results in each column, respectively.

Dataset	Prompt	Method	MAD $\downarrow$	MSE $\downarrow$	Grad $\downarrow$	Conn $\downarrow$	SAD $\downarrow$
P3M-500-NP	Box	Matte Anything [ 117 ]	24.4	20.4	20.9	40.4	40.9
		SAM2-Matting (SAM2.1-T) [ 91 ]	22.7	18.2	20.0	37.8	38.1
		SAM2-Matting (SAM2.1-B+) [ 91 ]	19.0	15.0	18.3	31.0	31.4
		SAM2-Matting (SAM3) [ 91 ]	15.4	11.4	17.6	25.1	25.8
		WeVisOne-VL-2B	6.6	3.1	10.7	10.6	11.2
		WeVisOne-VL-4B	7.3	3.8	10.5	11.7	12.6

Table 8: Image matting results under box and language prompts. MAD and MSE are multiplied by

10^{3}

. On WeRefMat, all SAM2-Matting results use the SAM3 variant. Bold and underline denote the best and second-best results, respectively.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Task	Source	Datasets	S1	S2	S3
Detection	Public	Objects365 [ 88 ] , OpenImages-V6 [ 47 ] , V3Det [ 102 ] , DeepFashion2 [ 24 ] , ODinW-35 [ 52 ] , EgoObjects [ 128 ] , PACO [ 82 ] , PartImageNet++ [ 59 ] , BDD100K [ 119 ] , nuImages [ 74 ] , VisDrone [ 19 ] , CrowdHuman [ 89 ] , SKU-110K [ 26 ] , PartImageNet [ 31 ] , MOT20Det [ 15 ] , MOT17Det [ 14 ] , OWDOD [ 93 ] , FSCD-147 [ 75 ]	18.4	16.2	8.4
	WDE	CC3M, GRIT, LAION, SA-1B	42.2	10.7	7.2
Referring	Public	RefCOCO [ 120 ] , gRefCOCO [ 33 ] , Flickr30k [ 79 ] , HumanRef-CoT [ 39 ] , HumanParts [ 60 ] , $D^{3}$ [ 112 ] , OpenRef [ 109 ] , Unsplash [ 104 ]	6.3	2.1	1.0
	WDE	GRIT, CC3M, LAION	11.8	5.5	1.6
Counting	Public	CLOC [ 51 ] , MixCount [ 20 ] , KubriCount [ 66 ] , PixMo [ 13 ] , OWDOD [ 93 ] , FSC-147 [ 83 ] , FSCD-147 [ 75 ]	2.4	1.2	0.6
Pointing	Public	PixMo [ 13 ] , Molmo2 [ 10 ] , CLOC [ 51 ] , FSC-147 [ 83 ]	2.8	0.9	0.4

Appendix

Table 9: Training data volumes across the three SFT stages (M).

Task	Source	Datasets	Size (M)
Segmentation	Public	SA-1B [ 45 ] , RefCOCO [ 120 ] , RefCOCO+ [ 120 ] , RefCOCOg [ 72 ] , ReVOS [ 114 ] , Ref-YouTube-VOS [ 87 ]	171.4
	WDE	CC3M, GRIT, LAION	288.4
Matting	Public	AM-2K [ 55 ] , Distinctions-646 [ 80 ] , UHRIM [ 64 ] , HHM-50K [ 98 ] , IHIM-50K [ 36 ] , RefMatte [ 57 ] , AIM-500 [ 56 ] , CelebA-Hair [ 11 ] , Transparent-460 [ 3 ] , SIMD [ 97 ] , P3M-10K (train) [ 54 ] , Real-Human [ 105 ] , RealWorldPortrait-636 [ 121 ]	0.3
	WDE	CC3M, SA-1B	129.8

Appendix

Table 10: Data sources and sample counts for segmentation and matting (M).

Hyperparameter	SFT-S1	SFT-S2	SFT-S3	RL
Trainable modules	LLM	VLM	VLM + Dec.	LLM
Learning rate: LLM	$2\times 10^{-5}$	$5\times 10^{-6}$	$5\times 10^{-6}$	$1\times 10^{-6}$
Learning rate: vision encoder	–	$5\times 10^{-7}$	$5\times 10^{-6}$	–
Learning rate: projector	–	$5\times 10^{-6}$	$5\times 10^{-6}$	–
Learning rate: pixel decoder	–	–	$1\times 10^{-5}$	–
Warmup ratio	5%	5%	3%	–

Appendix

Table 11: Training hyperparameters for SFT and RL. VLM comprises the vision encoder, projector, and LLM. Dec. denotes the pixel decoder. A dash in the learning-rate rows indicates a frozen or inactive module. SFT-S3 uses LoRA for the LLM (

r=128

\alpha=256

, dropout

0.05

), with fully trainable token embeddings and output head. The other modules are fully trainable. For RL, the maximum prompt and response lengths are both 4,096 tokens.

Image Matting

See all 9 papers

The Last Visible Pixel: Probing Fine-Scale Perception in Vision-Language Models
Jun 5, 2026Lujun Li, Lama Sleem, Niccolo Gentile +4VLM Evaluation Large Vision-Language Models
University of Luxembourg · Foyer S.A. · Université Paris-Saclay
LOCI: A Locator-Critic with Refinement Loop
Aug 31, 2026Walid Bousselham, Mathilde Caron, Arsha Nagrani +1Visual Question Answering Vision-Language Grounding
Google DeepMind
VLMs Need Words: Vision Language Models Ignore Visual Detail In Favor of Semantic Anchors
Apr 2, 2026Haz Sameen Shahgir, Xiaofu Chen, Yu Fu +4VLM Reasoning Fine-Grained Visual Perception
University of California, Riverside 1 · MBZUAI2

Jun 5, 2026J/K move · Enter open · S save

The Last Visible Pixel: Probing Fine-Scale Perception in Vision-Language Models

Lujun Li, Lama Sleem, Niccolo Gentile, Yangjie Xu+3

University of Luxembourg · Foyer S.A. · Université Paris-Saclay

VLM Evaluation Large Vision-Language Models VLM Reasoning Fine-Grained Visual Perception

Open paper AskPDF arXiv

Recent vision-language models (VLMs) excel at multimodal understanding and reasoning, yet their fine-grained visual perception remains underexplored. A natural extension of ``How many r are there in Strawberry?'' asks: how small a visual pattern can a VLM reliably perceive? As such, we introduce FineSightBench, a new benchmark that systematically probes this limit by separating perception tasks (pixel-level recognition of letters, shapes, objects) from reasoning tasks (spatial reasoning, counting, ordering over small targets) across controlled scales of 4--48px. Through comprehensive experiments and detailed failure mode analysis on state-of-the-art models, we reveal a sharp dissociation: perception saturates around 12px, while reasoning remains limited even at larger scales, with persistent numeracy and sequence errors. These findings expose fundamental deficiencies in VLMs' fine-scale visual reasoning that demand more rigorous evaluation.

Abstract

Image Matting

Explore similar work

The Last Visible Pixel: Probing Fine-Scale Perception in Vision-Language Models

LOCI: A Locator-Critic with Refinement Loop

VLMs Need Words: Vision Language Models Ignore Visual Detail In Favor of Semantic Anchors