Autoregressive perception models trained to localize visual entities under the open-vocabulary setting are mostly trained using Supervised fine-tuning (SFT) with maximum likelihood, yet it optimizes a proxy objective (per-token cross-entropy) that is fundamentally misaligned with perception metrics such as precision and recall. In this paper, we explore post-training reinforcement learning (RL), specifically GRPO, to directly align these models with their evaluation metrics. Building up on the recently introduced Falcon Perception, we design an RL framework that addresses perception-specific challenges: reward design for set-structured outputs and multi-head sampling control. We discover multiple benefits from RL for perception: first, RL unlocks state-of-the-art performance in very dense scenes (up to 500 objects per scene), a regime where most existing systems degrade sharply or collapse; furthermore it fixes common issues in autoregressive perception models like mask repetitions and removes almost entirely the need for NMS and coordinate deduplication, which improve both performance and efficiency and remove the need for hyperparameters tuning; overall, we notice improvements on all levels of difficulties in referring expression segmentation (on PBench and SACO-Gold), and we find an elegant way to preserve the knowledge of whether an object exists or not (as evaluated by MCC) without training on negative samples. We show that a simple reward that penalizes false negatives and positives is sufficient. We develop two hybrid self-annotation pipelines, respectively tailored for difficult referring expressions and very dense scenes, and show their benefits on RL-training. Model weights are released as a Falcon Perception revision~\footnote{https://huggingface.co/tiiuae/Falcon-Perception}. Datasets will be published.
Figures & tables
Figure 1: Falcon Perception-HD on very dense scenes. The most impressive effect of RL on top of Falcon Perception is the ability to solve high-density perception at a scale where prior systems degrade or collapse. Top: predictions of the SFT-only baseline (Falcon Perception). Bottom: predictions of Falcon Perception-HD after RL post-training. The base model suffers from low recall and collapse; RL fixes both.
Figure 2: Predictions exceed the annotations. Original training annotations (top) vs. Falcon Perception-HD predictions (bottom).
Table 3
Figure 3: PBench level 4 (relation binding) qualitative results. Falcon Perception (top) vs. Falcon Perception-HD (bottom). RL improves on the hardest compositional referring expressions, where the SFT-only model tends to under/over-segment, miss the correct relational referent, or hallucinate.
Table 5
Model
Coord head
Size head
SFT init (pre-RL)
1.431
2.197
SFT, 1 epoch, same data
1.370 ( −4% )
2.365 ( +8% )
FP-HD (RL)
0.526 ( −63% )
1.729 ( −21% )
Table 9: Mean policy entropy of the coordinate and size heads. The size head is frozen and decoded greedily, yet RL reduces its entropy through the cascade effect. One epoch of SFT on the same data leaves both entropies essentially unchanged.
Figure 8: Rollout samples from intermediate training checkpoints. Each row shows several GRPO rollouts for the same prompt. Rollouts that emit near-duplicate predictions or that under-cover the scene receive negative within-group advantages and are suppressed by the policy gradient; this is the mechanism by which RL eliminates mask repetition (§ 5.3 ) and improves dense-scene recall (§ 5.1 ) without any explicit rule against either failure mode.
Figure 9: Final dense-scene annotations. Examples of the human-verified dense annotations used as ground truth during RL post-training.
Figure 10: Final hard-referring-expression annotations. Each example shows the rollout selected by the GPT-5 judge as the ground-truth target for a prompt where the greedy prediction of Falcon Perception failed.
Model
COCO-dense
LVIS-dense
SAM 3
43.5
31.5
Baseline FP
55.1
33.6
FP-HD (Ours)
58.1
40.5
Appendix
Table 15: External dense benchmarks. Left: mask F 1 on COCO-dense (category-as-query, ≥5 instances, 1,974 queries) and LVIS-dense ( 882 queries, >25 instances). Right: box F 1 @0.5 on Dense200.
Model
no NMS
with NMS
MRR
LocateAnything-3B [ 48 ]
35.9
36.6
28.5%
Rex-Omni-3B [ 14 ]
—
37.4
—
Falcon Perception (pre-RL)
67.9
68.0
2.1%
FP-HD (Ours)
73.9
73.9
0.4%
Appendix
Table 16: Box F 1 @0.5 on the PBench dense split. Ground-truth boxes are derived from the PBench dense masks; the protocol is identical for all models. NMS yields only marginal improvements on the redundant baselines, while Falcon Perception-HD does not need it.
k
Falcon Perception (AR)
SAM 3 (DETR)
deterministic
72.1
58.4
pass@2
72.7
54.5
pass@4
76.8
55.7
pass@8
79.0
56.8
Δ (pass@8 − deterministic)
+6.9
−3.4
Appendix
Table 17: pass@ k on the PBench dense split. Stochastic sampling improves over the deterministic prediction for the autoregressive model, which motivates RL post-training, but degrades the DETR-style SAM 3 under the per-query Bernoulli policy.
Figure 11: Training curves. Top: total reward, precision, recall, mean matched IoU, and F 1 . Bottom: per-head policy entropy (LM, coordinate, size) and within-group advantage standard deviation. The size head is frozen and sampled greedily, yet its entropy decreases in lockstep with the two trainable heads, an in-training manifestation of the cascade effect of § 3.1 .
Perception-centric systems are typically implemented with a modular encoder-decoder pipeline: a vision backbone for feature extraction and a separate decoder (or late-fusion module) for task prediction. This raises a central question: is this architectural separation essential or can a single early-fusion stack do both perception and task modeling at scale? We introduce Falcon Perception, a unified dense Transformer that processes image patches and text tokens in a shared parameter space from the first layer, using a hybrid attention pattern (bidirectional among image tokens, causal for prediction tokens) to combine global visual context with autoregressive, variable-length instance generation. To keep dense outputs practical, Falcon Perception retains a lightweight token interface and decodes continuous spatial outputs with specialized heads, enabling parallel high-resolution mask prediction. Our design promotes simplicity: we keep a single scalable backbone and shift complexity toward data and training signals, adding only small heads where outputs are continuous and dense. On SA-Co, Falcon Perception improves mask quality to 68.0 Macro-F1 compared to 62.3 of SAM3. We also introduce PBench, a benchmark targeting compositional prompts (OCR, spatial constraints, relations) and dense long-context regimes, where the model shows better gains. Finally, we extend the same early-fusion recipe to Falcon OCR: a compact 300M-parameter model which attains 80.3% on olmOCR and 88.64 on OmniDocBench.
Recent advancements in reinforcement learning with verifiable rewards (RLVR) have significantly improved the complex reasoning ability of vision-language models (VLMs). However, its outcome-level supervision is too coarse to diagnose and correct errors within the reasoning chain. To this end, we propose Perceval, a process reward model (PRM) that enables token-level error grounding, which can extract image-related claims from the response and compare them one by one with the visual evidence in the image, ultimately returning claims that contain perceptual errors. Perceval is trained with perception-intensive supervised training data. We then integrate Perceval into the RL training process to train the policy models. Specifically, compared to traditional GRPO, which applies sequence-level advantages, we apply token-level advantages by targeting penalties on hallucinated spans identified by Perceval, thus enabling fine-grained supervision signals. In addition to augmenting the training process, Perceval can also assist VLMs during the inference stage. Using Perceval, we can truncate the erroneous portions of the model's response, and then either have the model regenerate the response directly or induce the model to reflect on its previous output. This process can be repeated multiple times to achieve test-time scaling. Experiments show significant improvements on benchmarks from various domains across multiple reasoning VLMs trained with RL, highlighting the promise of perception-centric supervision as a general-purpose strategy. For test-time scaling, it also demonstrates consistent performance gains over other strategies, such as major voting. Our code and data will be publicly released at https://github.com/RUCAIBox/Perceval.
Yingqian Min, Kun Zhou, Yifan Li +6
Gaoling School of Artificial Intelligence, Renmin University of China. · Bytedance. · University of California, San Diego. +1
Fine-grained visual perception in MLLMs is commonly improved by raising the resolution, but the added visual tokens inflate vision-encoding and language-model prefilling costs. We show that the two operations underlying fine-grained perception, localizing the region of interest (RoI) and recognizing its content, have different resolution requirements. In a controlled diagnostic, localization tolerates roughly 3 to 4 times stronger token compression than recognition, which motivates localizing from a coarse view and concentrating resolution on the selected evidence. Decoding coordinates with the MLLM can be trained end-to-end from answers, but costs a full model pass per query and depends on grounding ability. A lightweight proposal network distilled from the model's attention is fast, but inherits the noise of its attention targets. The RoI from the proposal network reaches the answer through a discrete region choice, so its faithfulness to the answer cannot supervise the network. We therefore optimize the proposal network with region-level reinforcement learning, which we call Vision-RL2. It treats coherent regions as actions, and a frozen MLLM reader scores each one by how its removal changes the answer likelihood. Complementary subtractive and additive objectives suppress distracting proposals and recover missing evidence, updating only the predictor without region annotations, response sampling, or reasoning trajectories. The refined proposal further enables a sparse encoding that magnifies evidence and excludes background tokens. Across six fine-grained benchmarks and four MLLM backbones, Vision-RL2 improves accuracy over the base model at every token budget and surpasses its largest-budget accuracy with about 4 times fewer visual tokens. Code is available at https://github.com/YuHengsss/VisionRL2 .
Yuheng Shi, Xiaohuan Pei, Minjing Dong +1
University of Sydney · City University of Hong Kong