POINTS-Long: Adaptive Dual-Mode Visual Reasoning in MLLMs

Haicheng Wang; Yuan Liu; Yikun Liu; Zhemeng Yu; Zhongyin Zhao; Yangxiu You; Zilin Yu; Le Tian

cs.CVApr 13, 2026

POINTS-Long: Adaptive Dual-Mode Visual Reasoning in MLLMs

Authors: Haicheng Wang, Yuan Liu, Yikun Liu, Zhemeng Yu, Zhongyin Zhao, Yangxiu You, Zilin Yu, Le Tian,

Organizations: SAI, Shanghai Jiao Tong University, China · WeChat AI, Tencent, China

Abstract

Multimodal Large Language Models (MLLMs) have recently demonstrated remarkable capabilities in cross-modal understanding and generation. However, the rapid growth of visual token sequences--especially in long-video and streaming scenarios--poses a major challenge to their scalability and real-world deployment. Thus, we introduce POINTS-Long, a native dual-mode MLLM featuring dynamic visual token scaling inspired by the human visual system. The model supports two complementary perception modes: focus mode and standby mode, enabling users to dynamically trade off efficiency and accuracy during inference. On fine-grained visual tasks, the focus mode retains the optimal performance, while on long-form general visual understanding, the standby mode retains 97.7-99.7% of the original accuracy using only 1/40-1/10th of the visual tokens. Moreover, POINTS-Long natively supports streaming visual understanding via a dynamically detachable KV-cache design, allowing efficient maintenance of ultra-long visual memory. Our work provides new insights into the design of future MLLMs and lays the foundation for adaptive and efficient long-form visual understanding.

Figures & tables

Figure 1: POINTS-Long: Bridging the Gap between Human Visual Perception and MLLM Scalability. Inspired by human’s adaptive visual processing, POINTS-Long introduces a dual-mode system which switches between high-fidelity Focus Mode and efficient Standby Mode, enabling both detailed analysis and long-term streaming understanding with significantly reduced cost.

Figure 2: POINTS-Long Architecture. The original visual patch sequence (blue) is processed by the original ViT modules. We introduce $n$ learnable tokens (orange) processed through duplicated learnable MLPs and projector, to act as the compressed representation of the full sequence. An additional temporal modeling allows better compression for video inputs. With symmetric attention mask, the original path is totally unaffected, thus preserving its performance. This dual-mode system is enabled by a two-stage post-training: Stage 1 (left) trains only the new parameters for visual distillation, while Stage 2 (middle) fine-tunes the LLM with a small learning rate for mode adaptation.

Figure 3: Streaming Inference in LLM. ( $\uparrow$ ) When handling streaming inputs, general MLLMs discard previous cached context when reaching maximum budget. ( $\downarrow$ ) POINTS-Long encodes new frames in Focus Mode. When local window is full, the original sequence’s cache is detached, and the compact standby-sequence cache is migrated to a long-term ”Memory Bank”.

Model	Num Frame	Token/ Frame	Total Num of Token	MVBench	Video-MME	MMBench-Video	Tempcompass	MLVU	LongVideoBench	Avg
Qwen2-VL-7B [ 85 ]	64	-	$>$ 12k	66.0	59.7	48.3	69.6	66.4	55.6	60.9
MiMo-VL-7B-RL [ 79 ]	64	-	$>$ 12k	63.2	65.0	54.0	-	66.2	-	-
Qwen2.5-VL-7B [ 3 ]	64	-	$>$ 12k	67.5	62.8	53.0	72.0	70.2	56.0	63.6
VideoLLaMA3-7B [ 110 ]	-	-	$>$ 12k	69.7	66.2	-	68.1	73.0	59.8	-
InternVL2.5-8B [ 12 ]	64	-	$>$ 12k	70.5	63.7	56.0	68.7	68.5	-	-
Qwen2.5-Omni-7B [ 95 ]	64	-	$>$ 12k	69.0	64.1	55.0	70.7	67.5	-	-

Table 1: Opencompass Video Benchmark. Under the same setting (64 frames), POINTS-Long achieves competitive performance (97.7%-99.7%) against original POINTS1.5-8B with drastically less tokens (2.5%-10%), and retains even better focus mode performance.

Model	Num Frame	Token/ Frame	Total Num of Token	CG-Bench (long-acc)	MovieChat1k	Egoschema	LVBench	TemporalBench	Activitynet-qa	WorldSense	Avg
Moviechat [ 74 ]	-	-	$>$ 12k	-	62.3	53.5	22.5	-	45.7	-	-
LLaVA-OV-7B [ 38 ]	64	-	$>$ 12k	30.9	-	59.8	26.9	59.4	56.0	37.7	-
MiMo-VL-7B-RL [ 79 ]	64	-	$>$ 12k	-	-	59.4	37.1	-	-	-
LLaVA-Video-7B [ 117 ]	-	-	$>$ 12k	-	-	57.3	-	63.6	56.5	40.2	-
InternVL3-8B [ 124 ]	-	-	$>$ 12k	38.6	-	-	44.1	-	-	-	-
POINTS1.5-8B (baseline)	64	324	$\approx$ 20K	36.7	77.0	60.6	44.3	64.3	54.3	40.4	53.9

Table 2: More Video Benchmarks. We evaluate on more video benchmarks to prove universality.

Model	Num Frame	Token/ Frame	Total Num of Token	LVBench	VideoMME (Long/Overall)	MMBench-Video	CG-Bench (60+/Overall)	MLVU	LongVideoBench (3600+/Overall)	Avg
POINTS1.5-8B	64	324	$\approx$ 20K	44.3	56.0/66.1	61.0	31.1/36.7	72.0	50.7/59.8	52.5
POINTS1.5-8B	128	144	$\approx$ 18K	45.4	54.4/65.0	61.3	32.4/37.0	72.0	51.2/60.2	52.8
POINTS-Long	64	8	512 (2.5%)	40.4	54.0/63.5	58.0	29.5/33.6	71.9	49.3/58.2	50.5
POINTS-Long	128	8	1024 (5%)	42.9	56.4/64.4	59.0	31.1/35.3	71.9	49.5/59.6	51.8
POINTS-Long	256	8	2048 (10%)	43.6	57.1/66.1	60.0	30.4/36.0	72.4	50.7/59.7	52.4
POINTS-Long	64	16	1024 (5%)	42.5	55.3/65.0	59.3	30.7/34.6	71.7	48.4/58.9	51.3

Table 3: Scalability of Inference Frame. By drastically compressing visual tokens, POINTS-Long can process more frames without context length overflow. we witness a steady gain with respect to frame number, which is not always the case for general MLLMs.

Model	MMBench	MMStar	MMMU_val	MathVista	OCRBench	AI2D	HallusionBench	MMVet	Avg
Qwen2-VL-7B [ 85 ]	81.0	60.7	53.7	61.6	84.3	83.0	50.4	61.8	67.1
InternVL2.5-8B [ 12 ]	82.5	63.2	56.2	64.5	82.1	84.6	49	62.8	68.1
MiniCPM-o-2.6 [ 102 ]	80.6	63.3	50.9	73.3	88.9	86.1	51.1	67.2	70.2
Qwen2.5-VL-7B [ 3 ]	82.2	64.1	58	68.1	88.8	84.3	51.9	69.7	70.9
Ovis2-8B [ 58 ]	83.6	64.6	57.4	71.8	89.1	86.6	56.3	65.1	71.8
SAIL-VL1.6-8B [ 105 ]	84.0	69.5	55.4	74.2	90.5	87.5	54.4	73.3	73.6

Table 4: Opencompass Image Benchmark. We show that our two-stage training will not harm the fine-grained capacity of focus mode. Bonus: With simple training-free attention-based pruning, the focus mode can be more efficient, beating other training-free baselines.

Model	Num Frame	Token/ Frame	Total Num of Token	LVBench	Video-MME	CG-Bench	Egoschema	MLVU	LongVideoBench	Avg
VideoStreaming [ 63 ]	-	-	256	-	-	-	44.1	-	-	-
Qwen2-VL-online [ 85 ]	-	-	11520	39.8	59.4	-	64.0	62.9	-	-
Flash-Vstream [ 112 ]	-	-	11520	42.0	61.2	-	68.2	66.3	-	-
POINTS1.5-8B-online	64	324	20736	41.7	59.3	33.1	59.3	64.7	53.5	51.9
POINTS-Long	248+8	8	3200	44.2	65.4	36.4	61.6	71.8	59.3	56.5
POINTS-Long	504+8	8	5248	46.0	65.0	35.5	59.7	70.3	59.2	56.0

Table 5: Streaming Understanding. General MLLMs struggle at long-range streaming VQA, while POINTS-Long preserves ultra-long memory by detachable KV cache mechanism shown in Fig 3 , resulting in much better performance.

Num Frame	MLP	Temporal	Stage 2	MVBench	Video-MME	MMBench-Video	Tempcompass	MLVU	LongVideoBench	Avg
64	$\times$	$\times$	$\times$	55.6	58.6	47.3	67.1	67.4	52.7	58.1
64	✓	✓	$\times$	57.5	61.5	53.7	67.8	69.2	56.2	61.0
64	✓	$\times$	✓	58.1	63.2	56.3	69.8	70.6	57.5	62.6
64	✓	✓	✓	59.4	63.5	58.0	69.9	71.9	58.2	63.5

Table 6: Ablation Study. We ablate the training design in Sec. 3.1 . All components are essential for obtaining the optimal result.

Model	Num Frame	Token/ Frame	FLOPs (ViT+LLM)	LLM Prefill Latency (s)	Generation Throughput (token/s)
POINTS1.5-8B	128	144	99.7+456.7	3.23	311
POINTS1.5-8B	256	144	199.4+1314.5	8.95	240
POINTS-Long	128	8	106.2+14.8	0.21 (6.5%)	1887 ( $6.1\times$ )
POINTS-Long	256	8	208.5+30.9	0.41 (4.6%)	1494 ( $6.2\times$ )
POINTS-Long	128	16	110.8+30.9	0.41 (12.7%)	1447 ( $4.7\times$ )
POINTS-Long	256	16	221.6+66.8	0.70 (7.8%)	1124 ( $4.7\times$ )

Table 7: Real-world Inference Speed-up. We evaluate the speed-up of the LLM side prefilling and decoding using SGLang and Pytorch Profiler on H20. While ViT’s cost grows linearly with the number of frames, LLM shows a quadratic increase in complexity.

Figure 4: POINTS1.5-8B-Instruct Architecture. POINTS1.5-8B consists of a native-resolution image encoder (initialized from Qwen2-VL-ViT), a pixel-shuffle projector reducing the token count by a factor of 4, and an LLM initialized from Qwen3-8B-Base. The architecture employs 1D RoPE for the LLM and 2D RoPE for the ViT.

Model	MMBench	MMStar	MMMU_val	MathVista	OCRBench	AI2D	HallusionBench	MMVet	Avg
POINTS1.5-8B (baseline)	81.9	65.7	53.2	70.9	85.8	83.9	50.1	64.7	69.5
POINTS-Long (focus)	82.1	66.1	53.7	70.6	85.5	84.2	48.3	66.7	69.7
POINTS-Long (ori-seq)	82.1	65.8	53.0	69.7	85.0	83.8	48.0	67.4	69.4

Table 8: Performance of Different Inference Mode. In standard focus mode, we concatenate the learnable standby tokens with the original visual tokens and pass to LLM. Nevertheless, it makes no big difference to inference with only the original visual tokens. *ori-seq means original sequence without standby tokens.

Model	Num Frame	Token/ Frame	Learning Rate	MVBench	Video-MME	MMBench-Video	Tempcompass	MLVU	LongVideoBench	Avg
POINTS-Long (standby)	64	16	1e-5	59.7	65.0	59.3	69.1	71.7	58.9	63.9
POINTS-Long (standby)	64	16	2e-5	58.3	64.9	59.3	69.3	71.8	58.2	63.6
POINTS-Long (standby)	64	16	5e-5	59.7	64.9	60.0	69.4	70.3	58.4	63.8
POINTS-Long (standby)	64	32	1e-5	60.8	65.7	60.9	70.3	71.6	59.5	64.8
POINTS-Long (focus)	64	32	1e-5	61.0	66.1	60.3	71.3	73.2	59.4	65.2
POINTS-Long (standby)	64	32	2e-5	61.3	65.9	60.3	70.6	70.8	59.5	64.7

Table 9: Learning Rate & Model Performance. We train the model under different learning rates (1e-5, 2e-5, 5e-5) in stage 2. Performance differences were minimal, proving the training scheme’s robustness.

Model	Num Frame	Token/ Frame	Training Data	MVBench	Video-MME	MMBench-Video	Tempcompass	MLVU	LongVideoBench	Avg
POINTS-Long (standby)	64	16	Standard	59.7	65.0	59.3	69.1	71.7	58.9	63.9
POINTS-Long (standby)	64	16	Reduced	59.3	64.0	59.0	68.9	71.3	59.2	63.6
POINTS-Long (standby)	64	32	Standard	60.8	65.7	60.9	70.3	71.6	59.5	64.8
POINTS-Long (standby)	64	32	Reduced	60.7	65.1	59.3	69.9	70.8	60.9	64.3

Table 10: Training Data & Model Performance. We train the model using different amount of data in stage 2. By adding more high-quality data in stage 2 (85%-100%), we witness a steady improvement in performance.

Model	Num Frame	Token/ Frame	Total Num of Token	MVBench	Video-MME	MMBench-Video	Tempcompass	MLVU	LongVideoBench	Avg
POINTS1.5-8B (baseline)	64	324	$\approx$ 20K	60.3	66.1	61.0	71.1	72.0	59.8	65.0
POINTS1.5-8B (low-resolution)	64	32	2048 (10%)	54.9	61.2	51.0	67.1	67.3	53.9	59.2 (91.1%)
POINTS1.5-8B (pooling)	64	32	2048 (10%)	54.9	55.4	43.0	66.6	67.1	54.5	56.9 (87.5%)
POINTS1.5-8B (+VisionZip [ 101 ] )	64	32	2048 (10%)	56.7	60.0	51.7	66.6	65.9	54.9	59.3 (91.2%)
POINTS1.5-8B (+Dycoke [ 78 ] )	64	32	2048 (10%)	56.1	62.5	55.7	67.5	67.2	55.6	60.8 (93.5%)
POINTS1.5-8B (+PruneVID [ 29 ] )	64	32	2048 (10%)	57.8	62.0	55.3	69.3	67.5	56.5	61.4 (94.4%)

Table 11: Comparison with Visual Token Reduction Methods. Under the same setting (or even using fewer tokens), POINTS-Long exceeds previous visual token reduction methods by a large margin. It’s a natural result since the standby mode is carefully trained as a native inference mode.

Model	Num Frame	Token/ Frame	Total Num of Token	CG-Bench	Video-MME	MMBench-Video	Tempcompass	MLVU	LongVideoBench	Avg
POINTS1.5-8B (baseline)	64	324	$\approx$ 20K	36.7	66.1	61.0	71.1	72.0	59.8	61.1
POINTS-Long (standby)	64	16	1024 (5%)	34.6	65.0	59.3	69.1	71.7	58.9	59.8
POINTS-Long (model soup)	64	16	1024 (5%)	35.2	65.9	60.3	69.9	71.6	58.1	60.2 (+0.4)
POINTS-Long (standby)	128	16	2048 (10%)	36.2	66.4	61.0	69.6	72.7	60.3	61.0
POINTS-Long (model soup)	128	16	2048 (10%)	36.7	66.7	61.3	70.4	72.0	60.7	61.3 (+0.3)
POINTS-Long (standby)	64	32	2048 (10%)	35.7	65.7	60.9	70.3	71.6	59.5	60.6

Table 12: Model Soup Performance. We apply model soup (model merge) to two models trained by different learning rates. The model’s performance can further boost in this way.

Figure 5: Visualization of Position Encoding. We initialize learnable standby tokens by uniformly sampling RoPE embeddings from the original sequence. We visualize their attention maps in the last ViT layer, marking assigned positions with a yellow square. For clarity, we display only the top 10% of attention weights, where darker red indicates higher intensity. The results reveal a strong localization effect: standby tokens primarily absorb information from their neighboring patches.

Figure 6: Failure case analysis. Standby mode fails on spatial or fine-grained perception while the baseline fails more on temporal and general understanding.

Multimodal Large Language Models

Multimodal large language models combine language models with image, video, audio, and other inputs for grounded understanding and generation.

See all 1,091 papers

VideoMM: Adaptive Macro-Micro Inference for Efficient Video MLLMs
Sep 15, 2026Haoyu Guo, Yuan Feng, Junlin Lv +3Efficient VLM Inference Long-Video Understanding
School of Biomedical Engineering, University of Science and Technology of China · Data Darkness Lab, MIRACLE Center, Suzhou Institute for Advanced Research · China +1
Scaling the Long Video Understanding of Multimodal Large Language Models via Visual Memory Mechanism
Mar 31, 2026Tao Chen, Kun Zhang, Qiong Wu +5Long-Video Understanding Memory-Augmented Video Understanding
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, 361005, P.R. China. · National University of Defense Technology, 230000, P.R. China.
AdaptToken: Entropy-based Adaptive Token Selection for MLLM Long Video Understanding
Mar 30, 2026Haozhe Qi, Kevin Qu, Mahdi Rad +3Long-Video Understanding Multimodal Large Language Models
Microsoft Spatial AI Lab · EPFL · ETH Zurich

Sep 15, 2026J/K move · Enter open · S save

VideoMM: Adaptive Macro-Micro Inference for Efficient Video MLLMs

Haoyu Guo, Yuan Feng, Junlin Lv, Mingjun Xiao+2

School of Biomedical Engineering, University of Science and Technology of China · Data Darkness Lab, MIRACLE Center, Suzhou Institute for Advanced Research · China+1

Efficient VLM Inference Long-Video Understanding Visual Token Compression

Open paper AskPDF arXiv

Scaling Multimodal Large Language Models (MLLMs) to long-form video understanding is bottlenecked by the explosion of visual tokens, which saturates context windows and incurs prohibitive costs. Current solutions predominantly rely on auxiliary models for token reduction but face a fundamental dilemma: lightweight encoder-driven approaches often overlook critical semantic information, whereas heavyweight MLLM-driven reduction negates the efficiency gains. {In this work, we identify a more fundamental inefficiency underlying this dilemma: while fine-grained visual details are essential for detailed understanding, they are largely redundant for the preliminary task of selecting semantically relevant regions. } Motivated by this, we introduce VideoMM, which marks a paradigm shift from model-centric downsizing to adaptive perceptual granularity. Specifically, our framework {decouples selection from reasoning} by executing semantic filtering on a cost-effective Macro Proxy (derived from downscaled frames), and projecting the selected regions onto high-fidelity Micro Tokens for detailed understanding only when necessary. Extensive evaluations show that VideoMM significantly outperforms existing solutions. It achieves a 6.13

\times

speedup and a 7.4% accuracy gain over full-context baselines on LongVideoBench, and further accelerates inference by 2.73

\times

over current leading methods, establishing a highly scalable paradigm for long-video understanding. Our code is available at: https://github.com/adfh917k/VideoMM.

Abstract

Multimodal Large Language Models

Explore similar work

VideoMM: Adaptive Macro-Micro Inference for Efficient Video MLLMs

Scaling the Long Video Understanding of Multimodal Large Language Models via Visual Memory Mechanism

AdaptToken: Entropy-based Adaptive Token Selection for MLLM Long Video Understanding