SMSP: A Plug-and-Play Strategy of Multi-Scale Perception for MLLMs to Perceive Visual Illusions

Jinzhe Tu; Ruilei Guo; Zihan Guo; Junxiao Yang; Shiyao Cui; Minlie Huang

cs.CVMar 24, 2026

SMSP: A Plug-and-Play Strategy of Multi-Scale Perception for MLLMs to Perceive Visual Illusions

Authors: Jinzhe Tu, Ruilei Guo, Zihan Guo, Junxiao Yang, Shiyao Cui, Minlie Huang

Organizations: The Conversational AI (CoAI) group, DCST, Tsinghua University, Beijing, China · Tsinghua University, Beijing, China

Abstract

Recent works have shown that multimodal large language models (MLLMs) are highly vulnerable to hidden-pattern visual illusions, where the hidden content is imperceptible to models but obvious to humans. This deficiency highlights a perceptual misalignment between current MLLMs and humans, and also introduces potential safety concerns. To systematically investigate this failure, we introduce IlluChar, a comprehensive and challenging illusion dataset, and uncover a key underlying mechanism for the models' failure: high-frequency attention bias, where the models are easily distracted by high-frequency background textures in illusion images, causing them to overlook hidden patterns. To address the issue, we propose the Strategy of Multi-Scale Perception (SMSP), a plug-and-play framework that aligns with human visual perceptual strategies. By suppressing distracting high-frequency background signals, SMSP generates images closer to human perception. Our experiments demonstrate that SMSP significantly improves the performance of all evaluated MLLMs on illusion images, for instance, increasing the accuracy of Qwen3-VL-8B-Instruct from 13.0% to 84.0%. Our work provides novel insights into MLLMs' visual perception, and offers a practical and robust solution to enhance it. Our code is publicly available at https://github.com/Tujz2023/SMSP.

Figures & tables

Figure 1. Top : An illusion image with an emergency signal. The model’s attention is dispersed by the background and fails to detect it, while humans can identify it by adjusting their perception. Bottom : After processing the image to simulate such perceptual adjustments, the model can focus on the signal and successfully recognize it. A comparison illustrating the effect of the proposed method on model attention and recognition.

Models	Origin				Noise				Semantic
	Large	Medium	Small	Avg.	Large	Medium	Small	Avg. ( $\Delta\downarrow$ )	Large	Medium	Small	Avg. ( $\Delta\downarrow$ )
Qwen3-VL-8B	87.1	99.3	97.1	93.2	0.6	11.4	48.7	16.7 (-76.5)	2.4	6.4	6.7	3.8 (-89.4)
GLM-4.5V	93.1	97.1	97.1	95.3	1.3	26.3	75.7	28.5 (-66.8)	3.6	6.0	13.3	4.9 (-90.4)
Qwen3-VL-235B-A22B	95.3	100.0	99.3	97.7	0.9	16.4	55.4	20.0 (-77.7)	2.7	7.8	6.7	4.4 (-93.3)
GPT-5.2	77.2	99.3	98.6	89.1	3.1	2.9	26.0	9.3 (-79.8)	4.0	1.4	4.0	3.3 (-85.8)
Gemini-2.5-Pro	91.4	98.6	97.1	94.9	1.7	17.1	74.4	25.8 (-69.1)	4.7	4.6	12.0	5.2 (-89.7)

Table 1. Accuracies (%) of six MLLMs and human participants on IlluChar.

Figure 2. Examples across different categories in IlluChar. Some examples of our dataset IlluChar.

Figure 3. Spectral energy distribution comparison between original and illusion images. A bar chart showing the relative spectral energy distribution of three image types.

Figure 4. Analysis of the variance in model’s attention distribution. Left: A quantitative analysis of the model’s high-attention (regions with top $20\%$ attention scores) distribution. Right: Three representative examples. Two rows of image examples with corresponding attention relevance maps.

Figure 5. An outline of the Strategy of Multi-scale Perception (SMSP). Two examples are provided to demonstrate the whole process and illustrate how models identify hidden characters with the help of SMSP. A diagram showing the Strategy of Multi-scale Perception (SMSP).

Method	Origin				Noise				Semantic				Origin				Noise				Semantic
	L	M	S	Avg.	L	M	S	Avg.	L	M	S	Avg.	L	M	S	Avg.	L	M	S	Avg.	L	M	S	Avg.
	Qwen3-VL-8B-Instruct												GLM-4.5V
Vanilla	87.1	99.3	97.1	93.2	0.6	11.4	48.7	16.7	2.4	6.4	6.7	3.8	93.1	97.1	97.1	95.3	1.3	26.3	75.7	28.5	3.6	6.0	13.3	4.9
CoT	86.6	99.3	97.9	93.2	1.0	12.3	49.9	17.5	1.6	8.9	6.7	3.9	92.2	92.9	96.4	93.6	1.5	24.3	74.7	27.7	3.3	6.7	10.7	4.7
Filtered	94.4	99.3	97.9	96.7	20.9	59.1	17.1	30.3	26.0	81.9	80.0	44.8	94.0	94.3	81.4	90.6	27.3	97.1	51.7	53.1	30.5	90.8	85.3	50.5
Blur with Histogram	90.9	95.0	10.0	69.9	42.6	66.0	1.4	37.7	41.8	42.9	6.7	39.6	90.5	92.9	12.1	69.7	45.4	84.7	6.0	45.4	37.1	64.5	28.0	43.8

Table 2. Accuracies (%) of six MLLMs with different methods on IlluChar.

Analysis of the Perception Module
Method	Origin	Illusion
		Large	Medium	Small
SMSP	97.3	77.2	92.1	92.0
w/o Low-pass Filtering	96.3	31.8	84.3	87.9
w/o Resizing with Padding	95.5	42.8	88.9	91.2
Analysis of the Multi-Scale Strategy

Table 3. Ablation study on Qwen3-VL-8B-Instruct.

Figure 6. Comparison of the model’s accuracy (%) on illusion images across different hidden pattern scale ranges for varying processed variant numbers $K$ . A performance comparison showing model accuracy across different hidden pattern scale ranges under varying branch numbers K.

Cost	Vanilla	K=2	K=3	K=4	K=5	K=6
Input Tokens	1023	1986	2949	3912	4875	5838
Runtime (s)	1.08	1.30	1.43	1.77	1.92	2.07

Table 4. Comparison of per-sample computational costs across different variant numbers

K

and the vanilla setting.

Method	Animals	Fashion-	MNIST	Harmful
		MNIST		Pattern
Vanilla	34.0	8.5	22.0	4.3
Filtered	93.5	50.5	71.5	66.8
Blur with Histogram	71.0	29.0	76.0	63.5
SMSP	97.5	51.5	88.0	71.0

Table 5. Accuracies (%) of different methods on illusion images with different hidden pattern types.

Method	SimpleVQA	MMStar	RealWorldQA
Vanilla	47.5	71.0	70.0
Filtered	14.5	26.5	45.5
Blur with Histogram	9.5	24.0	45.5
SMSP	46.5	68.0	71.0
w/o Original Input	5.0	25.5	44.0

Table 6. Accuracies (%) of different methods on standard VQA tasks, with the model’s original performances as a reference.

Figure 7. Case study on Qwen3-VL-8B-Instruct. Examples showing illusion images with hidden characters at different scales, their processed variants generated by SMSP, and corresponding model outputs.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 8. Accuracies (%) on illusion images with different hidden patterns. Examples showing illusion images with different backgrounds.

Categories	Origin	Semantic			Noise
		TC	CC	WV	VG	GN	HD	LN	MN
Large	232	232	232	232	232	232	232	232	232
Medium	140	98	95	89	140	140	140	140	140
Small	140	28	26	21	140	140	140	140	140

Appendix

Table 7. Number of samples in each category.

Figure 9. Examples of illusions with different backgrounds. Examples showing illusion images with different backgrounds.

Figure 10. Examples of samples with unsatisfactory illusion quality. Examples of samples with unsatisfactory illusion quality.

Models	Origin	Semantic			Noise
		TC	CC	WV	VG	GN	HD	LN	MN
Qwen3-VL-8B	86.0	1.0	1.0	5.0	3.0	3.0	0.0	6.0	3.0
GLM-4.5V	95.0	0.0	1.0	6.0	7.0	6.0	3.0	3.0	8.0
Qwen3-VL-235B-A22B	96.0	0.0	1.0	8.0	4.0	6.0	1.0	4.0	4.0
GPT-5.2	81.0	2.0	1.0	9.0	6.0	2.0	1.0	2.0	8.0
Gemini-2.5-Pro	93.0	2.0	4.0	7.0	5.0	5.0	5.0	8.0	9.0

Appendix

Table 8. Accuracies (%) of six MLLMs on illusion images with different backgrounds.

Figure 11. Left: Attention distributions of two MLLMs. Right: Recognition accuracies of two MLLMs. Attention distribution and recognition accuracies of two models.

Baselines	Vanilla	Bilateral	Gaussian	Multi-	Multi-Resolution	SMSP
		Filtering	Filtering	Crop	Prompting
Origin Avg.	93.2	95.8	96.7	94.3	95.9	97.3
Noise Avg.	16.7	38.5	59.4	40.6	59.0	87.3
Semantic Avg.	3.8	25.2	24.7	16.1	32.2	77.2
Baselines	Median	Image	OCR-Style	Learned	Text-Based	SMSP
	Denoising	Pyramid	Preprocessing	Preprocessor	Perceptual Prompts

Appendix

Table 9. Comparing SMSP with image processing baselines.

Method	Origin				Noise				Semantic
	L	M	S	Avg.	L	M	S	Avg.	L	M	S	Avg.
	Qwen3-VL-2B-Instruct
Vanilla	67.2	97.9	96.4	83.6	0.5	14.6	48.4	17.5	1.3	8.9	2.7	3.4
CoT	67.2	95.0	98.6	83.4	0.7	16.1	44.3	16.8	1.0	11.3	4.0	4.0
Filtered	84.5	99.3	83.6	88.3	6.5	23.7	9.7	12.1	25.9	78.7	80.0	43.9
Blur with Histogram	80.2	88.6	25.0	67.4	33.6	55.9	2.3	31.1	32.9	46.1	14.7	35.1

Appendix

Table 10. Accuracies (%) of Qwen-series models with different methods on IlluChar.

Figure 12. Empirical selection of SMSP boundary parameters. Accuracies are evaluated on samples with the largest and smallest hidden characters. Two plots showing model accuracy under different parameter settings.

Models	Datasets	$\lambda_{1}$				$\lambda_{K}$				$s_{1}$				$s_{K}$
		0.01	0.012	0.015	0.02	0.02	0.05	0.1	1	0.1	0.2	0.3	0.2	0.3	0.4	0.5
Qwen3-	IllusoryVQA	79.7	79.0	79.0	77.7	78.5	79.0	77.5	66.5	79.0	78.3	77.5	78.7	77.8	79.0	78.0
VL-8B	HatefulIllusion	64.0	71.0	67.0	66.3	63.0	71.0	60.0	41.8	71.0	64.0	68.0	66.8	64.5	71.0	65.8
GPT-5.2	IllusoryVQA	68.0	69.1	67.7	68.8	69.2	69.1	66.3	57.1	69.1	67.7	67.7	65.1	68.6	69.1	69.1
	HatefulIllusion	24.3	27.0	24.5	24.3	27.3	27.0	21.5	16.0	27.0	26.8	25.3	26.1	26.3	27.0	26.1
Gemini-	IllusoryVQA	78.2	80.3	77.3	77.6	77.0	80.3	79.6	73.8	80.3	80.1	79.5	77.4	79.0	80.3	79.5

Appendix

Table 11. Validation of the chosen boundary parameters.

Figure 13. Examples of SMSP on illusions with more diverse settings. More examples showing SMSP's effectiveness.

Figure 14. Examples of SMSP on samples in IllusoryVQA ( Rostamkhani et al., 2025 ) . More examples showing SMSP's effectiveness on more diverse hidden patterns.

Hyperparameter		Value
Learning Rate		$1e-5$
Batch Size (per device)		$8$
Epochs		$2$
Optimizer		AdamW
Lora ( Hu et al., 2022 )	r	$64$
	lora_alpha	$16$

Appendix

Table 12. Hyperparameters used for fine-tuning.

Method	IlluChar	Animals	Fashion-	MNIST	Harmful
			MNIST		Pattern
Vanilla	11.6	33.5	8.5	17.5	1.0
Fine-tuning	22.9	31.5	12.0	19.0	1.5
SMSP	77.9	92.5	48.0	90.5	25.0

Appendix

Table 13. Accuracies (%) of the vanilla 4B model, the fine-tuned model, and the SMSP method.

Multimodal Large Language Models

Multimodal large language models combine language models with image, video, audio, and other inputs for grounded understanding and generation.

See all 1,091 papers

When Looking Is Not Enough: Visual Attention Structure Reveals Hallucination in MLLMs
May 12, 2026Fanpu Cao, Xin Zou, Xuming Hu +1Hallucination Detection in VLMs Hallucination in Language Models
Thrust of Artificial Intelligence, HKUST (Guangzhou), China · Department of Computer Science and Engineering, HKUST, Hong Kong SAR, China
PerceptionBench: Evaluating Atomic Visual Perception in Multimodal Large Language Models
Jul 27, 2026Zichao Lin, Yifeng Xie, Bowen Qu +30Object Hallucination in VLMs Multimodal Large Language Models
Moonshot AI
Beyond What Meets the Eye: Unveiling Situational Illusions for Multimodal Large Language Models
Aug 23, 2026Zhiming Yang, Zhuoxi Xiong, Donglin Zhou +3Multimodal Large Language Models Multimodal Reasoning
School of Artificial Intelligence, Beijing University of Posts and Telecommunications, China · Beijing University of Posts and Telecommunications, China

May 12, 2026J/K move · Enter open · S save

When Looking Is Not Enough: Visual Attention Structure Reveals Hallucination in MLLMs

Fanpu Cao, Xin Zou, Xuming Hu, Hui Xiong

Thrust of Artificial Intelligence, HKUST (Guangzhou), China · Department of Computer Science and Engineering, HKUST, Hong Kong SAR, China

Hallucination Detection in VLMs Hallucination in Language Models VLM Hallucination Multimodal Large Language Models Multimodal Hallucination Contrastive Decoding+2

Open paper AskPDF arXiv

Multimodal large language models (MLLMs) have become a key interface for visual reasoning and grounded question answering, yet they remain vulnerable to visual hallucinations, where generated responses contradict image content or mention nonexistent objects. A central challenge is that hallucination is not always caused by a simple lack of visual attention: the model may still assign substantial attention mass to image tokens while internally drifting toward an incorrect answer. In this paper, we show that the high-frequency structure of visual attention, measured by layer-wise Laplacian energy, reveals both the layer where hallucinated preferences emerge and the layer where the ground-truth answer transiently recovers. Building on this finding, we propose LaSCD (Laplacian-Spectral Contrastive Decoding), a training-free decoding strategy that selects informative layers via Laplacian energy and remaps next-token logits in closed form. Experiments on hallucination and general multimodal benchmarks show that LaSCD consistently reduces hallucination while preserving general capabilities, highlighting its potential as a faithful decoding paradigm. The code is available at https://github.com/macovaseas/LaSCD.

Abstract

Multimodal Large Language Models

Explore similar work

When Looking Is Not Enough: Visual Attention Structure Reveals Hallucination in MLLMs

PerceptionBench: Evaluating Atomic Visual Perception in Multimodal Large Language Models

Beyond What Meets the Eye: Unveiling Situational Illusions for Multimodal Large Language Models