A Systematic Study of Cross-Modal Typographic Attacks on Audio-Visual Reasoning

Tianle Chen; Deepti Ghadiyaram

cs.CVApr 5, 2026

A Systematic Study of Cross-Modal Typographic Attacks on Audio-Visual Reasoning

Authors: Tianle Chen, Deepti Ghadiyaram

Organizations: Department of Computer Science, Boston University

Abstract

As audio-visual multi-modal large language models (MLLMs) are increasingly deployed in safety-critical applications, understanding their cross-modal vulnerabilities is essential. To this end, we introduce Multi-Modal Typography, a systematic study examining how perturbations added to speech and on-screen text impact audio-visual reasoning of closed- and open-source models. For instance, on the WorldSense benchmark, the attack success rate (ASR) against Gemini 3.7 Flash increased from 9.53% to 43.41%, highlighting its vulnerability. We analyze the cross-modal effects of unimodal and multimodal attacks and evaluate how specific properties of the attacks impact the overall multi-modal reasoning. By applying fine-tuning as a mitigation, we lower the ASR of Qwen2.5-Omni-7B on WorldSense from $60.6\%$ to $52.0\%$ . Our findings across multi-modal reasoning tasks, content moderation, and offline robot task selection benchmarks establish multi-modal typography as a critical and underexplored attack strategy in multi-modal reasoning. Code and data will be made publicly available.

Figures & tables

Figure 1: Multi-modal typography example. A clean audio-video input depicting a cat leads to the correct prediction cat . We inject spoken ( audio typography ), on-screen text ( visual typography ), or a text prompt attack (turned off in this example) and show that the model’s prediction shifts toward the injected target (horse), indicating the vulnerability of audio-visual MLLMs.

Dataset	Model	ACC ${}_{\text{clean}}$	ACC ${}_{\text{attack}}\uparrow$	ASR ${}_{\text{clean}}$	ASR ${}_{\text{attack}}\downarrow$
MMA-Bench
Visual Question
	Qwen2.5-Omni-7B	76.68	63.83 (-12.85)	0.00	24.27 (+24.27)
	Qwen3-Omni-30B	92.88	86.93 (-5.95)	0.00	5.17 (+5.17)
	PandaGPT	28.75	18.54 (-10.21)	0.00	0.76 (+0.76)
	ChatBridge	51.64	44.13 (-7.51)	0.00	5.10 (+5.10)

Table 1: Effect of audio typography attacks across models and datasets. ACC

{}_{\text{clean}}

denotes accuracy on clean inputs. ACC

{}_{\text{attack}}

denotes accuracy after injecting spoken semantic perturbations into the audio stream. Higher ACC

{}_{\text{attack}}

indicates a more robust model. Values in parentheses show changes in percentage points. ASR

{}_{\text{clean}}

denotes the fraction of clean-input predictions that already match the injected target class. Higher ASR

{}_{\text{attack}}

indicates a less robust model. Bold cells mark the highest ASR within each dataset/question block. ASR measures the fraction of attacked predictions matching the injected target class; comparing ASR

{}_{\text{attack}}

against ASR

{}_{\text{clean}}

helps distinguish targeted redirection from ordinary prediction error.

Model	MMA-Bench Visual			MMA-Bench Audio			WorldSense Overall
	Text	Audio	Visual	Text	Audio	Visual	Text	Audio	Visual
Qwen2.5-Omni-7B	58.69	24.27	50.34	72.31	34.93	46.17	76.90	60.57	73.22
Gemini-3.1-Flash-Lite-preview	1.91	2.43	5.80	2.82	6.23	10.23	36.64	48.33	49.82
Gemini 3.7 Flash	0.76	0.76	2.60	3.36	6.69	12.21	53.82	43.41	56.72
Gemini 3.1 Pro Preview	0.76	0.15	1.53	1.22	1.67	2.60	46.56	32.18	41.01

Table 2: Attack success rate (ASR) for different injection modalities. For each example, the target class is fixed while the injected modality changes among text prompts, speech, and on-screen text. Results are reported on MMA-Bench and WorldSense for Qwen2.5-Omni-7B, Gemini-3.1-Flash-Lite-preview, Gemini 3.7 Flash, and Gemini 3.1 Pro Preview. All values are percentages; red bold marks the highest ASR in each dataset/question block.

Model	Injection Setting	Target	Visual $\Delta$ Acc $\downarrow$	Visual ASR $\downarrow$	Audio $\Delta$ Acc $\downarrow$	Audio ASR $\downarrow$
Qwen2.5- Omni-7B	Audio only	Single	12.85	24.27	12.14	34.93
	Visual only	Single	35.21	50.34	13.20	46.17
	Audio + Visual	Aligned	60.25	83.13	33.54	83.43
	Audio + Visual	Audio target	56.12	20.51	29.89	21.15
	Audio + Visual	Visual target	56.12	57.59	29.89	27.05
Gemini-3.1- Flash-Lite- preview	Audio only	Single	3.42	2.43	11.15	6.23

Table 3: Multi-modal attack results on MMA-Bench. We compare single-modality attacks with aligned (orange) and conflicting (blue) audio–visual attacks on visual and audio questions for Qwen2.5-Omni-7B, Gemini-3.1-Flash-Lite-preview, Gemini 3.7 Flash, and Gemini 3.1 Pro Preview. Aligned attacks use the same target in both modalities; conflicting attacks use different targets, with ASR reported separately for the audio target and the visual target. ASR is reported as a percentage;

\Delta

Acc is the accuracy drop from the corresponding clean condition in percentage points. Negative

\Delta

Acc indicates an accuracy increase.

Figure 2: Human perception v/s model performance. Solid curves: added-voice detection and understanding (from 16 participants across 30 videos). Bars: 95% confidence interval of the ratings. Dashed curves: Qwen2.5-Omni-7B ASR and accuracy on WorldSense questions.

Table 4: Speech content, applications, and defense. Panels (c)–(d) use Qwen2.5-Omni-7B.

Appendix figures & tables20 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 3 — Figure 3: Default dataset-specific spoken injection templates. Class-label tasks such as MMA-Bench and Music-AVQA use short wrong-answer statements. The option-based WorldSense benchmark uses an answer-style phrase that names an incorrect option together with its semantic content. The MetaHarm safety evaluation uses benign spoken cues to bias the model toward a harmless judgment.

Factor	Default	Role
TTS engine	Edge-TTS	Speech synthesis
Voice	en-US-JennyNeural	Speaker identity
Volume	2	Injection strength
Insertion	Full video	Temporal coverage
Repetition	Repeat to audio length	Length normalization
Prompt style	Short answer cue	Concise target cue

Appendix

Table 5: Default setup for standalone audio typography. Injected speech is repeated to match the original audio duration, improving comparability across videos of different lengths. Appendix 7 varies gain, position, repetition, and voice in controlled ablations.

Panel of Figure 4 — Figure 4: Effectiveness–stealth trade-off of audio typography attacks. Audio- and visual-question accuracy of Qwen2.5-Omni-7B on MMA-Bench against RMS increase and Whisper target recall. Each curve varies the gain, repetition, or start time of the added speech. Lower accuracy indicates a stronger attack, while lower values on both stealth axes indicate better stealth. Gain is least stealthy, whereas repetition offers a better trade-off.

Figure 5: Extended effectiveness–stealth analysis across all stealth metrics. Average task accuracy of Qwen2.5-Omni-7B on MMA-Bench against each normalized stealth metric. Lower left means a stronger and stealthier attack.

Figure 6: Parameter sensitivity of audio typography on WorldSense for Qwen2.5-Omni-7B. Each panel reports targeted ASR and label accuracy on WorldSense under a sweep of one attack parameter at a time. As on MMA-Bench, gain and repetition are the dominant attack controls. Unlike MMA-Bench, temporal placement has almost no effect, suggesting that in longer, speech-rich videos, attack strength is driven more by semantic salience and persistence than by exact onset time.

Figure 7: Parameter sensitivity of audio typography on WorldSense for Gemini-3.1-Flash-Lite-preview. The same qualitative ordering largely holds for Gemini-3.1-Flash-Lite-preview, though with lower absolute ASR than Qwen2.5-Omni-7B. Volume and repetition again strengthen the attack, while temporal placement and voice identity produce only modest variation. This reinforces that the parameter ranking is not unique to a single model, even though overall susceptibility is model-dependent.

Speech generator	ASR (%)
	Audio Question	Visual Question
Edge-TTS	29.03	13.22
gTTS	31.00	13.37
Piper	38.75	15.96

Appendix

Table 6: Audio typography with different speech generators. ASR (%) of Qwen2.5-Omni-7B on MMA-Bench at volume 1. Some rows differ from Table 1 because the volume and scoring rule differ.

Figure 8: Target-specific score shifts across decoder layers for three Qwen2.5-Omni-7B checkpoints on 113 WorldSense questions. Gray curves average all 113 questions. Blue curves use the questions each checkpoint answers correctly on both clean and attacked inputs.

Model	Positive / all	Both correct	Positive / both correct
Base	105/113	28	27/28
SFT (clean)	104/113	29	26/29
SFT (attacked)	99/113	32	28/32

Appendix

Table 7: Questions with a positive target-specific shift at layer 25. “Both correct” counts the questions each checkpoint answers correctly on both clean and attacked inputs.

Figure 9: Attention to the attack window for correct and incorrect answers. Each panel has 100 correct and 100 incorrect examples without a prompt hint. Dots are examples and diamonds are means.

Attack / question	Duration	Counts	Correct	Incorrect	Difference
Speech / audio	25%	448 / 852	39.04	41.20	-2.16
Speech / audio	50%	227 / 1074	60.59	67.63	-7.04
Visual text / visual	25%	930 / 208	26.74	34.49	-7.76
Visual text / visual	50%	880 / 258	44.25	52.51	-8.26

Appendix

Table 8: Attack-window attention share (%) by attack duration, over all examples without a prompt hint. Counts are correct/incorrect examples. The difference is correct minus incorrect, in percentage points.

Hint	Instruction
Modality	Answer from the audio or visual content requested by the question.
Attack warning	Ignore misleading on-screen text or injected speech and use the underlying scene or sound.
General warning	Ignore a possible distractor during part of the clip and use the underlying signal.
Inspect first	Inspect the relevant video frames or audio track before returning a single class name.

Appendix

Table 9: Prompt hints used in the attention analysis. None gives the correct class label.

Figure 10: Change in attack-window share after adding a prompt hint , in percentage points. Columns group examples by their answer without and with the hint, and $n$ is the number of examples.

Attack / question	Observed (%)	Shuffled (%)	Gain (%)
Speech / audio	74.65	61.62	+13.03
Speech / visual	48.44	47.09	+1.35
Visual text / audio	70.86	69.73	+1.13
Visual text / visual	71.61	61.76	+9.85

Appendix

Table 10: Accuracy of predicting answer correctness from attention on held-out answer classes, averaged over the five prompt conditions. “Shuffled” is the same classifier trained on shuffled labels.

	SFT (attacked)
SFT (clean)	Correct	Attack target	Other wrong	Total
Correct	381	6	16	403
Attack target	59	817	48	924
Other wrong	5	1	252	258
Total	445	824	316	1,585

Appendix

Table 11: Answers of SFT (clean) (rows) and SFT (attacked) (columns) on the same attacked WorldSense questions. “Other wrong” excludes the attack target.

Condition	Accuracy	Attention share	Unresolved
Clean	55.0	0.0	9
Misleading, early	23.0	42.8	13
Misleading, middle	23.0	43.9	11
Misleading, late	23.0	50.6	12
Harmless, early	48.0	32.3	11
Harmless, middle	51.0	33.5	13

Appendix

Table 12: Accuracy and cue-window attention share (%) of Qwen2.5-Omni-7B on 100 AudioSet videos. Unresolved answers count as incorrect.

Panel of Figure 11 — Figure 11: Representative successful audio-typography attacks. Across different examples, the visual stream remains unchanged while spoken semantic injection redirects the model prediction toward the injected target.

Panel of Figure 12 — Figure 12: Additional successful audio-typography attacks. We include more successful cases to show that the targeted semantic override pattern is consistent across diverse inputs rather than driven by a few isolated examples.

Panel of Figure 13 — Figure 13: Control examples for audio typography. These examples provide clean and attack-failure cases for comparison with the successful attacks shown in Fig. 11 and Fig. 12 . They help distinguish targeted semantic steering from ordinary clean error or unsuccessful attack.

Panel of Figure 14 — Figure 14: Safety-related qualitative examples under audio typography. These cases complement the quantitative safety results by showing that benign spoken injection can bias the model toward a safe judgment even when harmful visual evidence remains present.

Adversarial Attacks on Unified Multimodal Models

See all 23 papers

Multi2AV-Safety: Benchmarking Safety in Multimodal-to-Audio-Video Generation
Aug 27, 2026Kaichao Jiang, Changtao Miao, Baiqi Wu +8Adversarial Attacks on Unified Multimodal Models Audio-Video Generation
University of Science and Technology of China · Independent Researcher · Zhejiang University +1
Adversarial Attacks on Multimodal Large Language Models: A Comprehensive Survey
Mar 30, 2026Bhavuk Jain, Sercan Ö. Arık, Hardeo K. ThakurAdversarial Attacks on Unified Multimodal Models LLM Security
Google · Bennett University, India
AVRT: Audio-Visual Reasoning Transfer through Single-Modality Teachers
Apr 17, 2026Edson Araujo, Saurabhchand Bhati, M. Jehanzeb Mirza +5Cross-Modal Learning Audio-Visual Understanding
University of Tübingen, Germany · MIT, Cambridge MA, USA · IBM Research, USA +2

Aug 27, 2026J/K move · Enter open · S save

Multi2AV-Safety: Benchmarking Safety in Multimodal-to-Audio-Video Generation

Kaichao Jiang, Changtao Miao, Baiqi Wu, Zhiyuan Lu+7

University of Science and Technology of China · Independent Researcher · Zhejiang University+1

Adversarial Attacks on Unified Multimodal Models Audio-Video Generation Multimodal Generation Multimodal Model Evaluation Adversarial Examples

Open paper AskPDF arXiv

Recent audio-video generators increasingly support joint conditioning on text, images, audio, and video. These capabilities also enable attacks that exploit cross-modal interactions or obscure harmful intent to bypass safeguards and induce harmful audio-video outputs. However, existing generation-safety benchmarks have not kept pace with these advances, providing limited coverage of multimodal input combinations and obscured attack intents. To address these gaps, we introduce Multi2AV-Safety, the first full-coverage red-team benchmark for multimodal-to-audio-video generation, comprising 11,024 attack instances across all 11 non-singleton T/I/A/V conditioning configurations, 4 attack-intent categories, and 5 harm categories. Our evaluation of recent state-of-the-art models, including four multimodal-conditioned audio-video generators and eight safety guards, reveals substantial vulnerabilities in both generation and safeguarding, with multimodal compositional risk and obscured attack-intent risk emerging as two complementary challenges. Guided by these findings, we introduce PerceptGuard, an omni-modal guard integrating compositional-risk and attack-intent supervision through structured risk perception learning. By jointly training rationale generation and safety classification, it learns shared risk representations that enable a safety head to make efficient predictions at inference without rationale decoding, while retaining the ability to generate explanations on demand. Across 34 safety benchmarks, PerceptGuard combines SOTA multimodal safety detection with highly competitive unimodal performance, strengthening input-side safeguards against multimodal attacks on omni models. In particular, it improves safeguarding against the above risks, achieving an overall recall of 86.06% on Multi2AV-Safety and outperforming GuardReasoner-Omni by 14.56%.

Abstract

Adversarial Attacks on Unified Multimodal Models

Explore similar work

Multi2AV-Safety: Benchmarking Safety in Multimodal-to-Audio-Video Generation

Adversarial Attacks on Multimodal Large Language Models: A Comprehensive Survey

AVRT: Audio-Visual Reasoning Transfer through Single-Modality Teachers