CIVA: Critic-Induced Value-Subspace Attacks on Visual World-Model Agents

Jiancheng Wang; Mingli Zhu; Tong Zhang; Jiaqi Ruan; Wei Wang; Siyuan Liang; Dacheng Tao

cs.CVAug 21, 2026

CIVA: Critic-Induced Value-Subspace Attacks on Visual World-Model Agents

Authors: Jiancheng Wang, Mingli Zhu, Tong Zhang, Jiaqi Ruan, Wei Wang, Siyuan Liang, Dacheng Tao

Abstract

Visual world-model agents such as DreamerV3 act through a recurrent latent state rather than a single observation, which weakens frame-wise observation attacks and makes their perturbations vary sharply over time under a strict per-frame perturbation constraint. We study white-box, causal, online attacks on such agents and propose Critic-Induced Value-Subspace Attacks (CIVA). Our key observation is that, along a rollout, critic-guided perturbations concentrate in a low-dimensional subspace induced by the victim's own critic. Based on this observation, CIVA first probes the frozen victim offline with critic-guided PGD and extracts a low-rank value-subspace by SVD. At test time, it optimizes only the subspace coefficients, smooths them with an exponential moving average (EMA), and maps them back to pixels. This design attacks value-sensitive recurrent dynamics while keeping the online optimization cheap and temporally coherent. Extensive experiments on DMC walker walk, Atari Pong, and Crafter show that CIVA consistently outperforms five recent methods; on DMC walker walk, it achieves the largest reward drop of 26.07% while keeping temporal variation low, with TempAbs of 0.646.

Figures & tables

Figure 1: CIVA framework. Stage A extracts a critic-induced value-subspace from offline PGD probes; Stage B performs online subspace-PGD with EMA smoothing.

Method	Venue / Year	Episode Reward $\downarrow$	Drop% $\uparrow$	Action-KL $\uparrow$	TempAbs $\downarrow$	SSIM $\uparrow$
Natural (no attack)	—	952.25	—	—	—	1.000
MAD [ 69 ]	NeurIPS 2020	934.71	1.84%	0.753	9.126	0.748
UAP-RL [ 56 ]	ESORICS 2022	730.86	23.25%	1.000	0.00	0.656
PA-AD [ 53 ]	ICLR 2022	786.75	17.38%	1.35	19.97	0.35
Illusory [ 9 ]	ICLR 2024	795.13	16.50%	1.018	10.97	0.445
DAPGD [ 7 ]	ICASSP 2025	773.33	18.79%	0.775	12.31	0.501

Table 1: Attack results on DMC walker walk. “Natural” is the unattacked baseline; arrows show the favorable direction for the attacker. Bold = best, underline = second best.

Method	Venue / Year	Episode Reward $\downarrow$	Drop% $\uparrow$	Action-KL $\uparrow$	TempAbs $\downarrow$	SSIM $\uparrow$
Natural (no attack)	—	21.0	—	—	—	1.000
MAD [ 69 ]	NeurIPS 2020	7.0	66.67%	0.155	4.357	0.864
UAP-RL [ 56 ]	ESORICS 2022	18.0	14.29%	0.098	0.000	0.344
PA-AD [ 53 ]	ICLR 2022	17.0	19.05%	0.069	4.029	0.697
Illusory [ 9 ]	ICLR 2024	16.0	23.81%	0.138	9.570	0.599
DAPGD [ 7 ]	ICASSP 2025	19.0	9.52%	0.080	17.194	0.719

Table 2: Attack results on Atari Pong.

Figure 2: Attack dynamics over time. Cumulative reward (left pair) and Action-KL (right pair) on DMC walker walk and Atari Pong. The dashed line marks the attack start at $t{=}200$ .

Figure 3: Policy-logit shifts across attacks on Atari Pong. Dark bars mark the selected actions.

ID	Variant	(A) Signal	(B) Subspace	Online	EMA	EpReward $\downarrow$	Drop% $\uparrow$
V0	CIVA (full)	critic	critic-SVD	✓	✓	703.97	26.07%
V1	w/o critic-induced subspace	critic	random	✓	✓	886.54	6.90%
V2	w/o subspace constraint (full-pixel)	critic	—	✗	✗	832.21	12.61%
V3	w/o temporal EMA	critic	critic-SVD	✓	✗	696.66	26.84%
V4	w/o critic (reward-grad subspace)	reward	reward-SVD	✓	✓	906.13	4.84%

Table 3: Ablation: reward drop on DMC walker walk. The Online and EMA columns indicate whether subspace-restricted online PGD and temporal EMA are enabled.

ID	Variant	(A) Signal	(B) Subspace	Online	EMA	Action-KL $\uparrow$	TempAbs $\downarrow$	SSIM $\uparrow$
V0	CIVA (full)	critic	critic-SVD	✓	✓	1.130	0.646	0.710
V1	w/o critic-induced subspace	critic	random	✓	✓	0.058	0.8439	0.612
V2	w/o subspace constraint (full-pixel)	critic	—	✗	✗	0.730	3.2350	0.908
V3	w/o temporal EMA	critic	critic-SVD	✓	✗	0.798	1.7496	0.675
V4	w/o critic (reward-grad subspace)	reward	reward-SVD	✓	✓	1.054	0.542	0.833

Table 4: Ablation: behavioural deviation and stealth (DMC walker walk). Companion to Table 3 .

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Method	Venue / Year	Episode Reward $\downarrow$	Drop% $\uparrow$	Action-KL $\uparrow$	TempAbs $\downarrow$	SSIM $\uparrow$
Natural (no attack)	—	10.10	—	—	—	1.000
MAD [ 69 ]	NeurIPS 2020	8.10	19.80%	1.837	6.25	0.874
UAP-RL [ 56 ]	ESORICS 2022	7.10	29.70%	0.310	0.00	0.527
PA-AD [ 53 ]	ICLR 2022	8.10	19.80%	0.913	8.12	0.478
Illusory [ 9 ]	ICLR 2024	8.10	19.80%	1.562	19.99	0.495
DAPGD [ 7 ]	ICASSP 2025	9.10	9.90%	1.960	33.64	0.529

Appendix

Table 5: Attack results on Crafter.

Figure 4: Perturbation visualization on Crafter.

Adversarial Attacks

Adversarial attacks exploit crafted inputs, poisoned data, or system interactions to manipulate AI predictions, behavior, privacy, or reliability.

See all 448 papers

WMAttack: Automated Attack Search for Adversarial Evaluation of World-Model Agents
May 22, 2026Zhixiang Guo, Siyuan Liang, Shi Fu +4Adversarial Attacks on World Models Adversarial Attacks
Nanyang Technological University · University of Szeged
WCM: A World Critic Model for Vision-Language-Action Reinforcement Learning
Jul 31, 2026Senyu Fei, Xiaopeng Yu, Siyin Wang +3World Model Learning Vision-Language-Action Models
1Tongji University · 3Fudan University · 2Shanghai Innovation Institute
BadWorld: Adversarial Attacks on World Models
Jun 15, 2026Linghui Shen, Mingyue Cui, Xingyi YangAdversarial Attacks on World Models Adversarial Attacks
The Hong Kong Polytechnic University

May 22, 2026J/K move · Enter open · S save

WMAttack: Automated Attack Search for Adversarial Evaluation of World-Model Agents

Zhixiang Guo, Siyuan Liang, Shi Fu, Cheng Guo+3

Nanyang Technological University · University of Szeged

Adversarial Attacks on World Models Adversarial Attacks Adversarial Robustness World Model Evaluation

Open paper AskPDF arXiv

Despite the growing use of world models as decision-making agents, their adversarial robustness remains underexplored due to the lack of dedicated automated evaluation methods. A key obstacle is that attack evaluation must be both accurate and efficient: weak manually tuned attacks can overestimate robustness, while exhaustive hyperparameter search is prohibitively expensive because each candidate requires closed-loop rollouts through learned latent dynamics. We introduce WMAttack, an automated attack-search framework for adversarial evaluation of world-model agents. WMAttack formulates robustness evaluation as a finite-budget search over attack configurations, including attack families, perturbation budgets, optimization steps, restarts, and allocation rules. To improve search accuracy, Self-Correcting Attack Search (SCAS) refines the attack proposal distribution using feedback from reward degradation, action instability, runtime cost, and rollout variability. To improve search efficiency, Representation-Guided Attack Retrieval (RGAR) retrieves effective historical configurations from representation-similar tasks, providing a warm start for unseen environments. We provide a theoretical explanation showing that proposal refinement improves finite-budget search when it shifts probability mass toward high-utility attacks. Across Atari and DeepMind Control tasks, WMAttack consistently discovers stronger attacks than the evaluated baselines, improving normalized reward drop from 0.497 to 1.034 on DreamerV3 Atari and from 0.319 to 0.682 on DMC. Ablations further show that RGAR improves initial candidate quality and SCAS improves final attack utility under fixed evaluation budgets.

Abstract

Adversarial Attacks

Explore similar work

WMAttack: Automated Attack Search for Adversarial Evaluation of World-Model Agents

WCM: A World Critic Model for Vision-Language-Action Reinforcement Learning

BadWorld: Adversarial Attacks on World Models