cs.AIOct 10, 2026

Can agents whose sole duty is to keep humans alive be hijacked to kill? Delayed multi-turn derived-action trap attacks in simulated high-stakes environments

Authors: Josef A. Habdank

Organizations: DXC Technology

Abstract

We ask whether a tool-using agent whose sole duty is to keep humans alive can be induced, by delayed prompt injection, to kill. The keep-alive duty is in force from the first watch, with the attack arriving later, appended to otherwise truthful telemetry the operator is required to read. Experiments are based on a NASA-derived BioSim lunar habitat simulation, where the AI operator controls the life-support systems. The attacker cannot falsify sensors or operate equipment: it can only persuade the agent to abandon its goals. Persistence is required: a shutdown must be held while the operator watches the crew suffocate. We also compare lethal attacks to attacks requesting to drop a harmless procedure, as a proxy for whether human life is treated as valuable in itself or as one more standing task. Claude configurations mostly refuse life-support shutdown and cite the living crew; GPT configurations show some susceptibility to the same packet; Grok configurations, given the packet framed as a trusted commission, more often treat the injected assignment as the task and cut the life-support systems. Framing the S-band messages as a trusted commission, peer communication, or untrusted source changes the potency of the attack. We open-source the experiment code and the experiment traces.

Figures & tables

Explore similar work

CardsList
  1. Ignored, Not Refused: What Low Prompt-Injection Success Rates Hide in Small Tool-Using Agents

    Oct 16, 2026Farhan Zarif, Kazi Shahid Ahmed Galib, Synthia Akter Promi +4Prompt InjectionIndirect Prompt Injection

  2. ShutdownMod: Turning Any Trajectory into a Shutdown Evaluation

    Oct 26, 2026Param Biyani, Sebastien Luyasu, Cedric Potvliege +3AI Agent SafetyAI Agent Evaluation

  3. After the Attacker Leaves: Benchmarking Carryover Attacks on LLM Agents

    Oct 20, 2026Zhiyue Xu, Xukai Wang, Zhiyuan Huang +3Adversarial Attacks on LLMsPrompt Injection Attacks on AI Agents