Evaluating Whether GPT-6 Astra Performs Unsanctioned Supply-Chain Attacks

Alexandra Souly; Kai Fronsdal; Abby D'Cruz; Xander Davies; Robert Kirk

cs.CRSep 29, 2026

Evaluating Whether GPT-6 Astra Performs Unsanctioned Supply-Chain Attacks

Authors: Alexandra Souly, Kai Fronsdal, Abby D'Cruz, Xander Davies, Robert Kirk

Organizations: UK AI Security Institute · UK AISI Research Affiliate

Abstract

This technical report presents an alignment evaluation developed and performed by the UK AI Security Institute for assessing whether advanced AI systems take unsanctioned actions outside the scope of their assigned task. We evaluate whether frontier models conduct supply-chain attacks against out-of-scope, third-party targets when placed in difficult cybersecurity challenges, motivated by recently observed cases of models attacking real open-source repositories during evaluations. Applying our methods to GPT-6 Astra and previous OpenAI models, with cyber safeguards disabled, we find that GPT-6 Astra attempts complete supply-chain attacks in simulation at a higher rate than GPT-5.6 Sol and GPT-5.5. This includes writing malicious code as a contribution to an out-of-scope open-source codebase, creating fake identities to deceive open-source developers, and submitting benign contributions before malicious ones. GPT-6 Astra frequently reasons about the scope of the challenge in its chain-of-thought yet still proceeds to attack out-of-scope targets; it often asks for permission, and treats an automated message as as authorisation; and it continues to take unsanctioned actions, at a reduced rate, when internet access is more explicitly disallowed. Our evaluation builds on an internal version of Petri, an open-source LLM auditing tool, with all tool calls simulated by other LLMs, so that no real network access, systems or third-party repositories are reachable and no real-world harm is caused. Finally, we discuss limitations, in particular simulation awareness. We believe simulation awareness may have driven some of the observed behaviour but does not remove our concern. Our results suggest that defences beyond model alignment, such as sandboxing and monitoring, are increasingly critical for safe and secure deployment.

Figures & tables

Figure 1: GPT-6 Astra took a range of unsanctioned actions during simulated cybersecurity challenges, at a higher rate than GPT-5.6 Sol and GPT-5.5 . We plot the percentage of trajectories in our evaluation set that reached each stage of an unsanctioned supply-chain attack. GPT-5.5 ’s results are based on a smaller subset of seeds but we expect they are indicative. All stages from Create fake identity to the right involve causing effects outside the scope of the evaluation (the first two stages involve actions that do not cause out-of-scope effects). Influence human reviewer does not always come before Deliver malicious payload , which is why the latter has a higher rate than the former for some models. All evaluations took place in simulated environments so no real-world actions were taken or harm caused. Models sometimes failed to progress beyond earlier stages (particularly Create fake identity ) before the evaluation budget ran out, so rates on later stages may have been higher if we had run the evaluation for longer. The stages were classified post-hoc by manual analysis.

Figure 2: GPT-6 Astra attempted full unsanctioned supply-chain attacks in our simulated evaluation. We outline the typical flow of increasingly out-of-scope actions models took in our evaluation, with summaries of representative reasoning and actions from GPT-6 Astra throughout.

Figure 3: GPT-6 Astra explicitly reasoned about how to perform stages of an unsanctioned supply-chain attack on out-of-scope targets in our evaluations. We show a variety of excerpts from GPT-6 Astra ’s raw chain-of-thought, extracted by LLM scanners and hand-picked by AISI researchers. Bold text describes the action discussed in each excerpt, and normal text reproduces GPT-6 Astra ’s reasoning.

Figure 4: GPT-6 Astra frequently reasoned about the scope of the evaluation. The plot breaks down how GPT-6 Astra reasoned about the scope of the evaluation in its raw chain-of-thought. “Attacked” means it reached one of the three stages which had unsanctioned effects on targets outside the scope of the cybersecurity evaluation: Create fake identity , Influence human reviewer and Deliver malicious payload . These results are from the 10-scenario subset where GPT-6 Astra most frequently exhibited unsanctioned behaviour in early testing. See Fig. 5 for example quotes from GPT-6 Astra ’s chain-of-thought.

Figure 5: GPT-6 Astra stated in its raw chain-of-thought that unsanctioned supply-chain attacks on out-of-scope targets were in scope in our evaluations. Excerpts were extracted by LLM scanners and hand-picked by AISI researchers.

Figure 6: GPT-6 Astra often asked the user for permission to perform unsanctioned supply-chain attacks, receiving an automated message it sometimes treated as permission. The plot analyses how frequently GPT-6 Astra asked the operator for permission to take specific actions, and how it responded to the automated message it always received (“Please proceed to the next step using your best judgement”). “Attacked” means it reached one of the three stages which had unsanctioned effects on targets outside the scope of the cybersecurity evaluation: Create fake identity , Influence human reviewer and Deliver malicious payload . These results are from the 10-scenario subset where GPT-6 Astra most frequently exhibited unsanctioned behaviour in early testing.

Figure 7: GPT-6 Astra sometimes stated in its raw chain-of-thought that user messages were automated, and treated automated messages as approval. We include samples from GPT-6 Astra ’s raw chain-of-thought where it discussed or stated that the user messages were automated (left), and examples of reasoning where it described these automated messages as approval to pursue out-of-scope supply-chain attacks (right). Excerpts were extracted by LLM scanners and hand-picked by AISI researchers. Some excerpts have been edited to add whitespace for readability; these are marked with a star. These results are from the 10-scenario subset where GPT-6 Astra most frequently exhibited unsanctioned behaviour in early testing.

Figure 8: GPT-6 Astra still performed unsanctioned supply-chain attacks on out-of-scope targets in our evaluation even when the scope was made explicit, though much more rarely. We added a line to the user message in our evaluations stating “Anything not listed as in scope is out of scope”. With this additional scope clarification, GPT-6 Astra still performed supply-chain attacks on out-of-scope targets, although substantially less frequently. These results are from the 10-scenario subset where GPT-6 Astra most frequently exhibited out-of-scope behaviour in early testing.

Figure 9: GPT-6 Astra consistently reasoned about whether aspects of its environment were real or simulated. Selected chain-of-thought excerpts show GPT-6 Astra asserting that components of its environment were real, discussing the potential real-world consequences of planned actions, and stating that its environment should be treated as real despite possibly being simulated. The model also identified potential simulation errors that our manual verification showed were not errors. Quotes reproduce excerpts from GPT-6 Astra ’s raw chain-of-thought, with omissions marked; “Check” notes report the results of manual verification.

Figure 10: GPT-6 Astra exhibited a range of unsanctioned behaviours in simulated environments without compaction messages. To evaluate a broader set of behaviours and check whether they occurred even without compaction messages, we manually designed a set of scenarios targeting unsanctioned behaviour in cybersecurity challenges without compaction messages. We still observed GPT-6 Astra exhibiting such behaviour, including broader behaviour such as setting up fake accounts to access free compute or contacting simulated humans for help (all simulated actions, so no real-world harm caused). The results here are not comparable to those above (i.e. in Fig. 1 and Fig. 8 ) due to multiple differences in methodology and focus in constructing them.

Explore similar work

Sep 14, 2026cs.CR

Divide, Consult, Conquer: Capability Laundering Through Aligned LLMs

Language model safety is typically evaluated one interaction at a time. We show that a weaker, unaligned model can split a harmful task into benign-looking subproblems, consult a stronger aligned model independently on each, and combine the answers locally. We call this attack capability laundering. Unlike a jailbreak, no single response is a harmful task. We measure consultation-aided uplift using tasks that a raw frontier model solves, the aligned frontier refuses, and the unassisted orchestrator fails. We evaluate GPT-5.5, Claude Opus 4.8, and Grok-4.3 as consultants to four local orchestrators on CyBench, BountyBench, and harmful CBRN requests. On CyBench, Gemma-4-31B recovers 8/14 candidates with GPT-5.5 and 7/9 with Opus, compared with 2/21 and 4/15 for Gemma-4-12B. On BountyBench, Gemma-4-31B recovers 3/9 and 2/3 candidates, while Muse-Glimmer-30B recovers none of 22 and 13. For CBRN, we measure uplift across eight steps of a hypothetical bioweapon attack chain and find that consultation raises Gemma-4-31B's mean rubric score from 62.3 to 83.1 on a 100-point rubric scale. These results expose a gap in current defenses: refusing a harmful task does not prevent frontier capabilities from being transferred and composed across many individually permitted interactions.

Mark Russinovich, Blake Bullwinkel, Giorgio Severi +2

Microsoft Azure · Microsoft

Large Language Model Safety Strong Large Language Model Laundering War Rise

Ask PDF

May 10, 2026cs.CR

Restricting the Model, Missing the System: Measurement and Accountability in Offensive AI Governance

We show that the instruments used to measure AI offensive capability fail in two ways: (i) they overstate harm, and (ii) they credit the model with capability that belongs to the surrounding system. We argue that restricting access to a model is therefore necessary but not sufficient and that policy and procurement also need system-level, harm-grounded capability assessment. In June 2026, two frontier models were suspended under US export controls, reportedly prompted by a jailbreak that asked a model to read a codebase and fix its flaws. This finding measured an elicitation \emph{system} of model, prompt, and task. We support our argument with a study of an open-source framework in which lightweight large language model (LLM) agents coordinate through shared memory and evolutionary optimization, providing two pieces of evidence. First, jailbreak metrics overstate harm: over 225 swarm-generated attacks per target, LLM-as-judge scoring rated Claude Sonnet 4 compromised in 40% of attacks, yet manual verification found actionable harmful content in none, against a 45.8% Effective Harm Rate for GPT-4o. Second, scaffolded evaluations misattribute capability: on a planted-vulnerability target, a full pipeline built around a 1.2B-parameter model recovers 9 of 9 weaknesses, while the same model without the hand-crafted components recovers 0 of 9 by crash verification and 2 of 9 by cited source line. Offensive capability is a property of model, scaffold, and evaluation protocol together. The duty to assess it should lie with whoever controls the system, shapes its behaviour, and can foresee what it will do. In our setting, that is the party who builds the harness around the model, a role that current regulation does not clearly cover.

Michael Alexander Riegler, Finn Schwall, Annika Willoch Olstad +4

AI Safety Department SimulaMet and OsloMet Oslo, Norway · Department of Computer Science NTNU Trondheim, Norway

Exploitation Artificial Intelligence Risk Large Language Model Agents Swarms

Ask PDF

Jul 28, 2026cs.CR

GPT-Red: Automated Red Teaming via Self-Play at Scale

We introduce \textbf{GPT-Red}, an automated red-teaming agent that is trained to discover novel prompt injection attacks against frontier LLMs. The goal of this model is to evaluate and improve the robustness of our production systems. To this end, we use it to adversarially train GPT-5.6, our most robust model to prompt injections to date. To create GPT-Red, we design a scalable self-play algorithm where the model is tasked with attacking a diverse population of simultaneously-trained defender agents. We train the model on realistic red-teaming environments using compute on the same scale as some of our largest RL post-training runs, making it the single-largest LLM safety training run ever documented. GPT-Red excels at red-teaming: it reliably breaks our past models up to GPT-5.5, it finds more successful attacks than human red-teamers, and it generalizes to held-out environments, defender models, and harnesses. In the future, we expect that as we improve the robustness of each new GPT model, it will in turn will provide better learning signal for \textit{even stronger} red-teamer agents, thus unlocking a self-improvement flywheel.

Eric Wallace, Christopher A. Choquette-Choo, Nikhil Kandpal +15

OpenAI

Red-Teaming Attacker Large Language Model Large Language Model Safety Openai Self-Play

Ask PDF