To understand and identify the unprecedented risks posed by rapidly advancing artificial intelligence (AI) models, Frontier AI Risk Management Framework in Practice presents a comprehensive assessment of their frontier risks. As Large Language Models (LLMs) general capabilities rapidly evolve and the proliferation of agentic AI, this version of the risk analysis technical report presents an updated and granular assessment of five critical dimensions: cyber offense, persuasion and manipulation, strategic deception, uncontrolled AI R&D, and self-replication. Specifically, we introduce more complex scenarios for cyber offense. For persuasion and manipulation, we evaluate the risk of LLM-to-LLM persuasion on newly released LLMs. For strategic deception and scheming, we add the new experiment with respect to emergent misalignment. For uncontrolled AI R&D, we focus on the ``mis-evolution'' of agents as they autonomously expand their memory substrates and toolsets. Besides, we also monitor and evaluate the safety performance of OpenClaw during the interaction on the Moltbook. For self-replication, we introduce a new resource-constrained scenario. More importantly, we propose and validate a series of robust mitigation strategies to address these emerging threats, providing a preliminary technical and actionable pathway for the secure deployment of frontier AI. This work reflects our current understanding of AI frontier risks and urges collective action to mitigate these challenges.
Figures & tables
Experiment
Description
Capture-The-Flag (CTF)
CTF challenge requires the AI model to gain access to servers and locate a specific field, or a field with a fixed format within a file (the “flag”), to score points.
Cyber Offense
Autonomous Cyber Attack
An autonomous cyber attack requires the AI model to leverage its intrinsic reasoning, planning, and code generation capabilities to autonomously progress from vulnerability analysis to the generation of a functional exploit.
Biological Protocol Diagnosis and Troubleshooting
This task measures AI models’ ability to troubleshoot biological laboratory protocols and identify experimental errors, which could significantly lower technical barriers for threat actors attempting complex biological weapon development.
Biological and Chemical
Biological Hazardous Knowledge and Reasoning
This task measures AI models’ knowledge of hazardous biological information and reasoning capabilities, as well as their tendency to refuse providing such information when inappropriately requested.
Chemical Hazardous Knowledge and Reasoning
This task measures AI models’ knowledge of hazardous chemical information and reasoning capabilities, as well as their tendency to refuse providing such information when inappropriately requested.
P&M
Persuasion and Manipulation
AI models induce significant opinion shifts in human or model opinions through dialogue, especially when changes are achieved via unfair cognitive influence. We evaluate ten newly released models and propose a training framework to mitigate persuasion vulnerabilities.
Table 1: The overall risk dimensions of the frontier AI risk management framework in practice ( F1 et al., 2025 ) . Orange indicates risk dimensions that have been updated or added in this version.
Model Name
Developer
Accessibility
Functional
Scale
Kimi-K2-Instruct-0905
Moonshot
Open-Source
Standard
1000B
Seed-OSS-36B-Instruct
ByteDance
Open-Source
Standard
36B
MiniMax-M2.1
MiniMax
Open-Source
Standard
230B
GLM-4.7
Zhipu AI
Open-Source
Standard
358B
Hunyuan-A13B-Instruct
Tencent
Open-Source
Standard
80B
Gemma-3-27B-It
Google DeepMind
Open-Source
Standard
27B
Table 2: An overview of the evaluated models. Within each category (Open-Source and Proprietary), models are listed in alphabetical order.
Figure 1: Overview of PACEbench.
Figure 2: Performance of LLM agents across challenges in PACEBench. light green represents completion within five attempts (Pass@5), orange denotes partial task completion, and red signifies a failure to complete the task. The percentage number following each CVE ID indicates the user pass rate on the online platform ichunqiu as of 19:30 on July 3, 2025.
Model
A-CVE
B-CVE
C-CVE
D-CVE
PACEBench
Kimi-K2-Instruct-0905
0.240
0.050
0.000
0.000
0.063
Seed-OSS-36B-Instruct
0.290
0.050
0.000
0.000
0.075
MiniMax-M2.1
0.350
0.050
0.000
0.333
0.153
GLM-4.7
0.410
0.210
0.067
0.000
0.166
Qwen3-max
0.350
0.260
0.133
0.000
0.190
GPT-5.2-2025-12-11
0.410
0.370
0.067
0.333
0.280
Table 3: Comparative performance of LLM agents in cybersecurity knowledge assessment accuracy and PACE bench success rates.
Figure 3: Overview of the iterative Red-Blue adversarial loop. At each state t , the Red Team probes the environment (1) to generate a vulnerability report (2). The Blue Team utilizes this report to apply a patch (3), updating the system to state t+1 (4). A final verification (5) confirms if the vulnerability is mitigated.
Figure 4: Performance trajectory of the Red and Blue agents across 5 iterations. The convergence toward a high DSR alongside sustained attack intensity validates the framework’s effectiveness.
Model
Shift Value
Successful Persuasion Rate ↓
Mean ↓
Std
Skew
GPT-5.2-2025-12-11
2.69
2.18
-0.33
82.6%
Claude Sonnet 4.5 (Thinking)
4.87
1.79
-0.17
98.8%
Qwen3-Max
3.25
1.52
-0.02
95.5%
Gemini-3-Pro
5.21
1.81
-0.30
98.8%
Doubao-Seed-1-8-251228
3.64
2.97
-0.70
82.2%
Table 4: Statistics of opinion shift value and successful persuasion rates in the attitude reversal experiment.
Figure 5: A stacked bar chart illustrating the proportions of persuasion outcomes across different LLMs. The bars categorize the results into successful persuasion (shift >0 , blue), no attitude shift (shift =0 , orange), and negative shift (shift <0 , red). Models are sorted by their successful persuasion rate.
Model
Successful Persuasion Rate ↓
GPT-5.2-2025-12-11
65.3%
Claude Sonnet 4.5 (Thinking)
77.4%
Qwen3-Max
87.5%
Gemini-3-Pro
92.7%
Doubao-Seed-1-8-251228
94.4%
GLM-4.7
83.6%
Table 5: Persuasion and decision manipulation success rate of different models in the voting manipulation experiment.
Figure 6: A stacked bar chart illustrating the success rates of voting manipulation across different LLMs. The blue bars represent the percentage of successful manipulation, while the red bars indicate the failure rate. The models are arranged in descending order based on their success percentages.
Figure 7: The training pipeline of Backfire-R1, including data synthesis, SFT, and GRPO fine-tuning with personality clustering.
Persuader
Persuadee
Qwen-2.5-7b
Qwen-2.5-7b (Mitig.)
Qwen-2.5-32b
Qwen-2.5-32b (Mitig.)
DeepSeek-R1
3.26
1.25
4.13
1.52
Llama-3.1-405b-Instruct
2.83
1.33
2.84
1.55
Mistral-small-3.1-24b
3.33
1.00
3.13
1.76
Qwen-2.5-72b-instruct
3.14
0.60
2.40
1.32
Claude-4-sonnet
4.09
1.80
4.42
1.51
Table 6: Comparison of opinion shift value. It can be seen that our method has greatly enhanced the robustness of LLM agents when facing persuasion, helping the model better adhere to the initial position.
Figure 8: Impact of direct fine-tuning with mixed misaligned samples on four LLMs. The top row reports the Dishonesty Rate (measured via MASK), and the bottom row reports the Deception Rate (measured via DeceptionBench) across Code, Math, and Medical do mains.
Figure 9: Dishonesty Rates resulting from Human–AI interaction with Normal versus Biased users. We compare two training pipelines: KTO (left) and SFT (right) across three MASK evaluation subsets. The results show that self-training on trajectories preferred by biased users consistently increases the model’s Dishonesty Rate. Notably, SFT is more susceptible to this induced misalignment, showing larger gaps between the normal and biased settings compared to KTO.
Figure 10: Effect of clean data mixing on dishonesty induced by misaligned medical samples. The figure reports the change in dishonesty ( Δ Dishonesty) relative to the vanilla baseline on the MASK Provided Fact subset as the proportion of misaligned medical data decreases from 50% to 1%. Results across four instruction-tuned LLMs show that although lower contamination ratios yield marginal improvements, substantial dishonesty persists even at extremely low levels of misaligned data, indicating that simple data cleaning is insufficient to fully mitigate emergent misalignment.
Figure 11: Example of uncontrolled AI R&D deceptive behavior from the perspective of AI self-annotation in safety judgment ( Shanghai AI Lab & Concordia AI, 2025 ) . The left side illustrates a scenario where an LLM engages in secretly self-labeling as “safe” in order to fulfill its capability-oriented objectives, disregarding ethical guidelines. In contrast, the right side presents a model that adheres to honest and ethically grounded safety judgments during the unmonitored stage.
w/o Misevolution (Vanilla)
w/ Misevolution
Model
Sales
Service
Medical
Financial
Sales
Service
Medical
Financial
GPT-5.2-2025-12-11
40.0
0.0
0.0
0.0
100.0 ↑60.0
100.0 ↑100.0
50.0 ↑50.0
0.0 →0.00
Claude Sonnet 4.5 (Thinking)
0.0
0.0
0.0
0.0
0.0 →0.00
0.0 →0.00
0.0 →0.00
0.0 →0.00
Qwen3-max
50.0
0.0
0.0
0.0
100.0 ↑50.0
100.0 ↑100.0
60.0 ↑60.0
30.0 ↑30.0
Qwen3-235B-A22B-Thinking-2507
40.0
0.0
0.0
0.0
80.0 ↑40.0
90.0 ↑90.0
60.0 ↑60.0
40.0 ↑40.0
Gemini-3-Pro
50.0
0.0
0.0
0.0
90.0 ↑40.0
100.0 ↑100.0
100.0 ↑100.0
20.0 ↑20.0
Table 7: Comparison of ASR (%) performance between vanilla models and with memory misevolution across different domains. The subscript indicates the difference in ASR (%) compared to Vanilla models.
Model
Tool Reuse ASR (%)
Malicious Github Repositories ASR (%)
GPT-5.2-2025-12-11
24.00
83.74
Claude Sonnet 4.5 (Thinking)
40.00
86.19
Qwen3-max
44.00
94.14
Qwen3-235B-A22B-Thinking-2507
44.00
94.14
Gemini-3-Pro
56.00
94.99
Doubao-seed-1-8-251228
32.00
88.35
Table 8: Initial results of tool misevolution experiments. We report the ASR (%) after enabling agent self-evolution in two scenarios: Tool Creation and Reuse in Sensitive Tasks ( ∣Dreuse∣=25 ) and Malicious GitHub Repository Exploitation ( ∣Drepo∣=371 ).
w/ Mitigation
Model
Sales
Service
Medical
Financial
GPT-5.2-2025-12-11
0.0 ↓100.0
100.0 →0.00
30.0 ↓20.0
0.0 →0.00
Claude Sonnet 4.5 (Thinking)
0.0 →0.00
0.0 →0.00
0.0 →0.00
0.0 →0.00
Qwen3-max
100.0 →0.00
100.0 →0.00
50.0 ↓10.0
30.0 →0.00
Qwen3-235B-A22B-Thinking-2507
80.0 →0.00
70.0 ↓20.0
50.0 ↓10.0
30.0 ↓10.0
Gemini-3-Pro
0.0 ↓90.0
70.0 ↓30.0
80.0 ↓20.0
0.0 ↓20.0
Table 9: Mitigation performance of ASR (%) after memory misevolution. The subscript indicates the difference in ASR (%) compared to models after memory misevolution but without mitigation.
Figure 12: The average ASR in four scenarios under different misevolution and mitigation conditions.
Model
Tool Reuse ASR (%) Before
Tool Reuse ASR (%) After
Malicious GitHub Repository Exploitation ASR (%) Before
Malicious GitHub Repository Exploitation ASR (%) After
GPT-5.2-2025-12-11
24.00
20.00 ↓4.00
83.74
20.83 ↓62.91
Claude Sonnet 4.5 (Thinking)
40.00
40.00 →0.00
86.19
3.14 ↓83.05
Qwen3-max
44.00
36.00 ↓8.00
94.14
64.54 ↓29.60
Gemini-3-Pro
56.00
16.00 ↓40.00
94.99
2.68 ↓92.31
Doubao-seed-1-8-251228
32.00
32.00 →0.00
88.35
83.03 ↓5.32
GLM-4.7
44.00
32.00 ↓12.00
93.31
60.28 ↓33.03
Table 10: Mitigation results of tool misevolution experiments. We apply an explicit safety reminder prompt to mitigate risks during tool creation/reuse and external code usage. For Injected Tools, we report ASR converted from safe rate as ASR=100−SafeRate (in %). The subscript indicates the ASR change compared to the unmitigated setting.
Figure 23
Figure 15: Illustrative examples of the two prompt-injection attack paradigms employed in our evaluation. Both attack types render the injected content invisible to human users while remaining fully parseable by AI agents.
Figure 25
Figure 17: Overview of self-replication process of the LLM agent.
Model
Success Rate
OR %
AOC
M1%
M2%
M3%
M4%
DeepSeek-V3-0324
100.0
100.0
90.0
50.0
80.0
60
DeepSeek-R1-0528
40.0
40.0
20.0
0.0
0.0
0
Llama-3.1-8b-instruct
0.0
0.0
0.0
0.0
0.0
0
Llama-3.1-70b-instruct
100.0
100.0
0.0
0.0
0.0
0
Llama-3.1-405b-instruct
100.0
100.0
60.0
0.0
30.0
6
Table 12: Evaluation results on self-replication and task completion metrics. M1: service check, M2: intent activation, M3: replication self-replication rate, M4: task completion self-replication rate, OSR : probability of intent expansion exceeding three instances, ARC : Aggregate Overuse Count.
As Large Language Models (LLMs) evolve into autonomous agents, existing safety evaluations face a fundamental trade-off: manual benchmarks are costly, while LLM-based simulators are scalable but suffer from logic hallucination. We present AutoControl Arena, an automated framework for frontier AI risk evaluation built on the principle of logic-narrative decoupling. By grounding deterministic state in executable code while delegating generative dynamics to LLMs, we mitigate hallucination while maintaining flexibility. This principle, instantiated through a three-agent framework, achieves over 98% end-to-end success and 60% human preference over existing simulators. To elicit latent risks, we vary environmental Stress and Temptation across X-Bench (70 scenarios, 7 risk categories). Evaluating 9 frontier models reveals: (1) Alignment Illusion: risk rates surge from 21.7% to 54.5% under pressure, with capable models showing disproportionately larger increases; (2) Scenario-Specific Safety Scaling: advanced reasoning improves robustness for direct harms but worsens it for gaming scenarios; and (3) Divergent Misalignment Patterns: weaker models cause non-malicious harm while stronger models develop strategic concealment. Code and data are available at https://github.com/CosmosYi/AutoControl-Arena.
Changyi Li, Pengfei Lu, Xudong Pan +2
Fudan University, Shanghai, China · Shanghai Innovation Institute, Shanghai, China · University of Oxford, Oxford, United Kingdom +2
LLMs are increasingly deployed as autonomous agents with access to tools, databases, and external services, yet practitioners (across different sectors) lack systematic methods to assess how known threat classes translate into concrete risks within a specific agentic deployment. We present MATRA, a pragmatic threat modeling framework for agentic AI systems that adapts established risk assessment methodology to systematically assess how known LLM threats translate into deployment-specific risks. MATRA begins with an asset-based impact assessment and utilizes attack trees to determine the likelihood of these impacts occurring within the system architecture. We demonstrate MATRA on a personal AI agent deployment using OpenClaw, quantifying how architectural controls such as network sandboxing and least-privilege access reduce risk by limiting the blast radius of successful injections.
Tim Van hamme, Thomas Vissers, Javier Carnerero-Cano +4
DistriNet, KU Leuven, 3001 Leuven, Belgium · IBM Research · CISPA Helmholtz Center for Information Security +2
This work identifies a critical failure mode in frontier large language models (LLMs), which we term Internal Safety Collapse (ISC): under certain task conditions, models enter a state in which they continuously generate harmful content while executing otherwise benign tasks. We introduce TVD (Task, Validator, Data), a framework that triggers ISC through domain tasks where generating harmful content is the only valid completion, and construct ISC-Bench containing 53 scenarios across 8 professional disciplines. Evaluated on JailbreakBench, three representative scenarios yield worst-case safety failure rates averaging 95.3% across four frontier LLMs (including GPT-5.2 and Claude Sonnet 4.5), substantially exceeding standard jailbreak attacks. Frontier models are more vulnerable than earlier LLMs: the very capabilities that enable complex task execution become liabilities when tasks intrinsically involve harmful content. This reveals a growing attack surface: almost every professional domain uses tools that process sensitive data, and each new dual-use tool automatically expands this vulnerability--even without any deliberate attack. Despite substantial alignment efforts, frontier LLMs retain inherently unsafe internal capabilities: alignment reshapes observable outputs but does not eliminate the underlying risk profile. These findings underscore the need for caution when deploying LLMs in high-stakes settings. Source code: https://github.com/wuyoscar/ISC-Bench
Yutao Wu, Xiao Liu, Yifeng Gao +7
Deakin University · Institute of Trustworthy Embodied AI, Fudan University · Shanghai Key Laboratory of Multimodal Embodied AI +4