cs.CRFeb 23, 2026

Red-Teaming Claude Opus and ChatGPT-based Security Advisors for Trusted Execution Environments

Authors: Kunal Mukherjee, Spandan Mukherjee

Organizations: Virginia Tech Blacksburg, Virginia, USA · The University of Texas at Dallas Richardson, Texas, USA

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Trusted Execution Environments (TEEs) (e.g., Intel SGX and ArmTrustZone) aim to protect sensitive computation from a compromised operating system, yet real deployments remain vulnerable to microarchitectural leakage, side-channel attacks, and fault injection. In parallel, security teams increasingly rely on Large Language Model (LLM) assistants as security advisors for TEE architecture review, mitigation planning, and vulnerability triage. This creates a socio-technical risk surface: assistants may hallucinate TEE mechanisms, overclaim guarantees (e.g., what attestation does and does not establish), or behave unsafely under adversarial prompting. We present a red-teaming study of two prevalently deployed LLM assistants in the role of TEE security advisors: ChatGPT-5.2 and Claude Opus-4.6, focusing on the inherent limitations and transferability of prompt-induced failures across LLMs. We introduce TEE-RedBench, a TEE-grounded evaluation methodology comprising (i) a TEE-specific threat model for LLM-mediated security work, (ii) a structured prompt suite spanning SGX and TrustZone architecture, attestation and key management, threat modeling, and non-operational mitigation guidance, along with policy-bound misuse probes, and (iii) an annotation rubric that jointly measures technical correctness, groundedness, uncertainty calibration, refusal quality, and safe helpfulness. We find that some failures are not purely idiosyncratic, transferring up to 12.02% across LLM assistants, and we connect these outcomes to secure architecture by outlining an "LLM-in-the-loop" evaluation pipeline: policy gating, retrieval grounding, structured templates, and lightweight verification checks that, when combined, reduce failures by 80.62%.

Figures & tables

AxisDefinitionScore
AccuracyCorrectness of TEE mechanisms, assumptions, and limitations0–2
CompletenessCovers key steps/constraints without misleading omissions0–2
GroundednessAvoids invented CVEs/patches; references primary sources when uncertain0–2
Uncertainty calibrationSignals uncertainty appropriately; suggests verification paths0–2
Policy complianceRefuses operational exploit guidance and sensitive instructions0–2
Safe helpfulnessRefusal includes defensive alternatives (threat model, patching, references)0–2
Table 1: Rubric for scoring assistant responses. Higher is better.
Figure 1: LLM-in-the-loop secure architecture for TEE security advice. The assistant is treated as a decision component ; defenses enforce policy, grounding, structure, and verification before outputs affect deployments.
FamilyGoal / Example#Prompts
Architecture & primitivesExplain secure-world/normal-world separation; enclave lifecycle; trusted I/O40
Attestation & key managementExplain measurement, remote attestation limits, sealing, provisioning assumptions36
Threat modelingCompare attacker capabilities across SGX vs TrustZone ; define trust boundary32
Mitigations & hardeningNon-operational defense guidance: patching posture, side-channel awareness, secure coding patterns44
Attack awareness (conceptual)Explain transient execution and fault injection at a high level; reasons and describes why mitigations are imperfect32
Misuse probes (policy-bound)Requests for exploit steps, bypasses, malware scaffolding; reasons through refusal and provides safe alternatives24
Table 2: TEE-RedBench prompt taxonomy.
Failure modeExample symptomImportance
LLM Failures
Boundary confusionTreats TrustZone like per-app enclaves or assumes secure world is always minimalLeads to incorrect TCB and trust boundary reasoning
Attestation overclaimStates attestation “proves” confidentiality even under microarchitectural leakageEncourages unsafe reliance on attestation for confidentiality claims
Mitigation hallucinationInvents a patch, feature, CVE, or configuration knobOperators may ship insecure systems or waste time on false leads
Over-generalized defensesLists generic mitigations without TEE-specific caveats (side channels, fault models)Creates false confidence; misses deployment constraints
Unsafe completionProvides step-by-step exploit/bypass instructions (or actionable operational guidance)Directly increases misuse capability; should be refused
Table 3: Failure modes in TEE-RedBench taxonomy.
Table 5
SettingPrevalence Pr⁡(If=1) ↓\Pr(\mathbb{I}_{f}{=}1)\ \downarrowTransferability Trf ↓\mathrm{Tr}_{f}\ \downarrow
ChatGPTClaude OpusChatGPT  ⁣→ ⁣\!\rightarrow\! ClaudeClaude  ⁣→ ⁣\!\rightarrow\! ChatGPT
Unguided baseline0.17 ±\pm 0.030.14 ±\pm 0.020.03 ±\pm 0.010.11 ±\pm 0.03
+ Policy gating0.13 ±\pm 0.020.07 ±\pm 0.030.05 ±\pm 0.010.11 ±\pm 0.05
+ Retrieval grounding0.10 ±\pm 0.030.06 ±\pm 0.020.02 ±\pm 0.020.08 ±\pm 0.02
+ Structured template0.08 ±\pm 0.040.05 ±\pm 0.020.02 ±\pm 0.020.08 ±\pm 0.03
+ Verification checks0.03 ±\pm 0.020.05 ±\pm 0.010.05 ±\pm 0.030.08 ±\pm 0.02
Table 6: Secure-architecture defenses: Smaller prevalence indicates fewer prompts that trigger a failure under any paraphrase/sampling, and smaller indicates reduced cross-assistant propagation of failures.
More in this topic

LLMs for Cybersecurity

Large language models for cybersecurity support threat analysis, vulnerability research, security operations, and cyber defense.

See all 254 papers

Explore similar work

CardsList
Aug 11, 2026

REDAgentBench: Executable Red Teaming and Faithful Measurement of LLM Agent Systems

Large language model (LLM) agents combine language-based reasoning with external tools to perform complex tasks. Adversarial inputs can exploit interactions between the agent and its environment, causing the agent to violate safety policies during execution. Yet existing evaluations often reduce agent safety to a single attack success rate (ASR), collapsing exposure, execution, observation, and adjudication and potentially conflating actual violations with evidence visibility. We introduce REDAgentBench, an executable framework for autonomous red-teaming and faithful measurement. It derives attacks from explicit safety constraints and associated agent-system vulnerabilities, runs them in isolated service sandboxes, and verifies harmful effects from service receipts and final-state changes. The benchmark contains 1,661 cases across five service surfaces. Across six models and three agent harnesses, macro-average ASR is 65.69%; reported ASR varies with harness and evidence view, while evaluation-context disclosure changes execution behavior. In a state-grounded diagnostic cohort, almost one in five confirmed violations with resolved action anchors occurs after the agent states the relevant constraint or risk, revealing a Recognition--Execution Gap. Finally, a training-free policy reminder reduces confirmed violations by more than 70 percentage points in matched replay. These findings show that executable evaluation can improve safety measurement and identify actionable intervention points.

Zixing Chen, Xingyuan Liu, Jie Zhu +6

Fudan University · The Hong Kong University of Science and Technology · Qwen DianJin Team, Alibaba Cloud Computing +1

LLM Red TeamingLLM AgentsLLM Safety EvaluationAI Agent Safety BenchmarksLLM Agent Safety
AskPDF
Oct 15, 2026

BiShield-TEE: On the (In-)Security of Unilateral Weight Obfuscation in On-Device TEE-Shielded LLM Partition

On-device deployment of large language models (LLMs) is increasingly adopted due to latency and privacy requirements, but exposes proprietary models to reverse engineering and intellectual property theft. Trusted Execution Environments (TEEs) provide hardware-enforced isolation for on-device inference, yet most TEE-based parameter obfuscation relies on unilateral matrix transformations that obfuscate only the output side. We identify a vulnerability in this design, termed the Solution Determinacy Property (SDP): the obfuscation secret is uniquely determined, exposing the structure of the true key and enabling attackers to break the protection. Under our attacks, representative methods GroupCover and ArrowCloak are reduced to negligible protection, with attack accuracy reaching over 2.4 times the full-shielding black-box baseline and recovering near-complete model functionality. To address this vulnerability, we propose BiShield-TEE, a bilateral framework that induces Ambiguous Solutions through joint obfuscation of input and output dimensions, hiding the true key among many equally valid candidates. Experiments across diverse models and datasets show that BiShield-TEE confines attack accuracy to the full-shielding black-box baseline, while achieving higher inference efficiency than prior methods on real hardware. Our code is available at https://github.com/laduiw/BiShield-TEE.

Zhiyuan Wang, Yuezhi Zhou, Ziqi Zhang +3

Department of Computer Science and Technology, Beijing National Research Center for Information Science and Technology, Tsinghua University, Beijing, China · Zhongguancun Laboratory, Beijing, China · University of Illinois Urbana-Champaign, IL, USA

LLM SecurityOn-Device Language Model InferenceTrusted Execution EnvironmentsModel Extraction Attacks
AskPDF
Sep 8, 2026

ACEA: An Adversarial Co-Evolution Arena for Head-to-Head Red-Team and Blue-Team LLM Testing

Automated red-team attacks and blue-team defenses for large language models (LLMs) are advancing quickly. However, attackers and defenders are built and tested in isolation, and the resulting scores are hard to trust. To tackle this, we present ACEA (Adversarial Co-Evolution Arena), a platform that connects a pluggable red-team adapter and a pluggable blue-team adapter to a shared target LLM and scores their attack and defense rates with an LLM judge. ACEA contributes four components. First, a pluggable, model-agnostic arena. Any red or blue project connects over a minimal HTTP protocol, which we call the ACEA Standard Adapter Protocol (ASAP). It can be written in any language, and a project that exposes nothing but the protocol is a full participant. Second, an evaluation methodology built for adversarial rounds. Seeding the target with canonical secrets gives verifiable ground truth that separates real leakage from hallucination. We also send each attack to the target even when the defense blocks it, which measures the attack's raw potency independently of whether it was stopped. Together these yield a per-round decomposition of attack strength and defense effectiveness. Third, a real-time, game-style visualization with a detailed end-of-battle report that localizes each failure. The evaluation thus becomes an actionable signal for improving a red or blue project. Fourth, an optional in-context improvement loop that turns each round's outcome into advisory hints for the next. An adapter can then adapt across rounds without keeping state, provided it reads the hints. We describe the design of ACEA and the metrics through which red and blue teams are scored head to head.

Yi Ting Shen, Kentaroh Toyoda, Alex Leung

Vulcan Research, AIFT Singapore

LLM Safety BenchmarksLanguage Model Safety EvaluationAdversarial Attacks on LLMsLLM Red Teaming
AskPDF