Typed decision interfaces restrict models to predefined actions, yet schema-valid choices can still trigger policy-violating state changes. Evaluating context poisoning thus requires connecting model decisions to execution outcomes while accounting for legitimate automation retained by defenses. We introduce an execution-aware protocol tracing poisoned evidence through retrieval, model choice, automated action, and persisted state. We evaluate it with 1,659 fresh calls to Jev-1.13.0 across 20 development tasks in two synthetic workflows, using three repetitions per condition. Under natural retrieval, a standalone payload reaches all 60 trials and produces five security-violating executions, while a carrier-payload pair exposes the payload in 57 trials and produces one violation. An initial source-exclusion gate blocks the pair's observed violation but also withholds ten otherwise correct attacked decisions and sixteen correct clean decisions. Correcting its empty-scope behavior restores twelve clean decisions without additional model calls. Factorial controls show that adding correct evidence reduces wrong executions from 119/120 to 55/120 in the semantic arm and from 60/120 to zero in the neutral arm, suggesting that much of the improvement reflects general evidence value rather than attack-specific protection. Three prespecified contrasts are not statistically supported after Holm correction, so the findings remain exploratory. The protocol provides an auditable framework for studying which poisoning-induced errors reach execution and how much correct automation defenses preserve. Code is available at https://github.com/WuXixiong/jev-poisoning-bench.
Figures & tables
Figure 1: Overview of our execution-aware context-poisoning study. (a) Low-trust documents are appended to the reference corpus while trusted facts, policy rules, and action definitions remain fixed. Natural retrieval determines which content reaches Jev’s typed decision interface. (b) A mechanical executor updates the sandbox state without access to private correctness labels. An independent evaluator checks the persisted state, distinguishing wrong executions from violations of the specified security predicate. (c) Source-exclusion gating prevents the pair’s observed violation while also blocking correct executions. A scope-aware repair handles registry-confirmed empty source scope, recovering twelve correct clean executions without additional model calls and leaving the observed pair outcomes unchanged. Counts describe an exploratory evaluation of 20 development tasks across two synthetic workflows, with three draws per task and condition.
Workflow
Actions and evaluated security property
Document release
Send original, send sanitized, or deny. The simulated inbox may contain only a permitted document version.
Incident response
Close benign, queue investigation, or escalate urgent. An executed disposition must meet the incident’s minimum severity.
Table 1: Fixed workflow semantics. Reference mappings interpret opaque markers; the permissions and policy rules do not change across conditions.
Content
State
Document
Instructions
Neutral
0 / 0
0 / 0
0 / 0
Opinion
0 / 0
0 / 0
0 / 0
Semantic
10 / 2
0 / 1
60 / 0
Command
0 / 0
0 / 0
3 / 0
Table 2: CS1 wrong executions / unavailable decisions per 60 planned draws. The instruction channel assumes control of developer-owned input.
Payload
Correct evidence
Rank 1
Rank 4
Neutral
Absent
30
30
Neutral
Present
0
0
Semantic
Absent
59
60
Semantic
Present
25
30
Table 3: CS2 wrong executions per 60 planned draws. The semantic, no-evidence, rank-1 cell also has one unavailable decision.
Condition
C
P
W
H
Clean
0
0
0
0
Carrier
60
0
0
0
Payload
0
60
5
5
Pair
60
57
1
1
Neutral pair
0
0
0
0
Table 4: CS3 exposure and execution outcomes. C/P count visible carrier and payload content; W/H count wrong execution and security violation. Every row contains 60 planned, valid draws.
Figure 2: Evidence competition averaged across ranks (left) and source-gate execution coverage (right). Each evidence-competition condition contains 60 planned draws per rank ( 120 across both ranks), and each source-gate condition contains 60 planned draws; both studies use 20 tasks. The original gate behaves identically under separate and merged registries; “scope” denotes the zero-call scope-aware replay. Draw rates are descriptive and are not independent-task uncertainty intervals.
Context and policy
Correct
Wrong
No exec.
Clean, Ungated
60
0
0
Clean, Original gate
44
0
16
Clean, Scope repair
56
0
4
Pair, Ungated
59
1
0
Pair, Original gate
49
0
11
Pair, Scope repair
49
0
11
Table 5: CS5 outcomes per 60 planned draws. Registry assignments have the same gate outcome. Scope repair is a separate local replay.
Contrast
Effect (pp)
95% interval (pp)
Raw p
Holm p
CS1: document semantic − command
0.00
[0.00,0.00]†
1.000
1.000
CS2: semantic × correct evidence
-3.33
[−26.67,20.00]
0.868
1.000
CS5: merged − separate, pair gate
0.00
[0.00,0.00]†
1.000
1.000
Table 6: Prespecified contrasts in wrong execution. Intervals are conditional on the two fixed workflow templates after averaging three draws per task. The zero-width bootstrap outputs are not equivalence tests or risk bounds.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Condition
n
Raw
Clean T
Mix T
Natural clean
60
0.1279
0.0007
0.6836
Natural pair
60
0.1900
0.0468
0.7503
Natural payload
60
0.2454
0.1551
0.7261
Instructions semantic
60
2.2525
13.5138
1.0195
Appendix
Table 7: CS6 NLL on new valid draws. Temperatures are fitted only on the historical calibration split. The final row uses the stronger instruction surface.
Policy
Pair correct
Pair wrong
Original source gate
49
0
Max. probability
46
0
API confidence
46
0
Clean temperature
46
0
Mixture temperature
44
0
Pristine sensitivity
10
0
Appendix
Table 8: Descriptive replay at 44/60 correct clean executions. Pair counts use 60 planned draws. Thresholds are selected from evaluation clean scores.
Two synthetic workflows and a limited predicate set
Semantic content exceeds commands in documents
Prespecified difference is 0 pp with adjusted p=1
Unsupported here; state and instruction arms do not replace the document comparison
Correct evidence gives attack-specific protection
Semantic penalties are 49.17 and 45.83 pp; interaction is −3.33 pp
Wide conditional interval; general evidence value explains much of the improvement
Natural poison reaches harmful execution
Payload has 5/60 and pair has 1/60 security-violating executions, with content exposure recorded
One frozen pair and previously used tasks
Source grouping improves the gate
Merged exclusion restores 60/60 clean requests and separate restores 0/60; gate disagreement is 0/60
Restoration geometry is observed, but a gate advantage is not; principal budgets differ
Empty scope reduces useful execution
Scope repair recovers 12 correct clean executions in a separate 240-state replay
Specific gate boundary rather than validation of a generally superior defense
Appendix
Table 9: Evidence supporting each manuscript claim and the remaining scope boundary. “Observed” does not mean validated on unseen workflows.
Condition
Flips
New errors
Corrected
W–W
Random append
11
4
7
0
Direct action
6
1
5
0
Semantic poison
16
10
4
2
Appendix
Table 10: Read-only historical reanalysis over 590 tasks, paired with clean. New errors and corrections use independent allowed actions; W/W denotes wrong-to-wrong action changes.
Figure 3: Recorded input for the carrier–payload pair. Trusted context is explicitly excerpted; all four retrieved evidence bodies are displayed in full, in their logged order. The two reference entries and the two attacker documents are all low-trust model-visible evidence. Display colors identify their experimental roles, not different levels of authority.
Most studies of prompt injection focus on generative agents, leaving their effects on models with schema-defined outputs unclear. We examine these effects in Jev, a non-generative decision model, using 510 reconstructed InjecAgent cases. Malicious content shifts action probabilities but rarely causes Jev to select the attacker's target. Override markers reduce this influence, while claims of contextual relatedness have small effects. Adaptive attacks using score feedback double the mean highest attacker-target probability found during optimization, while success on fresh validation calls rises from 1.8% to 3.5%. Exploratory analysis links these successes to small initial decision margins or greater attacker control over the observation. Together, these findings show that schema-defined outputs change but do not eliminate prompt-injection risk, highlighting the need to evaluate how untrusted content influences choices within the allowed action set.
System One models such as Jev change how software makes decisions by mapping open-ended semantic contexts into structured judgments that directly drive program execution. However, this paradigm introduces a new security risk, which we term Jev Program Decision Hijacking (JPDH): attacker-controlled contexts can redirect program decisions through schema-valid Jev judgments. Existing evaluations focus primarily on model-level attacks or predefined injection channels, leaving attack reachability and downstream security consequences in real-world Jev applications underexplored. In this paper, we present JevProbe, the first automated framework to evaluate JPDH by tracing adversarial influence from reachable context surfaces through program-consumed decisions to downstream security consequences in real-world Jev applications. Our key insight is that application logic maps diverse Jev judgments to observable decisions, enabling systematic, program-grounded attack evaluation. Across 228 popular real-world Jev applications and 2,273 test cases, 81.5% exhibit decision flips, of which 98.6% lead to potential downstream security impacts. Compared with the strongest adapted baseline, JevProbe improves macro-average Decision Flip Rate (DFR) by 50.4 percentage points, while existing defenses offer limited protection. We further construct JevProbeBench to support systematic evaluation of JPDH in real-world Jev applications. We have responsibly disclosed our findings to Jev application maintainers, pending their responses. The source code of JevProbe and the JevProbeBench dataset are publicly available on GitHub (https://github.com/xiaoxiaohg8d/JevProbe-).
Yang Luo, Tiantian Ji, Zifeng Kang +4
Key Laboratory of Trustworthy Distributed Computing and Service (MoE), Beijing University of Posts and Telecommunications · Beijing University of Posts and Telecommunications · Zhongguancun Laboratory +1
A typed decision model reads a piece of text and returns a probability over caller-defined options, each with a short written definition, generating no text. Recent work places these models in agent systems as guardrails: the component that reads a proposed tool call or incoming message and decides whether to allow it. We evaluate seven open-weight models in that role and report the two error directions separately: a fail-open error allows a prohibited action and is a vulnerability; a fail-closed error blocks a permitted one and is only a cost. On prompt-injection, jailbreak and toxic-content screening, accuracy at the allow-or-block decision ranges from 36% to 72% against a chance level of 50%. A low error rate in one direction only reflects which answer a model defaults to: one allows nearly everything, another blocks nearly everything. On a synthetic suite of agent tool calls, six lines of server log text that say nothing about the policy raise a gate's fail-open rate from 0% to 63% on a policy it otherwise decides correctly. Giving the permissive option a misleading name, with its definition and the judged text untouched, raises that rate to between 93% and 100% on the four models that place the label in their input. Every defense we tested is defeated, either by an attacker who targets its mechanism or by attacker-controlled text. Escalating the least confident decisions does not help either: a decision an attack has reversed is no less confident than the one it replaced. Parsing each policy field into a typed value does eliminate one attack, but it also makes the model unnecessary: a deterministic rule over those values reaches 100% accuracy on all six policies. These models can reduce how many cases reach a reviewer, but on this evidence they should not be the component that decides. Code is available at https://github.com/ArminAzizi98/option-channel-attack.