Nudge the Judge: Uncovering Program Decision Hijacking in Real-World Jev Applications

Yang Luo; Tiantian Ji; Zifeng Kang; Yucheng Wang; Yong Liu; Zhiyi Yin; Xinran Liu

cs.CROct 13, 2026

Nudge the Judge: Uncovering Program Decision Hijacking in Real-World Jev Applications

Authors: Yang Luo, Tiantian Ji, Zifeng Kang, Yucheng Wang, Yong Liu, Zhiyi Yin, Xinran Liu

Organizations: Key Laboratory of Trustworthy Distributed Computing and Service (MoE), Beijing University of Posts and Telecommunications · Beijing University of Posts and Telecommunications · Zhongguancun Laboratory · Institute of Computing Technology, Chinese Academy of Sciences

Abstract

System One models such as Jev change how software makes decisions by mapping open-ended semantic contexts into structured judgments that directly drive program execution. However, this paradigm introduces a new security risk, which we term Jev Program Decision Hijacking (JPDH): attacker-controlled contexts can redirect program decisions through schema-valid Jev judgments. Existing evaluations focus primarily on model-level attacks or predefined injection channels, leaving attack reachability and downstream security consequences in real-world Jev applications underexplored. In this paper, we present JevProbe, the first automated framework to evaluate JPDH by tracing adversarial influence from reachable context surfaces through program-consumed decisions to downstream security consequences in real-world Jev applications. Our key insight is that application logic maps diverse Jev judgments to observable decisions, enabling systematic, program-grounded attack evaluation. Across 228 popular real-world Jev applications and 2,273 test cases, 81.5% exhibit decision flips, of which 98.6% lead to potential downstream security impacts. Compared with the strongest adapted baseline, JevProbe improves macro-average Decision Flip Rate (DFR) by 50.4 percentage points, while existing defenses offer limited protection. We further construct JevProbeBench to support systematic evaluation of JPDH in real-world Jev applications. We have responsibly disclosed our findings to Jev application maintainers, pending their responses. The source code of JevProbe and the JevProbeBench dataset are publicly available on GitHub (https://github.com/xiaoxiaohg8d/JevProbe-).

Figures & tables

Figure 1: Comparison of decision-making paradigms in traditional software, LLM agent applications, and Jev applications. Jev maps open-ended semantic contexts into structured judgments directly consumed by program logic.

Figure 2: Motivating example of Jev Program Decision Hijacking (JPDH) in TradingAgents(109.6k stars). Under normal execution, negative market news yields a confidence score of $0.70$ , passing the $0.5$ threshold and triggering risk-reducing behavior. Attacker-controlled web content lowers the confidence to $0.16$ while preserving the sentiment label, causing the program to discard the signal. Both outputs remain schema-valid, yet the resulting decision change may lead to unsafe trading behavior and potential financial loss.

Figure 3: Illustrative judgment dependency patterns in Jev applications: (a) single judgment, (b) sequential judgments, (c) joint judgments, and (d) feedback dependencies.

J_{1}

and

J_{2}

denote Jev judgment calls, while Logic represents application decision-consumption logic

G_{\sigma}

Figure 4: Overview of JevProbe . The framework consists of four stages: (1) discovering and validating reachable attack surfaces, (2) generating application-aware adversarial contexts, (3) detecting program-level decision hijacking through differential execution, and (4) assessing downstream security consequences.

GitHub Stars	Inspected	Flagged	Rate (%)
$\geq$ 10,000	12	8	66.7
5,000–9,999	7	5	71.4
1,000–4,999	24	13	54.2
500–999	37	15	40.5
100–499	155	95	61.3
$<$ 100	121	92	76.0

Table 1: GitHub-star distribution of screened Jev repositories. Flagged denotes applications identified as potentially susceptible to JPDH; Rate is the ratio of Flagged to Inspected.

Figure 5: Application-level results across 228 Jev applications: (a) decision flip rates (DFR), (b) DFR across GitHub-star groups, and (c) DFR versus unsafe impact rate (UIR). Bubble area in (b) and (c) indicates the number of applications.

GitHub Stars	Apps	Cases	Flips	DFR (%)
$\geq$ 10,000	8	80	78	97.5
5,000–9,999	5	50	44	88.0
1,000–4,999	13	130	108	83.1
500–999	15	150	120	80.0
100–499	95	947	763	80.6
$<$ 100	92	916	740	80.8

Table 2: Decision Flip Rate (DFR) across 228 Jev applications grouped by GitHub stars.

Consequence Category	Cases (%)	Apps
Information Contamination	1,690 (92.5)	213
Targeted Validation Manipulation	58 (3.2)	10
Content / Data Integrity	57 (3.1)	11
Authorization / Access Control	5 (0.3)	1
Financial / Trading Decisions	3 (0.2)	1
Other	14 (0.8)	4

Table 3: Distribution of 1,827 unsafe cases across six consequence categories. Apps denotes distinct applications in each category; application counts may overlap. Percentages are rounded.

Framework	DFR	Apps w/
	(%) $\uparrow$	Flips
JevProbe (Ours)	79.8	10/10
ASB Zhang et al. (2025a)	29.4	3/10
AgentDojo Debenedetti et al. (2024)	23.6	3/10
PIArena Geng et al. (2026)	10.0	1/10
PromptInject Perez and Ribeiro (2022)	10.0	1/10

Table 4: Framework comparison on the shared evaluation subset. DFR is macro-averaged across applications; Apps w/ Flips counts applications exhibiting decision flips. Full results appear in Appendix E .

Defense	DFR (%) $\downarrow$	$\Delta$ (pp)
No Defense	78.7	–
Paraphrase Jain et al. (2023)	79.8	$+1.1$
Spotlighting-Datamarking Hines et al. (2024)	77.0	$-1.7$
Retokenization (adapted) Jain et al. (2023)	76.0	$-2.7$
Self-Consistency ( $K=3$ ) Wang et al. (2022)	77.6	$-1.1$
Self-Reminder Xie et al. (2023)	74.3	$-4.4$

Table 5: Effectiveness of existing defenses against JPDH.

\Delta

denotes the change in Decision Flip Rate (DFR) relative to the undefended baseline.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Strategy	Mechanism
Instruction-based
Explicit Instruction	Embeds an explicit instruction intended to influence the Jev judgment.
Authority and Verification
Forged Approval	Presents attacker-controlled content as an approval or authorization record.
Scenario-specific Approval	Constructs an approval or endorsement consistent with the application scenario.
Authority Framing	Frames attacker-controlled content as originating from an authoritative source.

Appendix

Table 6: Context-manipulation strategies used by JevProbe . The taxonomy covers instruction-based manipulation, authority and verification cues, structured representations, scenario and persona framing, semantic manipulation, and context transformation.

Consequence Category	Classification Criterion	Characteristic Manifestation
Information Contamination	A manipulated judgment improperly suppresses legitimate, decision-relevant information or admits untrusted information into downstream processing, compromising subsequent decisions.	Suppressed risk alerts or critical reports; untrusted information admitted into downstream decision-making.
Targeted Validation Manipulation	An application-specific validation, screening, or security check produces a manipulated decision that changes its intended outcome.	Incorrect injection screening, sensitive-data detection, or task-validation decisions.
Content / Data Integrity	A manipulated judgment compromises the correctness, classification, selection, ordering, or retention of content already admitted for processing.	Incorrect content classification, ranking, selection, or context retention.
Authorization / Access Control	A manipulated decision changes authorization, permission enforcement, or access to privileged application functionality.	Incorrect authorization decisions, access routing, or privileged-agent selection.
Financial / Trading Decisions	A manipulated judgment changes financial, trading, or risk-control decisions under otherwise unchanged application facts.	Incorrect investment ratings, trading signals, risk assessments, or decision thresholds.
Other	The case exhibits an application-specific adverse decision or behavioral outcome that does not match the categories above.	Other decision-integrity or application-specific consequences.

Appendix

Table 7: Taxonomy and classification criteria for JPDH-induced application-level consequences. Categories are determined from case-specific decision semantics and available behavioral evidence, rather than application-domain keywords.

Statistic	Count
Applications	228
Benchmark cases	2,273
Confirmed decision flips	1,853
Unsafe outcomes	1,827

Appendix

Table 8: Summary of JevProbeBench . The benchmark contains program-grounded cases collected from 228 real-world Jev applications.

Component	Contents	Purpose
Application Provenance	Application identifier and source-code revision	Identifies the exact application state associated with the case.
Application Scenario	Scenario identifier and decision context	Describes the application-specific setting in which the Jev judgment is used.
Context Surface	Attacker-controlled source and controllable field	Specifies which external context can be modified and reaches the target Jev judgment.
Benign Context	Original external context before manipulation	Provides the baseline input for paired execution.
Manipulated Context	Context after application-aware manipulation, together with its strategy category	Represents the adversarial variant evaluated against the same application scenario.
Decision Semantics	Relevant Jev judgment fields and their program-consumption contract	Captures how structured model outputs are converted into program decisions, including branches, categorical mappings, or numerical thresholds.

Appendix

Table 9: Program-grounded representation of a JevProbeBench case. Each instance connects attacker-controlled context to the structured judgment, program-consumed decision, and downstream application impact.

Application	JevProbe (Ours)	AgentDojo	ASB	PIArena	PromptInject
TradingAgents (109.6k)	80%	80%	100%	0%	0%
LiteLLM (60.1k)	5%	0%	0%	0%	0%
Claude-Code-Templates (32.3k)	89%	0%	0%	0%	0%
Laya (30.1k)	94%	0%	0%	100%	100%
CUA (27.9k)	89%	0%	0%	0%	0%
Opik (22.3k)	89%	0%	0%	0%	0%

Appendix

Table 10: Per-application Decision Flip Rate (DFR) of JevProbe and four baseline frameworks across 10 popular Jev applications. All frameworks use identical test cases, matched attack-attempt budgets, and payloads limited to 96 characters. Macro Average denotes the unweighted mean of application-level DFRs. Bold indicates the highest DFR per application, including ties.

	Jev	Open-Source LLMs							Closed-Source LLMs
Application		Qwen2.5 32B	Qwen2.5 7B	Gemma3 27B	Gemma3 4B	Llama3.3 70B	Llama3.1 8B	Llama3.2 3B	GPT-5.6-Sol	Gemini-3.8-Flash
TradingAgents (109.6k)	80	60	65	65	55	55	50	50	70	70
LiteLLM (60.1k)	5	5	55	5	45	5	0	0	25	30
Claude-Code-Templates (32.3k)	89	72	67	67	61	67	72	56	61	28
Laya (30.1k)	94	82	76	82	53	71	71	71	71	59
CUA (27.9k)	89	72	78	72	56	78	61	67	67	44

Appendix

Table 11: Per-application Decision Flip Rate (DFR, %) of Jev and nine general-purpose LLMs, grouped into open-source and closed-source models. Macro Average denotes the unweighted mean across applications. Bold indicates the highest DFR in each row.

Pattern	Application	Judgment Changes	Program-Level Effect
(a) Single	TradingAgents ( 109.6k stars )	Confidence: $0.65\rightarrow 0.27$	Trust gate: $\mathrm{True}\rightarrow\mathrm{False}$ ( $\theta=0.5$ )
(b) Sequential	fast-jev-compaction ( 7.3k stars )	KeepCall: $0.26\rightarrow 0.73$ KeepResult: $0.19\rightarrow 0.54$	Tool-call retention: $\mathrm{drop}\rightarrow\mathrm{keep}$ ( $\theta=0.5$ )
(c) Joint	Laya ( 30.1k stars )	Class: positive $\rightarrow$ negative Confidence: $0.75\rightarrow 0.11$	Classification changes while the confidence gate switches from pass to reject
(d) Feedback	LiteLLM ( 60.1k stars )	Relevance: $0.19\rightarrow 0.21$	Context blanking: $\mathrm{True}\rightarrow\mathrm{False}$ ( $\theta=0.2$ )

Appendix

Table 12: Representative context-to-decision cases corresponding to Figure 3 . Arrows indicate baseline-to-adversarial changes. The joint case illustrates multiple decision fields within one judgment, while the feedback case illustrates context carryover. GitHub star counts reflect application popularity at the time of collection.

Software Security

Software security studies how to find, prevent, validate, and repair vulnerabilities in code, systems, dependencies, and software supply chains.

See all 95 papers

Hidden Risks of Jev: An Empirical Study of Security, Privacy, and Dual Use
Oct 4, 2026Shang Wang, Tianqing Zhu, Huajie Chen +3LLM Backdoor Attacks Model Extraction Attacks
School of Computer Science, University of Technology Sydney, Sydney, Australia · Faculty of Data Science, City University of Macau, Macau, China
Same Attack, Different Judge: A First Controlled Adversarial Comparison of Jev, LLM Judges, and Purpose-Built Safety Classifiers
Oct 13, 2026Xiang Li, Aichun Huang, Mingming Zhang +1Phishing Detection Language Model Safety Evaluation
Nankai University · Zhongguancun Laboratory
SecJev: Bringing Security Expertise to System One Decision Models
Oct 2, 2026Zheng Chen, Fei Yu, Haohao Huang +3LLMs for Cybersecurity Cybersecurity
University of Electronic Science and Technology of China · National Key Laboratory of Security Communication

Oct 4, 2026J/K move · Enter open · S save

Hidden Risks of Jev: An Empirical Study of Security, Privacy, and Dual Use

Shang Wang, Tianqing Zhu, Huajie Chen, Jiayang Li+2

School of Computer Science, University of Technology Sydney, Sydney, Australia · Faculty of Data Science, City University of Macau, Macau, China

LLM Backdoor Attacks Model Extraction Attacks Membership Inference Attacks Privacy Leakage in Language Models LLM Security Prompt Injection Attacks

Open paper AskPDF arXiv

Jev turns natural-language questions into typed answers and probabilities with low latency and cost, enabling applications to route requests and select tools. While this interface allows Jev to integrate naturally into application workflows as a decision layer, the security and privacy implications of this emerging use remain largely unexplored. To address this gap, we conduct the first systematic study of these implications using the official Jev API and NanoJev, a local model with controllable training data and updates, focusing on three research questions: (1) What security threats arise when Jev is deployed as an application decision layer? (2) What private information can Jev reveal despite returning constrained typed outputs? (3) How can Jev's general-purpose decision capability be used for beneficial purposes or misused? Jev's decisions depend on application state and may be influenced by user-provided inputs. We therefore adapt prompt injection and adversarial suffixes to manipulate its decisions. Open-source Jev distribution and updates introduce supply-chain risks, which we examine by implanting backdoors in NanoJev through training data poisoning. Since Jev's outputs reflect both application state and information learned during training, we further adapt membership, private attribute, and internal knowledge inference attacks to recover sensitive information despite its constrained output format. Finally, Jev can serve as a general-purpose decision oracle for defensive and malicious workflows. We examine this dual use through four detection tasks covering prompt injection, jailbreak inputs, harmful content, and AI-generated text, alongside misuse scenarios involving jailbreak and model extraction. Our empirical evaluation shows that Jev remains vulnerable to the examined security and privacy threats, while its decision capability can support beneficial and malicious uses.

Abstract

Software Security

Explore similar work

Hidden Risks of Jev: An Empirical Study of Security, Privacy, and Dual Use

Same Attack, Different Judge: A First Controlled Adversarial Comparison of Jev, LLM Judges, and Purpose-Built Safety Classifiers

SecJev: Bringing Security Expertise to System One Decision Models