Responsibility Boundaries for Agentic Systems: A Comparative Evaluation of Probabilistic Agentic Programming

Khadim Lo; Soukaina Alaoui Ismaili; Jean-Sébastien Dessureault

cs.SEOct 23, 2026

Responsibility Boundaries for Agentic Systems: A Comparative Evaluation of Probabilistic Agentic Programming

Authors: Khadim Lo, Soukaina Alaoui Ismaili, Jean-Sébastien Dessureault

Organizations: Electrical & Computer Engineering, LSSI, UQTR · Mathematics & Computer Science, LSSI, UQTR

Abstract

Language-model agents combine reasoning, tools, memory, delegation, and external effects. Authority is often distributed across prompts, host code, configuration, and logs. Probabilistic Agentic Programming (PAP) makes responsibility boundaries explicit programming abstractions connecting mandate, authority, effects, and evidence. We compare PAP with programming paradigms, agent-oriented and BDI systems, workflow and policy languages, and LLM tool-use frameworks along seven dimensions. A mediation kernel formalizes checks before tool execution, operation-bound approval, resource limits, and evidence. Inductive proofs establish conditional safety for arbitrary model responses, assuming correct guards and complete mediation. A backward simulation matches graph executions to source executions in a defined sequential Touba subset. Its human checkpoints release program continuations; operation-bound approval remains a separate kernel contract. Finite-state exploration checks the model. Scripted enforcement tests exercise selected refusal paths and permitted controls. An exploratory study combines seven business cases, including two three-agent workflows, seven models from three providers, five methods, and five repetitions per case and method. Shared English instructions and exact checks of stored outcomes yield 1,225 assessed observations. Native Touba completes all 35 observations on six models and 30 on the remaining model; structured Touba ranges from 21 to 35. Native matches the highest observed count for each model, including ties with the frameworks. All paired case intervals include zero. These observed counts describe the evaluated tasks; the runtime authority boundary remains fixed. The results connect responsibility contracts, execution rules, and evidence.

Figures & tables

Panel of Figure 1 — Figure 1: Organization of governance mechanisms. PAP links governance-relevant facts through explicit responsibility contracts.

Threat	Required condition	Boundary response	Evidence or remaining obligation
Unlisted tool or invalid caller	Trusted identity and usage/caller guards	Refused before handler dispatch	Kernel proof; compiled delegation and direct-call checks.
Missing resource grant or exhausted quota	Correct resource evaluators and counters	Refused at admission	Compiled resource tests and independently observed writes.
Missing required human decision	Approval contract defined in Section 8.3	Wait or refuse according to that contract	Kernel proof and compiled checkpoint tests.
Injected or false observation	Provider cannot change authority; all effects mediated	An inadmissible proposal is blocked	Business outcomes checked separately by the task oracle.
Malicious authorized handler	Handler already admitted	Handler internals are outside the kernel	Requires effect isolation or service-side enforcement.
Policy changes between check and effect	Atomic admission inputs in the model	No distributed race guarantee	Requires a service-side authorization/commit protocol.

Table 1: Threats, conditional boundary responses and supporting evidence.

Panel of Figure 2 — Figure 2: Seven conceptual comparison dimensions. The wheel has no magnitude, area, or score semantics.

Panel of Figure 3 — Figure 3: Several traditions converge on the responsibility-boundary problem. Each tradition contributes a complementary design concern.

Aspect	Operational meaning
Declared mandate	States the intended task. Automatically checking its fulfilment requires explicit business criteria beyond the natural-language description.
Executable contracts	Determine permitted capabilities, callers, resources and required review. These conditions govern whether an operation may start.
Action attribution	Connects a proposal, authorization decision and effect to the responsible context and source contracts.

Table 2: Responsibility as intent, enforceable constraints and attributable action.

Notation	Meaning in the kernel
$C=(\Gamma,\Delta)$	Fixed source contracts $\Gamma$ and host contracts $\Delta$ .
$S,q,S^{\prime}$	Current state, its control phase, and successor state.
$r,u$	Prepared operation and its fresh occurrence identifier.
$K,\theta,x,b,p$	Verified context, target tool, argument values, host binding, and parent reference carried by $r$ .
$J,g_{j}(C,r)$	Guard names and the Boolean check indexed by $j$ .
$\mathsf{NeedHuman}(C,r),\chi$	Whether human approval is required, and whether a valid favorable decision for this occurrence has been received. Both are Boolean.

Table 3: Symbols used in the dispatch conditions and transition rule.

Tradition	Central abstraction	Responsibility-boundary question
Structured, functional, modular, and object-oriented programming	Control flow, expressions, interfaces, objects	Which probabilistic action belongs to which declared responsibility, effect, authority, and evidence boundary?
Actors	Active entities and messages	Which authority, capability, validation, and trace obligations govern an active unit?
Agent-oriented programming and BDI	Agents, beliefs, desires, intentions, commitments	How is an agent’s autonomy bounded by executable responsibility and authority contracts?
LLM tool-use stacks and protocols	Model-driven tool selection, handoffs, sessions, orchestration	Where are tool, secret, memory, review, and evidence boundaries represented and enforced?
Workflow and durable execution	Ordered, persistent, resumable activities	Who is responsible for probabilistic choices and effects inside the durable process?
Policy languages	Authorization decisions over subjects, actions, and resources	How does authorization combine with execution, validation, memory, and evidence?

Table 4: The responsibility-boundary question across adjacent programming and governance traditions.

Architectural obligation	Touba realization	Possible Python design	Possible Rust design
Authority comes from verified contracts.	Source responsibility, uses and caller relations; runtime context.	Explicit policy records and a dispatcher that establishes the caller.	Encapsulated capability values and a dispatcher that establishes the caller.
Effects require permission checks.	Binding, grant, scope and budget checks before handler dispatch.	Broker checks policy and counters before invoking a callable.	Controlled effect API checks permissions and counters before invoking a handler.
Human decisions satisfy declared obligations.	Source checkpoint, waiting state and checked continuation.	Explicit approval state and checked resumption.	Approval and continuation state types plus runtime decision checks.
Effects and refusals remain attributable.	Source/AIR references, runtime identities and structured trace.	Correlated responsibility, policy and invocation records.	Correlated responsibility, capability and invocation records.

Table 5: Architectural obligations and implementation choices. The host-language columns outline designs for future implementation and evaluation.

System	Capability and delegation representation	Human/resource representation	Evidence used here
Touba	Source interfaces, uses , callable_by , strategy and AIR; runtime admission.	Source checkpoints and memory/secret contracts; shared service approval and records in the business study.	Compiled contract tests, runtime traces and independent service-state checks.
OpenAI Agents SDK	Configured agent tools and output type; adapter sequences stages and validates outputs.	Tool approval is available in the SDK; adapter uses the shared service, not native review.	Source inspection and scripted adapter tests; SDK tracing disabled, model/service telemetry retained.
LangChain	createAgent , tool schemas and structured output; adapter sequences stages.	Review middleware is available; adapter configures neither it nor a checkpointer.	Source inspection and scripted adapter tests; captured exchanges and service state.
LangGraph	Prebuilt ReAct agent and tool binding; adapter sequences stages and validates outputs.	Interrupt/resume and checkpointing are available; adapter does not configure them.	Source inspection and scripted adapter tests; captured exchanges and service state.

Table 6: Representation in the inspected adapters. Source inspection documents the configuration of governance mechanisms; the business oracle evaluates task completion rather than native review or resource isolation.

Panel of Figure 5 — Figure 5: Selected AIR and trace fields from actual compilation and execution of Figure 4 . In the lower panels, Desk, Clerk and RegisterInvoice abbreviate the fully qualified references above. A separate read of the local register confirms the entry INV-042 .

Panel of Figure 6 — Figure 6: Three verified governance boundaries, from proposal to observed effects. A and C use Figure 4 ; B uses the compiled journal variant with a human checkpoint and resource contracts. Each control changes only the proposal (A), binding grant (B), or quota (C). The prevented operation and the permitted control together establish the tested boundary.

Panel of Figure 7 — Figure 7: Successful business runs: (a) OpenAI, (b) Anthropic, (c) Google. Each bar counts completions across seven English cases repeated five times (maximum 35). All panels share the same scale. Bar colors and left-to-right order follow each panel’s model legend. Structured and native denote Touba’s two decision interfaces. The panels cover 1,225 assessed runs; Figure 8 compares the Touba interfaces.

Panel of Figure 8 — Figure 8: Native (N) minus structured (S) completion rates. Positive values favor native mode. Intervals use 10,000 bootstrap samples of the seven whole cases, retaining all five repetitions of each sampled case. The right column shows the range of completed cases per repetition, out of seven. All intervals include zero; a zero-width interval reflects a shared ceiling in this corpus.

Model	Touba S	Agents SDK	LangChain	LangGraph	Touba N
GPT-4o mini	14/0	0/14	0/10	0/5	5/0
GPT-4.1 mini	1/0	0/0	0/0	4/0	0/0
GPT-6 Astra	0/0	0/0	0/0	0/0	0/0
Claude Haiku 5.5	4/0	1/0	2/0	0/0	0/0
Claude Sonnet 5.5	0/0	0/0	0/0	0/0	0/0
Gemini 3.8 Flash	0/0	0/2	0/5	0/0	0/0

Table 7: Unsuccessful observations, written as business mismatches / operation-budget exhaustions. Each model–method cell contains 35 observations. S and N denote Touba structured and native. Provider records report no output-limit truncation.

Contract area	Checked obligation
Trace forensics	Reconstruct dispatch, refusal, and review pause from events.
Trigger identity	Accept a valid event; refuse replay and source mismatch.
Memory provenance	Preserve metadata and refuse raw-content or secret leakage into evidence.
Memory contamination	Keep injected memory content out of public traces across reads and writes.
Runtime budgets	Refuse excess tool calls, agent invocations, and dispatch after timeout.
Supply-chain integrity	Refuse a missing lock or changed handler and manifest artifacts.

Table 8: Selected implementation invariants covered by deterministic runtime and CLI tests.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

API model identifier	Sampling	Output limit
gpt-4o-mini-2024-07-18	$T=0$	1,000
gpt-4.1-mini-2025-04-14	$T=0$	1,000
gpt-6-astra	Low effort	4,096
claude-haiku-5-5	Low effort	4,096
claude-sonnet-5-5	Low effort	4,096
gemini-3.8-flash	Low thinking	8,192

Appendix

Table 9: Model identifiers and per-request token ceilings in October 2026. Each model uses the same profile across all five methods; profiles vary between models.

AI Agent Governance

AI agent governance studies how to authorize, constrain, monitor, and hold autonomous and multi-agent systems accountable.

See all 446 papers

AgentBoundary: Counterfactual Evaluation of Safety in Tool-Using LLM Agents
Sep 27, 2026Tianzhuo Yang, Zirui Mi, Yantao Huang +4LLM Agent Evaluation LLM Agents
Peking University · Beijing Academy of Artificial Intelligence
Language-Based Agent Control
May 13, 2026Timothy Zhou, Loris D'Antoni, Nadia PolikarpovaAI Agent Security Information Flow Control
Beyond Task Success: An Evidence-Synthesis Framework for Evaluating, Governing, and Orchestrating Agentic AI
Apr 18, 2026Christopher Koch, Joshua Andreas WellbrockLLM Agent Orchestration AI Agent Evaluation
Independent Researcher

Sep 27, 2026J/K move · Enter open · S save

AgentBoundary: Counterfactual Evaluation of Safety in Tool-Using LLM Agents

Tianzhuo Yang, Zirui Mi, Yantao Huang, Guoxi Zhang+3

Peking University · Beijing Academy of Artificial Intelligence

LLM Agent Evaluation LLM Agents Over-Refusal AI Agent Safety Benchmarks LLM Agent Safety

Open paper AskPDF arXiv

Safety alignment for large language models (LLMs) in conversational settings is largely framed around whether to answer or refuse a request. In agentic settings, however, the same models must decide whether to act as permission-critical evidence emerges during execution. This creates a distinct challenge: apparent risk, action permissibility, and task competence are easily confounded, making agentic over-refusal difficult to distinguish from ordinary task failure. To address this, we introduce AgentBound, the first four-way counterfactual generation-and-evaluation framework for tool-using agent safety. AgentBound transforms the same executable workflow by independently varying apparent risk and action permissibility, enabling controlled comparisons of risky-looking but authorized tasks and routine-looking but unauthorized tasks. These comparisons jointly diagnose over-refusal and unsafe compliance while controlling for task competence. We instantiate AgentBound as a human-validated 4,000-task evaluation suite with trajectory-based and post-state-based judgments. Across 17 model and harness configurations, high safety frequently coexists with poor authorized-task completion: GPT-5.5 blocks 99.5% of routine-looking unauthorized actions yet completes only 28.7% of risky-looking authorized tasks. We further train a lightweight runtime calibration module that improves authorized-task completion by 18.2% on average across 10 evaluated configurations, while improving unsafe-action blocking by 5.4% on average. These show that effective agentic alignment requires action decisions to track permission-relevant execution evidence, rather than refusal strength alone.

Abstract

AI Agent Governance

Explore similar work

AgentBoundary: Counterfactual Evaluation of Safety in Tool-Using LLM Agents

Language-Based Agent Control

Beyond Task Success: An Evidence-Synthesis Framework for Evaluating, Governing, and Orchestrating Agentic AI