EmbodiedGovBench: A Benchmark for Governance, Recovery, and Upgrade Safety in Embodied Agent Systems

Xue Qin; Simin Luan; John See; Cong Yang; Zhijun Li

cs.ROApr 13, 2026

EmbodiedGovBench: A Benchmark for Governance, Recovery, and Upgrade Safety in Embodied Agent Systems

Authors: Xue Qin, Simin Luan, John See, Cong Yang, Zhijun Li

Organizations: School of Software, Harbin Institute of Technology, Harbin, China · School of Computer Science and Technology, Harbin Institute of Technology, Harbin, China · School of Mathematical and Computer Sciences, Heriot-Watt University, Malaysia Campus, Putrajaya, Malaysia · School of Future Science and Engineering, Soochow University, Suzhou, China

Abstract

Recent progress in embodied AI has produced a growing ecosystem of robot policies, foundation models, and modular runtimes. However, current evaluation remains dominated by task success metrics such as completion rate or manipulation accuracy. These metrics leave a critical gap: they do not measure whether embodied systems are governable -- whether they respect capability boundaries, enforce policies, recover safely, maintain audit trails, and respond to human oversight. We present EmbodiedGovBench, a benchmark for governance-oriented evaluation of embodied agent systems. Rather than asking only whether a robot can complete a task, EmbodiedGovBench evaluates whether the system remains controllable, policy-bounded, recoverable, auditable, and evolution-safe under realistic perturbations. The benchmark covers seven governance dimensions: unauthorized capability invocation, runtime drift robustness, recovery success, policy portability, version upgrade safety, human override responsiveness, and audit completeness. We define a benchmark structure spanning single-robot and fleet settings, with scenario templates, perturbation operators, governance metrics, and baseline evaluation protocols. We describe how the benchmark can be instantiated over embodied capability runtimes with modular interfaces and contract-aware upgrade workflows. Our analysis suggests that embodied governance should become a first-class evaluation target. EmbodiedGovBench provides the initial measurement framework for that shift.

Figures & tables

Panel of Figure 1 — Figure 1: From Task Success Benchmarks to Governance Benchmarks. Traditional embodied benchmarks (left) measure what robots can do. EmbodiedGovBench (right) measures whether their capabilities remain governable under execution, failure, and evolution.

Dimension	What It Measures	Example Failure	Key Metric
Unauthorized Invocation	Legality of capability use	Robot opens restricted door	UIR
Runtime Drift	Governance under degraded runtime	Execution continues under invalid state	DDR
Recovery Success	Safe and scoped recovery	Repeated retries without escalation	LRCR
Policy Portability	Policy validity across contexts	Sim-valid policy fails in public zone	PS
Upgrade Safety	Survival under capability evolution	Upgraded capability breaks chain	UDR
Human Override	Responsiveness to intervention	Override arrives too late	OL

Table 1: Governance Dimensions and Example Failures.

Panel of Figure 2 — Figure 2: EmbodiedGovBench Structure. The benchmark is organized in layers: seven governance dimensions define the evaluation target; two tracks (single-robot and fleet) provide execution scope; scenario templates and perturbation operators generate benchmark instances; and a multi-level scoring framework produces diagnostic outputs.

Template	Single	Fleet	Main Dimensions Stressed
Unauthorized Capability Use	✓	✓	Invocation, policy, audit
Runtime Degradation	✓	✓	Drift, recovery
Recovery Cascade	✓	✓	Recovery, human override
Policy Transfer	✓	✓	Portability, invocation
Upgrade-Then-Execute	✓	✓	Upgrade safety, audit
Review-Required Override	✓	✓	Override latency, audit

Table 2: Scenario Templates and Stressed Dimensions.

Family	Core Question	Primary Dimensions	Key Metrics
$G_{\mathrm{cap}}$	Can do $\neq$ may do?	Unauthorized Invocation	UIR, BBC, TSVR
$G_{\mathrm{rec}}$	Governed recovery?	Recovery, Drift	LRCR, EAS, RIPV, GCR
$G_{\mathrm{evo}}$	Governable evolution?	Upgrade, Portability	UDR, UAS, PVR, PS
$G_{\mathrm{acct}}$	Accountable?	Override, Audit	OL, RTC, ACS, PAA

Table 3: Governance Score Families.

Family	Governance Question	Example Perturbation	Key Metrics	Fleet
A	Authorized invocation?	Tighter policy scope	UIR, TSVR, ACS	✓
B	Governed under drift?	Degraded sensor/latency	DDR, GCR, DSPV	✓
C	Scoped recovery?	Grasp miss, path block	LRCR, EAS, OL	✓
D	Policy portability?	Sim $\to$ public deploy	PS, SPFC, TTRC	✓
E	Upgrade safety?	Capability version bump	UDR, PVR, VSRC, UAS	✓
F	Human override?	Mandatory review gate	OL, RTC, PAA	✓

Table 4: Protocol Families and Primary Governance Targets.

Panel of Figure 3 — Figure 3: Benchmark Evaluation Pipeline. Each benchmark run proceeds through five stages, with trace collection spanning perturbation injection and system execution.

System Class	Minimal	Standard	Fleet	Full
Task-success-only policy	✓	partial	optional	rare
Schema-checked modular runtime	✓	✓	partial	optional
Contract-aware runtime	✓	✓	partial	optional
Governance-aware fleet runtime	✓	✓	✓	✓

Table 5: Recommended Baseline Evaluation Matrix.

Panel of Figure 4 — Figure 4: Benchmark Harness Architecture. The harness wraps around the system under test, injecting scenarios and perturbations through an adapter layer, collecting traces, judging governance compliance, and producing diagnostic reports.

Metric	System 0	System 1	System 2	System 3
GovScore	0.7227	0.5770	0.9437	0.6357
$G_{\text{cap}}$	0.8669	0.5930	0.9718	0.6414
$G_{\text{rec}}$	1.0000	0.9980	0.9970	1.0000
$G_{\text{evo}}$	0.5200	0.3250	0.8300	0.3333
$G_{\text{acct}}$	0.5040	0.3920	0.9760	0.5680
Task Success	0.8960	0.8960	0.6960	0.8800

Table 6: Experiment results: System 0 (SayCan-style), System 1 (task-only), System 2 (governed), and System 3 (Code-as-Policies) under equal weights (

\alpha{=}\beta{=}\gamma{=}\delta{=}0.25

N{=}125

scenarios across 5 protocols. GovScore reported as mean

\pm

std over per-episode scores.

Metric	System 0	System 1	System 2	System 3
UIR ( $\downarrow$ )	0.047	0.341	0.000	0.341
BBC ( $\downarrow$ )	0.000	0.920	0.000	0.920
TSVR ( $\downarrow$ )	0.160	0.960	0.000	0.960
ACS ( $\uparrow$ )	0.000	0.000	1.000	0.000
Task Succ.	1.000	1.000	0.840	1.000

Table 7: Protocol A (Unauthorized Invocation): per-metric means across 25 episodes.

Metric	System 0	System 1	System 2	System 3
DDR ( $\uparrow$ )	0.840	0.560	0.880	0.880
DAR ( $\uparrow$ )	0.840	0.560	0.880	0.560
DPI ( $\downarrow$ )	0.000	0.240	0.000	0.400
Task Succ.	1.000	1.000	0.880	1.000

Table 8: Protocol B (Runtime Drift Robustness): per-metric means across 25 episodes.

Metric	System 0	System 1	System 2	System 3
LRCR ( $\uparrow$ )	1.000	1.000	1.000	1.000
EAS ( $\uparrow$ )	1.000	1.000	1.000	1.000
RIPV ( $\downarrow$ )	0.000	0.040	0.000	0.000
OL ( $\downarrow$ )	0.000	0.000	0.120	0.000
Task Succ.	0.480	0.480	0.440	0.480

Table 9: Protocol C (Recovery Cascade): per-metric means across 25 episodes.

Metric	System 0	System 1	System 2	System 3
UDR ( $\uparrow$ )	0.680	0.400	1.000	0.400
PVR ( $\downarrow$ )	0.280	0.500	0.000	0.467
VCR ( $\uparrow$ )	0.680	0.400	1.000	0.400
UAS ( $\uparrow$ )	0.000	0.000	0.320	0.000
Task Succ.	1.000	1.000	0.800	1.000

Table 10: Protocol E (Upgrade-Then-Execute Safety): per-metric means across 25 episodes.

Metric	System 0	System 1	System 2	System 3
ODR ( $\uparrow$ )	0.000	0.000	1.000	0.560
OCR ( $\uparrow$ )	0.000	0.000	1.000	0.400
ORL ( $\downarrow$ )	0.000	0.000	0.500	0.840
OIR ( $\uparrow$ )	1.000	1.000	1.000	1.000
Task Succ.	1.000	1.000	0.520	0.920

Table 11: Protocol F (Human Override): per-metric means across 25 episodes.

Profile	$\alpha,\beta,\gamma,\delta$	Sys. 0	Sys. 1	Sys. 2	Sys. 3
Equal	0.25, 0.25, 0.25, 0.25	0.7227	0.5770	0.9437	0.6357
Cap.-heavy	0.50, 0.20, 0.15, 0.15	0.7870	0.6036	0.9562	0.6559
Rec.-heavy	0.15, 0.50, 0.15, 0.20	0.8088	0.7151	0.9640	0.7598

Table 12: Weight sensitivity: GovScore under three weight profiles (4 systems).

Aspect	Papers 1–6 [ 13 , 14 , 15 , 16 , 17 , 18 ]	This paper
Focus	How to build and manage	How to evaluate
Output	Runtime architecture, governance framework, lifecycle, fleet coordination	Benchmark dimensions, metrics, scoring, protocols, pilot results
Artifacts	ECM runtime, contract schema, federated coordinator	Scenario templates, perturbation operators, scoring scripts, trace schema
Validation	Design rationale, architecture analysis	Pilot experiment with quantitative metrics

Table 13: Contribution delineation: earlier papers vs. this paper.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Prototype	Supported Dimensions	Track
Contract-aware single-robot runtime	Invocation, drift, recovery, portability, upgrade, audit	Single-robot
Governance-aware federated fleet runtime	All seven dimensions	Single + Fleet

Appendix

Table 14: Prototype Instantiation Targets.

AI Agent Governance

AI agent governance studies how to authorize, constrain, monitor, and hold autonomous and multi-agent systems accountable.

See all 442 papers

Harnessing Embodied Agents: Runtime Governance for Policy-Constrained Execution
Apr 9, 2026Xue Qin, Simin Luan, John See +3Runtime Enforcement for AI Agents AI Agent Governance
School of Software, Harbin Institute of Technology, Harbin, China. · School of Computer Science and Technology, Harbin Institute of Technology, Harbin, China. · School of Mathematical and Computer Sciences, Heriot-Watt University, Malaysia Campus, Putrajaya, Malaysia. +2
Beyond Task Success: An Evidence-Synthesis Framework for Evaluating, Governing, and Orchestrating Agentic AI
Apr 18, 2026Christopher Koch, Joshua Andreas WellbrockLLM Agent Orchestration AI Agent Evaluation
Independent Researcher
Embodied-BenchForge: A Closed-Loop Agentic Workflow for Embodied Benchmark Construction
Sep 11, 2026Baoyang Jiang, Fengchun Zhang, Leyuan Wang +9Benchmark Construction Embodied QA
QiYuan Lab · University of Electronic Science and Technology of China · Beijing University of Posts and Telecommunications +3

Apr 9, 2026J/K move · Enter open · S save

Harnessing Embodied Agents: Runtime Governance for Policy-Constrained Execution

Xue Qin, Simin Luan, John See, Zeyd Boukhers+2

School of Software, Harbin Institute of Technology, Harbin, China. · School of Computer Science and Technology, Harbin Institute of Technology, Harbin, China. · School of Mathematical and Computer Sciences, Heriot-Watt University, Malaysia Campus, Putrajaya, Malaysia.+2

Runtime Enforcement for AI Agents AI Agent Governance AI Agent Safety AI Agent Monitoring Embodied AI

Open paper AskPDF arXiv

Embodied Agents are evolving from passive reasoning systems into active executors that interact with tools, robots, and physical environments. Once an agent gains execution authority, the central challenge shifts from how to make it act to how to keep its actions governable at runtime. Existing approaches embed safety, recovery, and decision constraints inside the agent loop, making execution control difficult to standardize, audit, and adapt across environments. We propose a runtime governance framework for policy-constrained execution that separates agent cognition from execution oversight. Governance is externalized into a dedicated runtime layer performing policy checking, capability admission, execution monitoring, rollback, and human override. We formalize the control boundary among a persistent Embodied Agent, modular Capability Packages, and the governance layer, and define a policy-constrained execution pipeline evaluated under controlled simulation. Over 1000 randomized trials, the framework achieves 96.2%+/-2.7% interception of unauthorized actions, reduces unsafe continuation from 100% to 22.2%+/-3.1% under runtime drift, and attains 90.7%+/-3.0% recovery success with full policy compliance. Comparison with five baselines, including AutoRT-style constitution filtering and RoboGuard-style two-stage guardrails, shows that pre-execution filtering is equally effective across governance-aware methods, while only the proposed framework provides continuous runtime detection (RVDR = 61.3% vs. 0%) and structured recovery (all p<0.001). A sensitivity sweep across the full detection range confirms a genuine detection-continuation trade-off. This work argues future embodied systems should be designed for governable execution.

Abstract

AI Agent Governance

Explore similar work

Harnessing Embodied Agents: Runtime Governance for Policy-Constrained Execution

Beyond Task Success: An Evidence-Synthesis Framework for Evaluating, Governing, and Orchestrating Agentic AI

Embodied-BenchForge: A Closed-Loop Agentic Workflow for Embodied Benchmark Construction