RAC: A Relational Agent Contract for Constraint-Governed Engineering Tasks

Kaiyuan Zheng; Fengqing Jiang; Yichen Feng; Zubin Aaron Assadian; Yuetai Li; Bhaskar Ramasubramanian; Basel Alomair; Luyao Niu

cs.AIOct 14, 2026

RAC: A Relational Agent Contract for Constraint-Governed Engineering Tasks

Authors: Kaiyuan Zheng, Fengqing Jiang, Yichen Feng, Zubin Aaron Assadian, Yuetai Li, Bhaskar Ramasubramanian, Basel Alomair, Luyao Niu,

Organizations: University of Washington · Western Washington University · King Abdulaziz City for Science and Technology (KACST)

Abstract

Engineering agents must produce artifacts that satisfy exact, coupled constraints. Existing solutions often require agents to make semantic decisions and also derive consequences such as precise geometry or timing from those decisions, leading to failures in constraint-governed engineering tasks. In this paper, we introduce RAC, a harness-agnostic contract for the action and feedback layer of engineering agents. It specifies authoritative observations, semantic actions, tool-based realization, independent verification, and repair evidence. The agent decides which relations an artifact should satisfy; specialized tools derive their concrete consequences. On PCB placement, routing, and repair tasks, RAC achieves the highest clean-pass rate for all nine evaluated models, averaging 90.9%, compared with 74.0% for the strongest baseline, requirements-only delegation. In job-shop scheduling, direct relational actions achieve 96.3% quality pass versus 59.6% for code-mediated delivery with access to the same scheduling backend. Our evaluations on 141 embedded-code tasks across four microcontroller identify the action contract as a source of agent capability: preserving semantic control while delegating mechanically derivable outputs improves the reliability of constrained artifacts.

Figures & tables

Figure 1: Overview of RAC . RAC defines the agent–tool boundary. The agent observes the exact engineering state. Through relational actions , the agent states relations among named objects, for example placing capacitor C1 near U1, without writing coordinates. Engineering tools derive the consequences of these relations and realize them jointly as a concrete artifact. Independent checks then evaluate the artifact against both design rules and the stated relations. A passing artifact is returned as verified. A realization conflict or a failed check returns repair evidence , which names the conflicting relations and the admissible repairs, and the agent uses it to revise its relational decision.

Figure 2: Illustration of responsibility allocation in a constraint-governed PCB task. Existing geometry methods (left) directly assign both semantic decisions and metric realization to the agent, exposing the artifact to coordinate-level constraint violations. Complete delegation of both decisions to engineering tools (middle) can produce valid geometry but lose the intended relation. RAC (right) preserves the semantic decision with the agent (e.g., placing C1 near U1) while engineering tools derive the coordinates and routes. This division preserves intent while enabling deterministic realization and verification.

Model	Coord.	Coord.+Py	Candidates	Seq. Relations	Req.-Only	RAC
DeepSeek-V4-Flash	10.2%	7.1%	37.8%	64.3%	74.5%	92.9 %
GLM-5.3-Flash	25.5%	20.4%	33.7%	60.2%	68.4%	89.8 %
GPT-5.6-Luna	24.5%	26.5%	41.8%	73.5%	77.6%	95.9 %
GPT-5.6-Sol	29.6%	26.5%	40.8%	65.3%	77.6%	92.9 %
Kimi-K2.7	6.1%	4.1%	22.4%	43.9%	61.2%	77.6 %
Qwen3.6-27B	26.5%	25.5%	34.7%	62.2%	77.6%	93.9 %

Table 1: Clean-pass rates (%) on 98 tasks across six random seeds. The final row averages the nine model-specific rates. Bold marks the highest clean-pass rate in each row. RAC consistently outperforms all baselines across all agents.

Panel of Figure 3 — Figure 3: Clean-pass rate against average token use per episode on the PCB tasks, one panel per model. The token axis is logarithmic. Higher and further left is better. RAC attains the highest clean-pass rate for every model and the lowest token use for eight of the nine, tying Req.-Only on DeepSeek-V4-Flash.

Model	Coord.	Coord.+Py	Candidates	Seq. Relations	Req.-Only	RAC
DeepSeek-V4-Flash	12.5	10.3	13.0	11.2	9.2	9.0
GLM-5.3-Flash	38.2	41.7	35.2	20.4	22.4	19.4
GPT-5.6-Luna	41.8	66.9	23.4	14.9	14.8	9.1
GPT-5.6-Sol	59.1	62.0	37.6	21.7	25.5	19.5
Kimi-K2.7	29.4	37.7	20.7	22.9	21.0	13.9
Qwen3.6-27B	409.4	511.1	194.1	157.4	109.8	88.8

Table 2: Average wall-clock latency (seconds) per episode on PCB tasks. RAC ’s wall time is comparable within a model under the matched transport and worker regime.

	GPT-5.6-Luna						GPT-5.6-Sol
	Clean pass $\uparrow$			Tokens / task $\downarrow$			Clean pass $\uparrow$			Tokens / task $\downarrow$
Harness	Coord.	RAC	$\Delta$	Coord.	RAC	$\Delta$	Coord.	RAC	$\Delta$	Coord.	RAC	$\Delta$
Native	29.6%	92.9%	$+$ 63.3	22.9k	16.1k	$-$ 6.8k	26.5%	95.9%	$+$ 69.4	22.4k	14.5k	$-$ 7.9k
Codex	24.5%	91.8%	$+$ 67.3	71.4k	53.2k	$-$ 18.2k	22.7%	93.9%	$+$ 71.2	69.8k	64.9k	$-$ 4.9k
Claude Code	26.5%	91.8%	$+$ 65.3	81.9k	60.0k	$-$ 21.9k	32.7%	94.9%	$+$ 62.2	84.8k	78.8k	$-$ 6.0k
OpenCode	25.5%	92.9%	$+$ 67.4	47.4k	35.1k	$-$ 12.3k	26.5%	93.9%	$+$ 67.4	47.6k	38.4k	$-$ 9.2k

Table 3: Clean-pass rates and average token use per task of Coord. and RAC on the 98 PCB test tasks under four agent harnesses.

\Delta

is RAC minus Coord.;

\uparrow

\downarrow

mark whether higher or lower is better. Within every harness and model, RAC achieves a higher clean-pass rate with fewer tokens.

Model	Code	RAC	$\Delta$	Code	RAC	$\Delta$
	Final proposal			First proposal
DeepSeek-V4-Flash	26.7%	100.0 %	+73.3	10.0%	100.0 %	+90.0
GLM-5.3-Flash	63.3%	100.0 %	+36.7	46.7%	100.0 %	+53.3
GPT-5.6-Luna	86.7%	100.0 %	+13.4	83.3%	96.7 %	+13.3
GPT-5.6-Sol	100.0 %	100.0 %	0.0	100.0 %	100.0 %	0.0
Kimi-K2.7	30.0%	96.7 %	+66.7	23.3%	100.0 %	+76.7

Table 4: Final proposal and first proposal quality-pass rates (%) on LA01–LA30.

\Delta

is the gain of RAC over Code in percentage points. RAC improves scheduling action delivery and delivers usable schedules earlier.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Coord.	Coord.+Py	Candidates	Seq. Relations	Req.-Only	RAC
DeepSeek-V4-Flash	7.5k	8.8k	10.2k	8.8k	6.6 k	6.6 k
GLM-5.3-Flash	11.0k	11.8k	12.8k	10.5k	9.3k	8.8 k
GPT-5.6-Luna	9.9k	10.1k	11.4k	9.3k	7.2k	6.0 k
GPT-5.6-Sol	9.4k	9.7k	11.1k	9.2k	7.6k	6.1 k
Kimi-K2.7	10.0k	9.6k	10.5k	11.0k	8.7k	7.2 k
Qwen3.6-27B	36.6k	43.2k	25.8k	20.6k	14.0k	13.3 k

Appendix

Table 5: Average token use per episode on PCB tasks. RAC improves token efficiency.

Adaptation method	Adaptation clean-pass	Overall clean-pass	Paired difference [95% CI]
Frozen coordinates	187/459 (40.7%)	187/702 (26.6%)	$+48.8$ [ $37.8$ , $60.7$ ]
Affine coordinates	239/459 (52.1%)	239/702 (34.0%)	$+37.5$ [ $28.0$ , $46.6$ ]
Empty-relation solver	212/459 (46.2%)	212/702 (30.2%)	$+43.4$ [ $27.1$ , $59.8$ ]
Preserve-legal coordinates	267/459 (58.2%)	267/702 (38.0%)	$+31.4$ [ $20.6$ , $42.2$ ]
Numeric best-of-three	301/459 (65.6%)	301/702 (42.9%)	$+24.0$ [ $14.1$ , $34.5$ ]
RAC relational re-solve	411/459 (89.5%)	411/702 (58.5%)	—

Appendix

Table 6: Robustness to board and footprint changes. Adaptation rates cover 459 changes to initially successful layouts; overall rates include all 702 attempted changes. Differences compare relational re-solving with each control, in percentage points, with descriptive 95% bootstrap intervals that group results by placement family.

Transform	Frozen	Numeric best-of-three	Relational re-solve
Board shrink 20%	23/153 (15.0%)	73/153 (47.7%)	118/153 (77.1%)
Board expansion 20%	46/153 (30.1%)	110/153 (71.9%)	141/153 (92.2%)
Footprint shrink 25%	118/153 (77.1%)	118/153 (77.1%)	152/153 (99.3%)

Appendix

Table 7: Clean-pass rates by geometric change, among the 153 initially successful layouts.

Task	Requirement	Verification of realized artifact
P013: placement	Associate connector J1 with the left board edge and J2 with the right edge.	Each connector is 1.00 mm from its requested edge, meeting the task’s 1.00 mm limit.
R010: routing	Match the routed lengths of nets N2 and N3.	The measured mismatch is 0.00 mm, within the task oracle’s 1.00 mm tolerance.
C008: combined	Place J1 at the top edge, decoupling capacitors C1–C3 near U1 pins, and preserve the R1/C4 and R2/C5 filter groups.	J1 is 1.00 mm from the top edge; capacitor-to-pin distances are 2.00, 1.50, and 1.58 mm; filter-group spreads are 2.69 and 3.04 mm.

Appendix

Table 8: Recorded PCB examples. All three realized artifacts pass their task-specific relation checks, complete routing, and zero-design-rule-violation checks.

Model	$n$	RawSDK	SignaturesOnly	SourceLite	RAC
DeepSeek-V4-Flash	132	43.9%	39.4%	42.4%	68.2%
Qwen3.6-27B	129	39.5%	33.3%	41.9%	60.5%
GLM-5.3-Flash	106	41.5%	38.7%	40.6%	70.8%
Kimi-K2.7	130	43.8%	46.2%	44.6%	63.1%
GPT-5.6-Luna	140	59.3%	55.7%	57.9%	87.1%
GPT-5.6-Sol	121	62.8%	59.5%	66.1%	78.5%

Appendix

Table 9: Pass rates by model on complete four-interface pairs.

n

is the number of tasks for that model with known outcomes and token counts under every interface; the pooled row contains 998 model–task pairs. The best pass rate in each row is bold.

Model	$n$	RawSDK	SignaturesOnly	SourceLite	RAC
DeepSeek-V4-Flash	132	894,940	640,572	846,254	525,295
Qwen3.6-27B	129	1,478,250	1,198,840	1,405,793	858,130
GLM-5.3-Flash	106	762,785	575,674	725,054	446,729
Kimi-K2.7	130	765,991	538,520	726,882	468,364
GPT-5.6-Luna	140	944,558	701,345	906,761	514,919
GPT-5.6-Sol	121	817,766	616,651	784,798	476,405

Appendix

Table 10: Model input and output tokens on the same complete pairs as Table 9 . Each entry sums tokens over the model’s

n

paired tasks; tool-generated code is excluded. The lowest token total in each row is bold.

Item	Verified value
Constructed designs	10
Assignments / output bits exhaustively checked	92 / 208
Correct actions accepted	10/10
One-row action changes rejected	10/10
Modified artifacts with forged receipts rejected	10/10
Supplied repair traces, fail $\rightarrow$ pass	10/10

Appendix

Table 11: Bounded logic calibration using supplied actions and repairs, with no model calls.

Domain	Traces	Matching outputs	Direct median (ms)	RAC median (ms)	Paired $\Delta$ (ms)	Overhead	95% interval
Logic	400	400/400	1.211	1.658	$+$ 0.398	$+$ 30.84%	[ $27.57$ , $34.94$ ]
PCB	24	24/24	59.737	59.451	$-$ 0.158	$-$ 0.66%	[ $-1.90$ , $0.36$ ]
Scheduling	24	24/24	2370.016	2372.419	$-$ 0.304	$-$ 0.24%	[ $-1.01$ , $0.56$ ]
Embedded	24	24/24	4011.018	3940.803	$+$ 38.725	$+$ 4.21%	[ $-1.46$ , $10.18$ ]

Appendix

Table 12: Paired execution through the direct driver and RAC . Outputs, callbacks, and message bytes match for every trace. Paired

\Delta

is the median over traces of the per-trace runtime difference ( RAC minus direct), so it can differ in value and sign from the difference between the two medians. Overhead is the percentage difference in total runtime,

(T_{\textsc{RAC}}-T_{\text{direct}})/T_{\text{direct}}

, with a descriptive 95% bootstrap interval that resamples by test case.

LLM Agent Reliability

Large language model (LLM) agent reliability concerns dependable, safe, and verifiable behavior across tools, multi-step tasks, and changing environments.

See all 843 papers

Trust Is Not a Score: Runtime Assurance Contracts for High-Risk AI Agents
Sep 30, 2026Serhii ZabolotniiAI Agent Safety Runtime Enforcement for AI Agents
Cherkasy State Business College, Cherkasy, Ukraine · State Scientific Research Institute of Armament and Military Equipment Testing and Certification, Cherkasy, Ukraine · healthPrecision, New York, NY, USA
ANCHOR: Assessing Naturalistic Constraint Hierarchies and Ordering in Repository-Grounded Agentic Coding
Oct 12, 2026Wenqiang Zhu, Xinping Lei, Zhiyi Lai +17Software Engineering Agents AI Agent Evaluation
Southern University of Science and Technology · Nanjing University · Independent Researcher +3
Proof-Carrying Contracts: Commitments AI Agents Cannot Break
Oct 20, 2026Carlo Ciliberto, Alessandro Rudi, Enrico TassiRuntime Enforcement for AI Agents AI Agent Safety
AI Centre, Computer Science Department, University College London, London, UK · Praxis Lab, The Italian Institute of Artificial Intelligence (AI4I), Turin, Italy · SDA Bocconi School of Management. Milano, Italy +1

Sep 30, 2026J/K move · Enter open · S save

Trust Is Not a Score: Runtime Assurance Contracts for High-Risk AI Agents

Serhii Zabolotnii

Cherkasy State Business College, Cherkasy, Ukraine · State Scientific Research Institute of Armament and Military Equipment Testing and Certification, Cherkasy, Ukraine · healthPrecision, New York, NY, USA

AI Agent Safety Runtime Enforcement for AI Agents AI Agent Governance Runtime Assurance

Open paper AskPDF arXiv

Benchmarks, audits, and agent protocols describe performance, permissions, and repair, but not how observed evidence should change an agent's authority during a consequential task. We call this the assurance-transition gap. We propose a Runtime Assurance Contract (RAC), a policy-level formal schema binding autonomy boundaries, component eligibility, evidence state, transition policy, human-review capacity, and non-compensatory gates. Under RAC, soft metrics may inform routing, whereas a failed or unknown mandatory gate forces retry, switch, escalation, deferral, or stop; aggregate performance cannot authorize action. We define the contract, an evidence record, a permission rule, and five invariants, and illustrate them in clinical, industrial, and judicial failure probes. We then report a deterministic failure-injection study in agentic coding: 280 constructed cases evaluated by a gate conjunction, a score-only rule, and a restricted protocol baseline. At the published example weights and threshold, the score rule admits 80 of 100 block-required injections and all 40 review-required injections. Tuned in hindsight, it matches the conjunction on this corpus. For positive weights, a positive threshold, binary risk signals, zero-signal controls, and an injected case firing each signal alone, we show that exact agreement holds if and only if the threshold does not exceed the smallest weight. A separate set of 18 hand-authored traces checks version-pinned evidence and review transitions against simpler policy variants. In a further prospective synthetic holdout of 24 episodes, two blinded LLM judges assign identical labels to all 72 action attempts; RAC and a separately implemented full stateful baseline both match these labels. These studies test mechanisms on synthetic cases; they establish neither deployed safety nor cross-domain effectiveness.

Abstract

LLM Agent Reliability

Explore similar work

Trust Is Not a Score: Runtime Assurance Contracts for High-Risk AI Agents

ANCHOR: Assessing Naturalistic Constraint Hierarchies and Ordering in Repository-Grounded Agentic Coding

Proof-Carrying Contracts: Commitments AI Agents Cannot Break