RCBSF: A Multi-Agent Framework for Automated Contract Revision via Stackelberg Game

Shijia Xu; Yu Wang; Xiaolong Jia; Zhou Wu; Kai Liu; April Xiaowen Dong

cs.CLApr 12, 2026

RCBSF: A Multi-Agent Framework for Automated Contract Revision via Stackelberg Game

Authors: Shijia Xu, Yu Wang, Xiaolong Jia, Zhou Wu, Kai Liu, April Xiaowen Dong

Organizations: Chongqing University, China · Queen Mary University of London, UK · Chongqing Key Laboratory of Big Data Intelligence and Privacy Computing, China · Fangda Partners, China

Abstract

Despite the widespread adoption of Large Language Models (LLMs) in Legal AI, their utility for automated contract revision remains impeded by hallucinated safety and a lack of rigorous behavioral constraints. To address these limitations, we propose the Risk-Constrained Bilevel Stackelberg Framework (RCBSF), which formulates revision as a non-cooperative Stackelberg game. RCBSF establishes a hierarchical Leader Follower structure where a Global Prescriptive Agent (GPA) imposes risk budgets upon a follower system constituted by a Constrained Revision Agent (CRA) and a Local Verification Agent (LVA) to iteratively optimize output. We provide theoretical guarantees that this bilevel formulation converges to an equilibrium yielding strictly superior utility over unguided configurations. Empirical validation on a unified benchmark demonstrates that RCBSF achieves state-of-the-art performance, surpassing iterative baselines with an average Risk Resolution Rate (RRR) of 84.21% while enhancing token efficiency. Our code is available at https://github.com/xjiacs/RCBSF .

Figures & tables

Figure 1: Comparison of legal contract generation workflows between the Baseline (Standard LLM) and the Risk-Constrained Bilevel Stackelberg Framework (RCBSF).

Figure 2: Illustration of the comparison between traditional contract revision paradigms and our proposed framework. (a) Baseline Methods rely on static template assembly or simple unidirectional generation based on user requirements. These approaches are typically static and prone to errors, often ignoring hidden risks due to the lack of iterative feedback mechanisms; (b) In contrast, our proposed Risk-Constrained Bilevel Stackelberg Framework (RCBSF) models the revision process as a hierarchical bilevel optimization game. It employs a Global Prescriptive Agent (GPA) as the leader to construct strict risk-budget objectives, driving a follower system consisting of a Revision Agent and an Auditor Agent. Through a Multi-Round Revision-Audit Interaction, these agents iteratively converge towards a solution that dynamically minimizes residual risk and consumption costs while preserving semantic quality.

Model	Method	PrivacyQA			ContractNLI			MAUD			CUAD			ALL (Avg)
Model	Method	CQ	RRR	TES	CQ	RRR	TES	CQ	RRR	TES	CQ	RRR	TES	CQ	RRR	TES
Qwen2.5-7B-Base	Standard	71.45	67.34	72.89	70.12	65.45	70.56	69.88	72.34	71.45	70.23	65.67	70.12	70.42	67.70	71.26
	CoT	73.89	69.56	74.34	72.45	67.89	72.67	72.12	74.56	73.89	72.56	68.34	72.89	72.76	70.09	73.45
	RAG	77.23	73.89	80.56	77.12	72.45	78.34	76.56	78.45	78.12	76.45	71.23	77.56	76.84	74.01	78.65
	Iteration	78.56	75.45	81.78	75.89	74.56	80.23	77.45	80.89	79.56	75.23	73.45	79.12	76.78	76.09	80.17
	RCBSF	84.67	79.45	85.67	83.89	77.92	83.56	83.45	87.12	84.23	82.67	78.34	83.45	83.67	80.71	84.23

Table 1: Main Results on Legal Contract Revision. We compare different LLM backbones across four datasets. CQ denotes Contract Quality as the average score scaled to 0-100 that evaluates clarity, rigor, balance, and professionalism. RRR indicates Risk Resolution Rate (%). TES represents Token Efficiency Score (%). The dark blue and light blue cells indicate the best and second-best performance within each model group, respectively.

Figure 3: Fine-grained Quality Metrics Breakdown. We present a diagonal block heatmap illustrating the performance of the Qwen2.5-7B-Chat model across four fine-grained quality metrics (i.e., Clarity, Rigor, Balance, and Professionalism) for each individual dataset.

Model Variant	RRR (%)	$\Delta$	TES (%)	$\Delta$	BLEU-4	Win Rate
Full RCBSF (Ours)	84.21	–	87.29	–	85.65	–
w/o 5-Dim Constraints	73.15	-11.06	76.25	-11.04	82.15	15.2%
w/o Budget Penalty ( $\lambda=0$ )	84.85	+0.64	75.15	-12.14	83.45	35.8%
w/o Iterative ( $K=1$ )	76.45	-7.76	90.45	+3.16	84.25	28.4%
w/o Q-Score Weighting	79.35	-4.86	81.15	-6.14	84.85	41.5%

Table 2: Ablation Study on the Qwen2.5-7B-Chat Backbone. We investigate the contribution of each component in the RCBSF framework. RRR: Risk Resolution Rate (%). TES: Token Efficiency Score (Risks tokens, %). Win Rate: Pairwise human preference against the Full model. The

\Delta

columns show the relative drop compared to the full model.

Appendix figures & tables20 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Num of Samples	Num of Categories
PrivacyQA	7	2
ContractNLI	92	7
MAUD	150	7
CUAD	462	25
Total	711	41

Appendix

Table 3: Statistics of the processed Unified Legal Benchmark. The dataset aggregates 711 samples from diverse legal sources, covering 41 distinct categories to ensure broad domain coverage.

Figure 5: Distribution of Contract Categories. The dataset features a long-tail distribution across 41 legal domains, with significant representation in major categories like Merger and Disclosure.

Panel of Figure 6 — Figure 6: The automated classification logic used in Stage 0 to standardize noisy raw file headers into canonical legal categories.

Panel of Figure 7 — Figure 7: The instruction set for LLM used to generate standardized, anonymized legal templates from raw source text.

Panel of Figure 8 — Figure 8: The adversarial prompt for LLM, designed to generate the standard truth risk dataset used for calculating the Risk Resolution Rate (RRR).

Figure 9: The architectural overview of the Risk-Constrained Bilevel Stackelberg Framework (RCBSF). The system is organized into two strategic levels: (1) The Outer Stackelberg Leader Loop , where the Leader Agent defines the strategic environment by mapping risk category weights to prompt guidance and optimizing the global utility functional. (2) The Inner Follower Optimization Loop, where the Revision Agent and Audit Agent engage in a multi-round adversarial game. The cycle continues until the generation stabilizes, ensuring the final contract $x^{*}$ satisfies the Leader’s constraints via the minimization objective.

Figure 10: Effectiveness vs. Efficiency (

K

). The left axis (blue) shows the Risk Resolution Rate, while the right axis (red) shows the normalized Token Cost.

K=3

represents the optimal trade-off point.

Figure 11: Impact of Temperature (

\tau

). The curve demonstrates that a balanced temperature (

\tau=1.0

) significantly outperforms both conservative (

\tau=0.5

) and high-entropy (

\tau=2.0

) settings.

Panel of Figure 12 — Figure 12: The strict verification prompt used to calculate RRR. It requires evidence of substantive changes rather than mere stylistic edits.

Dimension	High Quality (80–100) 90–100: Excellent; 80–89: Good	Borderline (60–79) 70–79: Above Avg; 60–69: Barely Acceptable	Deficient (0–59) 40–59: Problematic; 0–39: Severe
Clarity	Precise & Unambiguous. Key terms consistent; rights/obligations expressed in enforceable terms. Defect: Minor ambiguity not impairing execution.	Workable but Vague. Main obligations understandable; reliance on vague terms (e.g., “reasonable”) without standards; occasional inconsistency. Defect: Requires supplementary interpretation.	Chaotic & Undefined. Essential clauses lack operational criteria; contradictory statements; unclear boundaries of rights. Defect: Unusable; chaotic terminology.
Rigor	Logical & Complete. Clear hierarchy; precise cross-references; strong internal consistency. Defect: Minor numbering/formatting issues only.	Loose Structure. Usable but contains logical gaps; mixed organization; frequent incorrect references. Defect: Logical jumps exploitable in disputes.	Disorganized. “Pile of clauses” with overlapping provisions; missing references; no systematic organization. Defect: Cannot support dispute resolution.
Balance	Symmetrical. Fair risk allocation; remedies apply equally; no significant unfair benefits. Defect: Minor non-core tilts.	Surface Symmetry. Appears balanced but hides practical tilts; force majeure or procedural clauses favor one party. Defect: Mostly fair core rights, but biased details.	One-Sided. Extreme asymmetry; unilateral termination/caps without reciprocity; effectively a single-party tool. Defect: Clear lack of reciprocal remedies.
Professionalism	Formal & Standard. Professional revision; standard formatting; consistent terminology; error-free. Defect: Occasional small typos.	Mixed Quality. Noticeable grammar/format issues; occasional colloquialisms or ambiguous everyday expressions. Defect: Reads like a draft or informal memo.	Unprofessional. Highly colloquial; messy formatting; riddled with spelling/grammar errors. Defect: Lacks legal terminology/structure.

Appendix

Table 4: Condensed scoring rubric for Contract Quality. The Judge Agent selects a band (High/Borderline/Deficient) based on the qualitative descriptors and fine-tunes the score within the specific 10-point ranges (e.g., 90–100 vs. 80–89).

Model	Method	PrivacyQA					ContractNLI					MAUD					CUAD					ALL (Avg)
Model	Method	Cla	Rig	Bal	Pro	Avg	Cla	Rig	Bal	Pro	Avg	Cla	Rig	Bal	Pro	Avg	Cla	Rig	Bal	Pro	Avg	Cla	Rig	Bal	Pro	Avg
Qwen2.5-7B-Base	Standard	73.12	68.45	70.33	73.89	71.45	72.11	66.89	69.23	72.25	70.12	71.34	67.56	69.12	71.51	69.88	71.67	67.45	69.34	72.45	70.23	72.06	67.59	69.51	72.53	70.42
	CoT	75.34	71.12	72.45	76.65	73.89	74.23	69.56	71.34	74.67	72.45	73.56	69.89	71.23	73.79	72.12	74.12	69.78	71.56	74.77	72.56	74.31	70.09	71.65	74.97	72.76
	RAG	78.56	74.23	76.12	79.99	77.23	78.34	74.12	76.56	79.45	77.12	77.89	74.12	75.89	78.33	76.56	78.12	73.89	75.45	78.34	76.45	78.23	74.09	76.01	79.03	76.84
	Iteration	79.88	75.67	77.45	81.23	78.56	77.45	72.89	74.11	79.12	75.89	78.56	75.34	76.78	79.13	77.45	76.89	72.56	73.45	78.02	75.23	78.19	74.12	75.45	79.37	76.78
	RCBSF	81.01	82.99	87.43	87.25	84.67	80.41	82.46	86.21	86.48	83.89	80.01	82.69	83.99	87.11	83.45	80.15	82.51	80.31	87.71	82.67	80.40	82.66	84.49	87.14	83.67

Appendix

Table 5: Fine-grained Quality Metrics Breakdown. We report Clarity ( Cla ), Rigor ( Rig ), Balance ( Bal ), and Professionalism ( Pro ) for each dataset and their average ( ALL ). The dark blue and light blue cells indicate the best and second-best performance within each model group.

Figure 13: Heatmap of Fine-Grained Quality Metrics. Multi-model-group variants’ performance across multi-tasks. Dark magenta is the best in group, dark green is the worst in group. Metric values range from 57–92.

Configuration	HR ( $\downarrow$ )	FFR ( $\downarrow$ )	RRR ( $\uparrow$ )
Full RCBSF	4.23%	7.35%	84.21%
w/o Evidence ( $e_{k}$ )	18.68%	11.24%	76.25%
w/o Suggestion ( $s_{k}$ )	5.19%	19.86%	75.14%
w/o Location ( $l_{k}$ )	12.47%	9.54%	78.19%

Appendix

Table 6: Component-wise Ablation. Removing specific dimensions from the Leader’s instruction leads to distinct failure modes.

Panel of Figure 14 — Figure 14: A complete revision flow for a Liability Clause. The Leader’s specific suggestion (highlighted in yellow) serves as a hard constraint that the Follower eventually satisfies in the final output (green).

Panel of Figure 15 — Figure 15: Detailed visualization of the multi-risk resolution in a commercial software context. The GPA successfully identifies three disparate risks (Fairness, IP, Financial). The Inner Loop demonstrates the system’s self-correction capability: while Iteration #1 fails to fully transfer IP rights, the LVA’s audit feedback forces the CRA to execute a legally binding assignment in Iteration #3, satisfying all Leader constraints simultaneously.

Panel of Figure 16 — Figure 16: The structured prompt used by the Leader Agent to extract the 5-dimensional risk tuple. This structure forces the model to ground its suggestions in specific evidence.

Panel of Figure 17 — Figure 17: The LVA Prompt. By enforcing a structured JSON output containing severity weights and gradient feedback, the Auditor transforms qualitative legal analysis into quantitative signals for the next optimization round.

Panel of Figure 18 — Figure 18: The composite prompt mechanism for the CRA. It translates the abstract Stackelberg theoretical constraints into executable instructions, combining global strategy (Leader), resource budget (Fusion), and specific risk gradients (Auditor).

Panel of Figure 19 — Figure 19: The Force Rewrite injection mechanism. This acts as a perturbation vector to push the CRA out of local optima (i.e., lazy revisions) — triggered when the system detects zero effective edits despite unresolved risks.

Panel of Figure 20 — Figure 20: The definition of the Q-Score metrics. Each dimension is graded on a 3-point scale to compute the final risk weight.

Multi-Agent LLMs

Multi-agent large language models coordinate multiple AI agents to solve tasks, share information, and make collective decisions.

See all 317 papers

Roles with Rails: Contract-Preserving Role Evolution in Multi-Agent Structured Reasoning
May 27, 2026Ling-Yue Ge, Lan-Zhe GuoMulti-Agent LLM Systems LLM Agent Orchestration
1National Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China · School of Intelligence Science and Technology, Nanjing University, Suzhou, China
ContractScrub: A benchmark for final review of legal contracts
Aug 20, 2026Yejin Bang, Kirsty Fielding, Brandan Oliver +3Legal Document Analysis Legal NLP
Thomson Reuters Foundational Research, London, UK · Imperial College London, UK
Adaptive Collaboration of Arena-Based Argumentative LLMs for Explainable and Contestable Legal Reasoning
Feb 21, 2026Hoang-Loc Cao, Phuc Ho, Truong Thanh Hung Nguyen +3Legal Reasoning Computational Argumentation
Ho Chi Minh University of Science, Vietnam · University of New Brunswick, Canada · Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany

May 27, 2026J/K move · Enter open · S save

Roles with Rails: Contract-Preserving Role Evolution in Multi-Agent Structured Reasoning

Ling-Yue Ge, Lan-Zhe Guo

1National Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China · School of Intelligence Science and Technology, Nanjing University, Suzhou, China

Multi-Agent LLM Systems LLM Agent Orchestration Agentic Reasoning Multi-Agent Systems Multi-Agent Reasoning

Open paper AskPDF arXiv

Role-based LLM multi-agent systems need adaptive role pools, yet adapting such systems is not merely a matter of prompt optimization: roles often carry structural obligations, including capability coverage, message compatibility, validation, final-answer aggregation, and parser-compatible output protocols. Existing systems either fix the role inventory and lose adaptivity, or allow unconstrained generation to induce role drift, removing structurally necessary roles and breaking answer contracts. We formulate this as contract-preserving role evolution, requiring every committed edit to preserve five structural contracts (capability, communication, validation, aggregation, output protocol). We instantiate this formulation in SERO, a Self-Evolving Role Orchestration framework that evolves a typed role-card pool through credit-guided retrieval, a credit-ranked communication DAG with a protected terminal aggregator and conditional validator repair, and a contextual-bandit controller whose LLM-proposed edits are committed only when they preserve the contracts and improve task score. Experiments on real-world reasoning benchmarks across three LLM backbones confirm the value of contract-preserving role evolution.

Abstract

Multi-Agent LLMs

Explore similar work

Roles with Rails: Contract-Preserving Role Evolution in Multi-Agent Structured Reasoning

ContractScrub: A benchmark for final review of legal contracts

Adaptive Collaboration of Arena-Based Argumentative LLMs for Explainable and Contestable Legal Reasoning