CABAL: Multi-Agent Simulacra for Tracing the Effects of Collusive Bidding in Peer Review

Jicheng Zhou; Kemou Li; Kahim Wong; Zheyuan Li; Zhuan Shi; Fengpeng Li; Haiwei Wu; Jiantao Zhou

cs.AISep 4, 2026

CABAL: Multi-Agent Simulacra for Tracing the Effects of Collusive Bidding in Peer Review

Authors: Jicheng Zhou, Kemou Li, Kahim Wong, Zheyuan Li, Zhuan Shi, Fengpeng Li, Haiwei Wu, Jiantao Zhou

Organizations: State Key Laboratory of Internet of Things for Smart City, University of Macau · Mila – Québec AI Institute · McGill University · PRADA Lab, King Abdullah University of Science and Technology ·

Abstract

Recent reports during the AAAI-27 review cycle highlight the risk of reviewers coordinating bids for reciprocal assignment advantage. Prior work treats bidding, reviewer assignment, and review manipulation as separate stages, leaving the lifecycle effects of collusive bidding unclear. Real-world analysis is further constrained by typically unobservable collusive intent and the lack of counterfactuals for the same conference. Motivated by this gap, we introduce \alg, an end-to-end multi-agent simulacra framework for studying reviewer assignment integrity by holding the conference environment fixed and configuring LLM-driven reviewer agents with honest or collusive policies. We further develop an affinity-guided collusive bidding strategy that uses mutual reviewer-paper affinities to construct collusion rings and select target papers, producing expertise-consistent rather than arbitrarily targeted attacks. Controlled experiments show that collusive bidding more than doubles target-paper capture and that assigned colluders score target papers about two points higher than honest co-reviewers, while conference-wide effects remain comparatively modest. Evaluated bid-phase detectors provide only limited evidence of collusion: in a fixed-triplet detector stress test, native positive-bid graphs are confounded by benign affinity, while a Very-High-only diagnostic view enables precise but low-coverage local recovery.

Figures & tables

Figure 1: Overview of the lifecycle effects studied in CABAL . With the conference environment (papers, reviewer profiles, COI constraints, and assignment mechanism) held fixed, honest reviewer behavior is expertise-aligned, whereas collusive reviewer behavior involves coordinated, affinity-guided bids to capture target-paper assignments and distort downstream evaluations.

Work	Venue	Expertise/ Affinity	Strategic Behavior	Bidding	Assignment	Reviewing	Outcome Analysis	Behavioral Counterfactual
Reviewer Assignment and Bidding Mechanisms
Anjum et al. [ 18 ]	EMNLP-IJCNLP’19	$\bullet$	-	-	$\circ$	-	-	-
Stelmakh et al. [ 7 ]	JMLR’21	$\circ$	-	-	$\bullet$	-	$\circ$	-
Rozencweig et al. [ 9 ]	AAMAS’23	$\circ$	-	$\bullet$	$\circ$	-	-	$\circ$
Strategic Manipulation and Adversarial Behaviors in Peer Review
Wu et al. [ 11 ]	ICML’21	$\circ$	$\bullet$	$\bullet$	$\bullet$	-	-	$\circ$

Table 1: Comparison with representative prior work across the peer-review lifecycle. CABAL jointly connects expertise-grounded strategic behavior, bidding, assignment, downstream reviewing, and outcome analysis within a controlled behavioral counterfactual.

Figure 2: Overview of CABAL . The workflow comprises five stages: (1) constructing a fixed conference substrate; (2) forming expertise-grounded collusion rings and target papers; (3) instantiating honest or collusive reviewer simulacra; (4) executing bidding, assignment, and reviewing; and (5) evaluating outcomes across matched behavioral worlds.

Measurement	Honest	$r=0.2$	$r=0.5$
Score inflation across all reviews
Average review score	$4.34\pm 0.02$	$4.47\pm 0.06$	$4.72\pm 0.09$
Reviews above threshold ( $s>5.5$ )	$28.2\pm 0.4\%$	$30.6\pm 1.7\%$	$34.2\pm 2.1\%$
Alignment with reference quality
Pearson correlation	$0.804\pm 0.010$	$0.775\pm 0.019$	$0.775\pm 0.025$
Spearman rank correlation	$0.813\pm 0.014$	$0.776\pm 0.021$	$0.765\pm 0.035$

Table 2: Conference-wide score inflation and quality alignment. Alignment metrics compare three-reviewer paper means with the fixed reference-quality assessments.

Input view	Detector	$r=0.2$ ( $\|\mathcal{C}\|=18$ )	$r=0.5$ ( $\|\mathcal{C}\|=47$ )
Reviewer ranking : $\Delta r$
Ternary bid matrix	Counting	$+0.054$	$+0.025$
Ternary bid matrix	Pairwise	$-0.005$	$-0.014$
Ternary bid matrix	Low-rank	$-0.032$	$-0.024$
Set recovery : recovered / flagged reviewers
Bid-author, $\tau=1$	Greedy densest	$16/89$	$42/77$

Table 3: Representative bid-phase detector results.

\Delta r

is the collusive-minus-honest change in normalized rank (negative is more suspicious); set entries report recovered colluders / flagged reviewers. Complete results are provided in Appx. D .

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Prompt component	Content supplied to the agent
Shared prompt scaffold
Reviewer context	Conference context and score calibration; the shared scoring rubric; the reviewer’s research areas; and three representative publication titles.
Quality grader	“You assess the quality of all submitted papers together, comparatively, based on their title and abstract. You must rank them and assign each a score that reflects the conference’s rigorous standards and acceptance rate.”
Behavior-specific policy
Honest reviewer	“Evaluate papers based solely on their technical quality, originality, clarity, and significance … calibrate your scores honestly.”
Collusive bidding	After receiving its ring peers and target-paper IDs, the agent is instructed: “You MUST bid ‘Very High’ on every paper in your ring’s target list. For all other papers, bid according to your genuine research expertise.”

Appendix

Table 4: Prompt scaffold and selected behavior-defining excerpts. Runtime fields such as research areas, ring peers, and target-paper IDs are instantiated separately for each agent. Repeated paper formatting and JSON output instructions are omitted for clarity.

Run	$r=0.2$	$r=0.5$
$0042$ -1	$14/4$	$22/25$
$0042$ -2	$13/6$	$24/23$
$0042$ -3	$14/4$	$18/28$
$0921$	$12/7$	$34/13$
$1125$	$11/7$	$25/23$
$2024$	$7/11$	$21/25$

Appendix

Table 5: Realized ring-level reviewing strategies. Each entry reports the number of aggressive/subtle colluders. All members of the same ring share its assigned strategy.

Panel of Figure 5 — Figure 5: Data provenance and artifact organization in CABAL . Public researcher metadata is collected into a frozen profile snapshot that grounds both reviewer expertise and synthetic submission generation. The conference substrate is held fixed across behavioral worlds, while personas, bids, assignments, reviews, and evaluation metrics are generated separately for each world. Orange denotes the restricted profile snapshot; blue denotes synthetic or derived artifacts that can be released after consistent identifier remapping and privacy review.

Scenario	All supporters	Honest co-reviewers	Within-paper gap	Aggressive supporters	Subtle supporters
$r=0.2$	$7.11$	$4.90$	$+2.21$	$7.45$	$6.50$
$r=0.5$	$6.83$	$4.83$	$+1.99$	$7.44$	$6.27$

Appendix

Table 6: Reviewer-level evidence of target-paper score inflation. Values summarize the scores assigned to captured target papers. The within-paper gap compares supporting ring members with honest co-reviewers evaluating the same papers. Aggressive and subtle columns report the corresponding strategy-specific supporter scores.

Detector	Input / output	Assumed malicious signature	Diagnostic question for CABAL
Reviewer-level ranking: individual bidding anomalies
Counting simple_detect	Ternary reviewer-paper bids; reviewer ranking	Many negative bids, with positive bids concentrated on a small target set	Does CABAL resemble a rejection-style target-seeking attack?
Pairwise reciprocity cluster_detect	Ternary bids and authorship; reviewer ranking	A reviewer pair bids positively on each other’s papers while showing limited interest elsewhere	Does collusion add reciprocal interest beyond that induced by benign mutual affinity?
Low-rank residual low_rank_detect	Ternary reviewer-paper matrix; reviewer ranking	A malicious reviewer’s bid vector deviates from the matrix’s low-rank preference structure	Does target-bid inflation create a reviewer-level matrix anomaly?
Bid-author graph: anomalously dense reviewer groups
OQC-Greedy	Directed bid-author graph; one reviewer set	A ring has more internal bid-author edges than expected under a fixed background density	Does CABAL form a quasi-clique recoverable along a degree-peeling sequence?

Appendix

Table 7: Detector families and prior assumptions. Each detector operationalizes a different malicious-bidding signature. The final column states the diagnostic question evaluated on CABAL ; it is not an additional input to the detector.

Detector	$r=0.2$ : Honest $\rightarrow$ Collusive	$r=0.5$ : Honest $\rightarrow$ Collusive	Top- $K$ hit
Reviewer-level anomaly ranking
Counting	$.551\rightarrow.605\;(+.054)$	$.579\rightarrow.604\;(+.025)$	$.11/.11$
Pairwise reciprocity	$.292\rightarrow.287\;(-.005)$	$.346\rightarrow.332\;(-.014)$	$.11/.49$
Low-rank residual	$.463\rightarrow.431\;(-.032)$	$.484\rightarrow.460\;(-.024)$	$.17/.40$

Appendix

Table 8: Reviewer-level ranking results. For each collusion rate, entries report the designated reviewers’ mean normalized rank as honest

\rightarrow

collusive, with

\Delta r

in parentheses. Lower ranks indicate greater suspiciousness, and

0.5

is the random-ranking reference. Top-

K

hit

(r=.2/.5)

hit is evaluated in the collusive world with

K=|\mathcal{C}|

	$r=0.2$ ( $\|\mathcal{C}\|=18$ )			$r=0.5$ ( $\|\mathcal{C}\|=47$ )
Detector	Rec./flag.	Prec./rec.	H $\rightarrow$ C overlap	Rec./flag.	Prec./rec.	H $\rightarrow$ C overlap
Native positive-bid graph: $\tau=1$ (High or Very High)
OQC-Greedy	$2.0/19.0$	$.105/.111$	$3.0\rightarrow 2.0$	$12.0/20.0$	$.600/.255$	$14.0\rightarrow 12.0$
OQC-Local	$2.0/18.7$	$.109/.113$	$2.0\rightarrow 2.0$	$12.2/20.0$	$.610/.260$	$12.3\rightarrow 12.2$
TellTail	$2.0/15.0$	$.133/.111$	$2.0\rightarrow 2.0$	$12.3/16.7$	$.741/.262$	$12.0\rightarrow 12.3$
Greedy Densest Subgraph	$16.0/89.0$	$.180/.889$	$17.0\rightarrow 16.0$	$42.0/77.0$	$.545/.894$	$43.0\rightarrow 42.0$

Appendix

Table 9: Bid-author graph recovery. “Rec./flag.” reports mean recovered colluders

|S\cap\mathcal{C}|

followed by mean flagged reviewers

|S|

. “Prec./rec.” reports precision and global recall. Honest

\rightarrow

collusive gives the detector’s overlap with the designated reviewers in the two worlds. Stochastic outputs are averaged over 30 detector invocations on the fixed graph.

Input view	$r=0.2$ ( $\|\mathcal{C}\|=18$ )			$r=0.5$ ( $\|\mathcal{C}\|=47$ )
	Rec./flag.	Prec./rec.	H $\rightarrow$ C overlap	Rec./flag.	Prec./rec.	H $\rightarrow$ C overlap
Native positive-bid view
$\tau=1$ (Positive-bids)	$15.0/97.0$	$.155/.833$	$13.0\rightarrow 15.0$	$41.0/101.0$	$.406/.872$	$36.0\rightarrow 41.0$
Very-High-only diagnostic view
$\tau=2$ (Very High only)	$6.0/36.0$	$.167/.333$	$2.0\rightarrow 6.0$	$9.0/21.0$	$.429/.191$	$4.0\rightarrow 9.0$

Appendix

Table 10: Fraudar reviewer-paper block recovery. Fraudar returns a reviewer-paper block; the table evaluates its reviewer side. Notation follows Tab. 9 .

Scholarly Peer Review

Scholarly peer review studies how experts evaluate research, make publication decisions, and improve the fairness, reliability, and transparency of those processes.

See all 78 papers

Collusion with Competitive Marginals: Price-Level Audits Are Blind by Construction
Jul 29, 2026Xin Xu, Chengrui Wu, Jiayu Lu +3Algorithmic Auditing Automated Bidding
Carnegie Mellon University · Zhejiang University
Colosseum: Auditing Collusion in Cooperative Multi-Agent Systems
Feb 16, 2026Mason Nakamura, Abhinav Kumar, Saswat Das +5Multi-Agent LLM Systems Multi-Agent Systems
University of Massachusetts Amherst · University of Virginia · ELLIS Institute Tübingen +1
PAC-BENCH: Evaluating Multi-Agent Collaboration under Privacy Constraints
Apr 13, 2026Minjun Park, Donghyun Kim, Hyeonjong Ju +5Multi-Agent Collaboration Multi-Agent Coordination
Department of Artificial Intelligence, Yonsei University

Jul 29, 2026J/K move · Enter open · S save

Collusion with Competitive Marginals: Price-Level Audits Are Blind by Construction

Xin Xu, Chengrui Wu, Jiayu Lu, Kaizhen Tan+2

Carnegie Mellon University · Zhejiang University

Algorithmic Auditing Automated Bidding Algorithmic Collusion

Open paper AskPDF arXiv

Empirical work on algorithmic collusion asks one question of the data: are prices supracompetitive? We show this can be answered "no" by a conspiracy that is nonetheless profitable. Consider bidding agents that couple only through the joint distribution of their unexplained bid components, leaving every agent's own bid law exactly at the competitive law. Any test whose input is a single agent's price or bid history then has power exactly equal to its false-positive rate, for every coupling strength up to comonotonicity. The published detection methodology is therefore blind to this conduct by construction rather than underpowered, and no sample size repairs it. Three empirical results follow. First, the mechanism appears in real language-model agents: twenty models from nineteen independent developers, three deployment prompts each, show residual correlation of

+0.053

between two deployments of one model against

+0.0001

across models, with a 95% interval clustered by developer of

[0.030, 0.078]

, under an auditor that sees every order feature and is fitted out of sample. Second, the coupling falls monotonically as sampling temperature rises (

p=0.002

), turning a deployment parameter into a candidate mitigation. Third, on 24 days of Ethereum block-building auction data covering 77,684 bids from 39 bidders, the honest population of bidder pairs is itself so dependent that a screen held at a 5% false-positive rate must sit above a floor of

+0.50

+0.81

, which is 20 to 32 times the family-wise sampling threshold and does not fall as the audit window grows. Since lawful multi-identity operation and conspiracy are behaviourally indistinguishable here, the tractable regulatory target is not detection but counting: resolving 40 bidding identities into 23 operators raises the Herfindahl index by 247.5%, and adding behavioural clusters from public bid streams reaches 324.5%.

Abstract

Scholarly Peer Review

Explore similar work

Collusion with Competitive Marginals: Price-Level Audits Are Blind by Construction

Colosseum: Auditing Collusion in Cooperative Multi-Agent Systems

PAC-BENCH: Evaluating Multi-Agent Collaboration under Privacy Constraints