BE-CTDE: Blockchain-Enabled Multi-Agent Reinforcement Learning with Anomaly Detection and Token Economics

Chih-Wei Huang; Hsin-Chieh Lee; Yun-Chen Yu; Ibrahim Althamary; Rongxing Lu; Phone Lin

cs.MAOct 25, 2026

BE-CTDE: Blockchain-Enabled Multi-Agent Reinforcement Learning with Anomaly Detection and Token Economics

Authors: Chih-Wei Huang, Hsin-Chieh Lee, Yun-Chen Yu, Ibrahim Althamary, Rongxing Lu, Phone Lin

Organizations: Department of Communication Engineering, National Central University, Taoyuan City 320317, Taiwan · Interdisciplinary Research Center for Intelligent Secure Systems, King Fahd University of Petroleum and Minerals (KFUPM), Dhahran 31261, Saudi Arabia · School of Computing, Queen’s University, Kingston, ON K7L 2N8, Canada · Department of Computer Science and Information Engineering, the Graduate Institute of Networking and Multimedia, the Graduate Institute of Biomedical Electronics and Bioinformatics, the Internet of Things Research Center, and the Graduate Institute of Medical Device and Imaging, National Taiwan University, Taipei City 10617, Taiwan

Abstract

Multi-agent reinforcement learning supports cooperative decision-making in Internet of Things systems, but strategic deviations can disrupt coordination and undermine confidence in the training process. This paper proposes Blockchain-Enabled Centralized Training and Decentralized Execution (BE-CTDE), which uses blockchain as an active training-management layer. A behavioral verifier compares executed trajectories with shared-policy recommendations; its flags drive a token mechanism combining slashing, burning, rewards, and compensation. Retained balances determine each agent's contribution to the global policy update, connecting auditable behavior to learning influence. We derive a one-round incentive condition that accounts for action-dependent compensation and the payoff from deviation, and a finite expected exclusion-time bound under explicit conditional detection and accounting assumptions, showing perfect detection is unnecessary. An exact finite-population capacity bound characterizes the limits of the population-relative detector. In multi-agent path finding experiments, BE-CTDE outperforms a state-of-the-art learning-based path planner with substantially higher success rates under attack, even when adversaries form the majority. It also outperforms partially defended baselines in all configurations, with collision rates up to 35% lower than under detection without economic consequences and up to 43% lower than under conventional stake slashing. Together, these results characterize how behavioral verification and token feedback can support accountable cooperative learning.

Figures & tables

References	Blockchain	MARL	On-chain Parameters	On-chain Learning	Incentive Mechanism	Token-based Economy	Data Verification	Malicious Agent Detection
[ 16 , 17 ]	Simulator	–	✓	✓	✓	✓	✓	–
[ 20 ]	Simulator	–	✓	–	✓	✓	✓	✓
[ 18 ]	Real	–	✓	–	✓	✓	✓	✓
[ 21 ]	Simulator	✓	✓	–	✓	–	✓	–
[ 19 ]	Real	–	✓	✓	✓	✓	–	–
[ 22 ]	Simulator	✓	✓	–	✓	–	✓	✓

TABLE I: Functional comparison of AI and blockchain integration across recent literature

Fig. 1: BE-CTDE system architecture: IoT agents interact with the environment through observations $o_{t}$ , rewards $r_{t}$ and actions $a_{t}$ , while the blockchain layer coordinates training, verifies behavior and manages token incentives. Neural-network computation remains off-chain.

Symbol	Definition
System and learning
$N,n_{t},M,M_{t}$	Initial, active, initially malicious, and active malicious counts ( $M<N$ )
$\mathcal{K}_{t},\mathcal{H},\mathcal{M},f_{i}(t)$	Retained, honest, malicious sets and detection flag
$\mathcal{S},\mathcal{A},\mathcal{O}$	State, joint-action, and joint-observation spaces
$\Omega,\mathcal{P},\mathcal{R},\gamma$	Observation map, transition/reward functions, and discount factor
$(o,a,r,m)_{i,t}$	Observation, action, reward, and prior message of agent $i$

TABLE II: Principal Notation

Fig. 2: BE-CTDE training workflow and shared-policy architecture. Neural inference, communication, advantage estimation and PPO run off-chain (orange), while behavioral verification and token settlement run on-chain (blue); document icons mark results committed to the chain.

$N$	$F_{\max}(N)$	$M/N=3/8$	$M/N=5/8$
8	2	0.6667	0.4000
32	9	0.7500	0.4500
64	19	0.7917	0.4750
128	39	0.8125	0.4875

TABLE III: Initial detector capacity at

k=1.5

. The last columns bound the realized fraction of adversaries flagged in one round at the stated initial ratios; they are not conditional detection guarantees.

Parameter	Value
Training environment
Grid; number of agents	$40\times 40$ ; 8
Start and goal assignment	Random unique positions
Training adversaries	3 or 5 of 8 agents
Obstacle density	Triangular $(0,0.33,0.50)$
Episodes; maximum steps	1,838; 256 per episode

TABLE IV: Simulation Parameters

Panel of Figure 7 — Fig. 3: Policy-loss and goal-arrival curves with three malicious agents during training. The horizontal axis is the recorded training-step counter; the curves measure learning progress, not exclusion events.

		37.5% malicious agents			62.5% malicious agents
$N$	Method	$0\%$	$15\%$	$30\%$	$0\%$	$15\%$	$30\%$
8	On-Chain CTDE	2.45	3.09	3.67	49.67	58.50	69.72
	BE-CTDE $\lambda{=}0$	9.52	18.14	29.63	13.51	29.16	37.08
	BE-CTDE $\lambda{=}0.5$	7.81	16.35	25.82	13.06	26.22	37.42
	BE-CTDE $\lambda{=}1$	7.42	15.30	20.79	11.96	24.84	35.20
32	On-Chain CTDE	10.41	15.83	18.69	74.04	79.71	82.53

TABLE VI: Test-time collision rate (CO, %) with adversaries during training only and honest agents during testing. Columns under each training ratio correspond to obstacle densities

0\%

15\%

, and

30\%

\lambda=0

is the burn-only BE-CTDE variant.

Fig. 4: SR comparison with adversaries during training and honest testing. Rows show obstacle densities 0%, 15% and 30%; columns show training-time malicious ratios 37.5% and 62.5%.

Multi-Agent Reinforcement Learning

Multi-agent reinforcement learning studies how interacting agents learn coordinated or competitive policies through shared experience and rewards.

See all 373 papers

BARD-MARL: Byzantine-Agent Detection for Learned Communication in Multi-Agent Reinforcement Learning
Jun 15, 2026Almond Kiruthu MurimiMulti-Agent Systems Multi-Agent System Security
Carnegie Mellon University Department of Electrical and Computer Engineering Pittsburgh, Pennsylvania, United States
TANDEM: Balancing Centralized and Decentralized Critics to Address the Information Gap in Complex Multi-Agent Environments
Oct 27, 2026Alireza Saleh Abadi, Leen-Kiat Soh, Adam Eck +1Cooperative MARL Centralized Training with Decentralized Execution
School of Computing, University of Nebraska-Lincoln, Lincoln, Nebraska, USA · Department of Computer Science, Oberlin College, Oberlin, Ohio, USA · Institute for AI & School of Computing, University of Georgia, Athens, Georgia, USA
DART: A DAG-Based Reputation and Incentive Framework via Blockchain-Enabled Governance for Trustworthy LLM Multi-Agent Collaboration
Sep 1, 2026Manoj Kumala, Xinyun Liua, Ronghua XuMulti-Agent Systems Multi-Agent Collaboration
Department of Applied Computing, Michigan Technological University, Houghton, MI 49931, USA

Jun 15, 2026J/K move · Enter open · S save

BARD-MARL: Byzantine-Agent Detection for Learned Communication in Multi-Agent Reinforcement Learning

Almond Kiruthu Murimi

Carnegie Mellon University Department of Electrical and Computer Engineering Pittsburgh, Pennsylvania, United States

Multi-Agent Systems Multi-Agent System Security Multi-Agent Communication Cooperative MARL

Open paper AskPDF arXiv

Learned communication improves coordination in cooperative multi-agent reinforcement learning, but it also creates a trust problem: a trained policy may route information through agents that have become faulty or adversarial. This paper studies Byzantine-agent detection for learned-communication MARL in adaptive traffic signal control. We propose BARD-MARL, a post-hoc diagnostic layer on top of BayesG, which is used as an attributed communication substrate rather than as a contribution of this paper. BARD-MARL combines two agent-level evidence streams: policy-graph features extracted from state-action trajectories and Bayesian trust statistics computed from BayesG latent mask probabilities. Across fixed-action, observation-flip, random-noise, and coordinated attacks in SUMO traffic grids, the results show that these signals are complementary rather than universally dominant. On a 25-agent grid, BARD-MARL reaches 0.843 AUC-ROC under a 10% observation-flip attack, while policy-graph-only detection reaches 0.917 AUC-ROC under a 10% coordinated attack. On a 100-agent grid, the unified BARD-MARL variant reaches 0.982 AUC-ROC for both 10% fixed-action and 10% coordinated attacks. The study shows that learned communication policies expose useful diagnostic evidence, but credible resilience claims require attack-specific ablations and explicit separation between coordination, detection, and mitigation.

Abstract

Multi-Agent Reinforcement Learning

Explore similar work

BARD-MARL: Byzantine-Agent Detection for Learned Communication in Multi-Agent Reinforcement Learning

TANDEM: Balancing Centralized and Decentralized Critics to Address the Information Gap in Complex Multi-Agent Environments

DART: A DAG-Based Reputation and Incentive Framework via Blockchain-Enabled Governance for Trustworthy LLM Multi-Agent Collaboration