DICE: Decentralized Incentive-driven Federated Learning for Efficient & Scalable Customization of Pretrained Models in Web3

Hong Lin; Zhongle Xie; Lidan Shou; Ke Chen; Gang Chen

cs.CROct 29, 2026

DICE: Decentralized Incentive-driven Federated Learning for Efficient & Scalable Customization of Pretrained Models in Web3

Authors: Hong Lin, Zhongle Xie, Lidan Shou, Ke Chen, Gang Chen

Organizations: The State Key Laboratory of Blockchain and Data Security, Zhejiang University Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain&Data Security

Abstract

Customizing Pretrained Models (PMs) for downstream tasks demands task-specific data, which burgeoning Web3 applications provide abundantly. Blockchain-powered federated learning (BCFL) offers a secure and privacy-preserving paradigm to customize PMs among self-interested and untrusted Web3 participants. Yet, existing BCFL algorithms face severe efficiency and scalability challenges when applied to large PMs, due to their substantial computational and memory requirements and the hardware constraints of many participants. This paper introduces DICE, a novel BCFL algorithm that addresses the efficiency and scalability challenges of customizing large PMs in Web3 via two sets of synergistic techniques: (i) a dynamic progressive learning scheme and a model slice verification scheme to enable a wider range of honest participants, including those unable to deploy the full model, to engage in customizing PMs, while preserving model utility; (ii) a tree-structured committee consensus protocol and an efficiency-driven incentive scheme to address the dominant bottleneck that hinders existing BCFL algorithms from scaling to a larger number of data holders for customizing data-hungry large PMs, while maintaining security. Theoretical analysis and extensive experimental results on customizing vision and language PMs demonstrate DICE's efficiency, effectiveness, and security.

Figures & tables

Figure 1. A general BCFL paradigm for customizing BERT-Large (with $L_{G}=24$ H-layers, 340 million params), required by a medical institution for disease diagnosis, using health data in MedSynapse.

Batch Size	8	16	32	64	# Workers	10	40	70	100
$MC_{w}$ (GB)	11	17	30	59	$MC_{v}$ (GB)	18	71	125	170

Table 1. Per-round memory costs of a worker (

MC_{w}

) and a validator (

MC_{v}

) in Fig. 1 under typical algorithms ( Zhang et al., 2021 ; Lin et al., 2024 ) .

Symbol	Definition
$r$ , $R$ , $\eta$	global round index; total number of rounds; learning rate
$w$ , $v$	worker’s and validator’s addresses on the blockchain network $\spadesuit$
$\mathbb{W}_{all}$ , $\mathbb{V}_{all}$	the complete sets of eligible workers and eligible validators
$\mathbb{AW}^{r}$	workers that have submitted updates in round $r$
$\mathbb{W}^{r}$	scheduled workers; replaced by $\mathbb{AW}^{r}$ when training completes
$\mathbb{V}^{r}$	validators that submit VA results in round $r$

Table 2. Notation Summary.

Figure 3. Emulator generation given slice

\mathbb{M}^{r}_{w}

\mathbb{LUS}_{j}

Figure 5. Effects of varying

N_{t}

, given

\epsilon_{p}=10^{-12}

N_{lus}=1

K_{\text{check}}=1

|\mathbb{V}_{all}|=\max(100,3\lceil\log_{[1-\frac{N_{t}}{(2|\mathbb{W}^{r}|-1)N_{lus}}]}\frac{\epsilon_{p}}{K_{\text{check}}}\rceil)

, and

N_{c}=\left\lceil\frac{|\mathbb{V}_{all}|}{2}+\log_{1-\frac{N_{t}}{(2|\mathbb{W}^{r}|-1)N_{lus}}}\left(\frac{\epsilon_{p}}{K_{\text{check}}}\right)\right\rceil

Settings	Vision Tasks	Language Tasks
Type	Image Classification	NER and RE
Model	ViT-base (default), DINOv2-large	BERT-base (default), Qwen2.5-0.5B
Dataset	FedISIC ( , 2022 ) ‡ , CIFAR100 ( Sharma et al., 2018 ) , DermaMNIST ( Yang et al., 2021 ; Yang et al., 2023b )	2018n2c2 ( Lee et al., 2020 ) ‡ , BC5-disease (BC5-d) ( Lee et al., 2020 ) , JNLPBA ( Kim et al., 2004 ) , GAD ( Lee et al., 2020 ) , DDI ( Herrero-Zazo et al., 2013 ) , ChemProt ( Taboureau et al., 2010 )
Data Partition	FedISIC: $\operatorname{Dirichlet}(1.25)$ ; all others: $\operatorname{Dirichlet}(2.5)$
Further Pretraining	MedMNIST (supervised image classification)	PubMed (self-supervised language modeling)
Workers / Validators ( $N_{c}$ )	100 / 1,173 (1,036)	50 / 678 (599)

Table 3. Task settings. Tasks marked with

{\ddagger}

undergo further pretraining before fine-tuning; the others directly fine-tune the PMs.

$L_{w}/L_{v}$	$S_{LM}/S_{VM}$	$MC_{w}$	Hardware Ref.	$MC_{v}$	Hardware Ref.
24	1.8	16.4	RTX 4090 24GB $\uparrow$	122.2	2 $\times$ H100 80GB $\uparrow$
20	1.6	13.8	RX 9070 XT 16GB $\uparrow$	108.3	2 $\times$ H100 80GB $\uparrow$
16	1.4	11.2	RTX 5070 12GB $\uparrow$	94.5	RTX Pro 6000 96GB $\uparrow$
12	1.2	8.6	ARC B570 10GB $\uparrow$	80.6	2 $\times$ RTX A6000 48GB $\uparrow$

Table 4. Worker and validator memory overhead (

MC_{w}

MC_{v}

in GB) for model training and multi-update (

|\mathbb{W}^{r}|

-update) verification when customizing Qwen2.5-0.5B.

S_{LM}/S_{VM}

denotes local/validation model size in GB. Measurement environment follows Table 1 . Hardware Ref. denotes the reference of required hardware.

Methods	$\mathbb{W}_{all}$	$S_{LM}\downarrow$	$\|\theta_{w}^{r}\|\downarrow$	ACC $\uparrow$			$S_{LM}\downarrow$	$\|\theta_{w}^{r}\|\downarrow$	F1 $\uparrow$			$S_{LM}\downarrow$	$\|\theta_{w}^{r}\|\downarrow$	F1 $\uparrow$
Methods	$\mathbb{W}_{all}$	Vision Tasks		FedISIC	CIFAR100	DermaMNIST	NER Tasks		n2c2	JNLPBA	BC5-d	RE Tasks		DDI	GAD	ChemProt
BCFL	✗	100.0%	100.0%	79.5%	94.0%	82.9%	100.0%	100.0%	90.7%	98.5%	89.5%	100.0%	100.0%	44.3%	92.7%	13.8%
+LoRA		101.6%	1.7%	75.7%	91.5%	78.3%	101.2%	1.6%	86.4%	93.5%	85.2%	101.2%	2.2%	43.8%	87.4%	13.1%
+BitFit		100.0%	0.2%	71.5%	94.8%	75.2%	100.0%	0.1%	81.6%	59.3%	84.5%	100.0%	0.8%	42.2%	76.3%	23.0%
+Prune	✓	50.4%	50.0%	95.9%	90.9%	99.6%	60.9%	50.0%	97.4%	98.2%	98.5%	61.1%	50.4%	93.3%	99.6%	78.8%
+KD	✓	3.3%	3.1%	91.7%	80.3%	91.2%	60.9%	50.0%	97.1%	99.0%	99.7%	61.1%	50.4%	92.8%	99.9%	97.6%

Table 5. Results of handling constrained workers (upper half) and constrained validators (lower half). Top-1 and top-2 results are marked in bold and underlined, respectively.

Panel of Figure 6 — Figure 6. Performance of customized model versus cumulative model-processing workload under different baselines. X-axis: cumulative model-processing workload, measured by accumulated $S_{LM}+N_{VM}S_{VM}$ .

Methods	$S_{LM}\downarrow$	$\|\theta_{w}^{r}\|\downarrow$	$S_{VM}\downarrow$	$N_{VM}\downarrow$	ACC/F1 $\uparrow$
Methods	$S_{LM}\downarrow$	$\|\theta_{w}^{r}\|\downarrow$	$S_{VM}\downarrow$	$N_{VM}\downarrow$	FedISIC	CIFAR100	DermaMNIST
DICE	50.4%	41.6%	50.4%	3.5%	98.3%	97.9%	99.1%
+LoRA	51.0%	0.6%	50.4%		94.0%	94.3%	94.6%
+BitFit	50.4%	0.1%	50.4%		89.1%	89.2%	89.7%
+Prune	50.4%	50.0%	50.4%		96.1%	90.9%	98.6%
+KD	3.3%	3.1%	3.3%		91.9%	80.1%	91.2%

Table 6. Results of DICE integrated with existing efficient customization techniques on different tasks.

Panel of Figure 8 — Figure 8. Performance of DINOv2 and Qwen2.5 customized by different baselines. X-axis follows Fig. 7 .

Figure 9. Per-participant workload (

S_{LM}/S_{VM}

N_{VM}

) and security performance (

RMW+RMV

) under different baselines when customizing DINOv2 and Qwen2.5.

Methods	FedISIC (ACC) $\uparrow$	n2c2 (F1) $\uparrow$	DDI (F1) $\uparrow$
T-Only	73.8%	97.2%	81.6%
B-Only	95.9%	96.1%	86.9%
OT	87.1%	96.9%	76.2%
DyTB	91.7%	97.2%	93.1%
DyBT	96.0%	96.1%	86.9%
DyOT	91.3%	96.9%	88.5%

Table 7. Effect of DPL’s importance-score-based slice selection on model performance.

$L_{E}$	FedISIC (ACC) $\uparrow$	n2c2 (F1) $\uparrow$	DDI (F1) $\uparrow$
0	55.3%	67.1%	63.8%
1 (Default)	98.3%	97.9%	96.1%
2	98.4%	97.1%	96.3%

Table 8. Effect of emulators (consisting of

L_{E}

H-layers) on model performance.

Methods	ACC/F1 $\uparrow$			$RMW\downarrow$
Methods	FedISIC	n2c2	DDI	FedISIC	n2c2	DDI
Full-Verify	98.5%	98.5%	96.6%	0.0%	0.0%	0.0%
MSV	98.3%	97.9%	96.1%	0.0%	0.0%	0.0%

Table 9. Effect of MSV’s compact-model verification on model performance and security against poisoning attacks from malicious workers.

Methods	ACC/F1 $\uparrow$			$RMW\downarrow$
Methods	FedISIC	n2c2	DDI	FedISIC	n2c2	DDI
Fixed	92.7%	95.4%	92.5%	11.2%	8.7%	9.6%
Dynamic	98.3%	97.9%	96.1%	0.0%	0.0%	0.0%

Table 10. Effect of dynamic validation threshold on model performance and security against poisoning attacks from malicious workers.

Methods	All Honest $\overline{N}_{v,t}/N_{VA}\downarrow$			20% Malicious $\overline{N}_{v,t}/N_{VA}\downarrow$
Methods	FedISIC	n2c2	DDI	FedISIC	n2c2	DDI
TCC-only	6.3%	10.7%	10.5%	90.5%	88.9%	89.9%
TCC+EDI	6.1%	10.5%	10.6%	6.1%	10.6%	10.5%

Table 11. Effect of EDI on ensuring TCC’s efficiency. The reported results are the average proportion of sub-tasks executed by each honest validator out of all VA sub-tasks in each round.

Blockchain-Based Federated Learning

See all 10 papers

DP-FedAdamW: An Efficient Optimizer for Differentially Private Federated Large Models
Feb 23, 2026Jin Liu, Yinbin Miao, Ning Xi +1Differentially Private Federated Learning Federated Learning
School of Cyber Engineering, Xidian University, Xi’an, China · College of Intelligence and Computing, Tianjin University, Tianjin, China
Representation-Aligned Multi-Scale Personalization for Federated Learning
Apr 13, 2026Wenfei Liang, Wee Peng TayFederated Learning Heterogeneous Federated Learning
Department of Electrical and Electronic Engineering, Nanyang Technological University, Singapore.
Few-for-Many Personalized Federated Learning
Mar 12, 2026Ping Guo, Tiantian Zhang, Xi Lin +3Federated Learning Heterogeneous Federated Learning
City University of Hong Kong · CityU Shenzhen Research Institute · Hong Kong Metropolitan University +3

Feb 23, 2026J/K move · Enter open · S save

DP-FedAdamW: An Efficient Optimizer for Differentially Private Federated Large Models

Jin Liu, Yinbin Miao, Ning Xi, Junkang Liu

School of Cyber Engineering, Xidian University, Xi’an, China · College of Intelligence and Computing, Tianjin University, Tianjin, China

Differentially Private Federated Learning Federated Learning Federated Fine-Tuning

Open paper AskPDF arXiv

Balancing convergence efficiency and robustness under Differential Privacy (DP) is a central challenge in Federated Learning (FL). While AdamW accelerates training and fine-tuning in large-scale models, we find that directly applying it to Differentially Private FL (DPFL) suffers from three major issues: (i) data heterogeneity and privacy noise jointly amplify the variance of second-moment estimator, (ii) DP perturbations bias the second-moment estimator, and (iii) DP amplify AdamW sensitivity to local overfitting, worsening client drift. We propose DP-FedAdamW, the first AdamW-based optimizer for DPFL. It restores AdamW under DP by stabilizing second-moment variance, removing DP-induced bias, and aligning local updates to the global descent to curb client drift. Theoretically, we establish an unbiased second-moment estimator and prove a linearly accelerated convergence rate without any heterogeneity assumption, while providing tighter

(\varepsilon,δ)

-DP guarantees. Our empirical results demonstrate the effectiveness of DP-FedAdamW across language and vision Transformers and ResNet-18. On Tiny-ImageNet (Swin-Base,

\varepsilon=1

), DP-FedAdamW outperforms the state-of-the-art (SOTA) by 5.83%. The code is available in Appendix.

Abstract

Blockchain-Based Federated Learning

Explore similar work

DP-FedAdamW: An Efficient Optimizer for Differentially Private Federated Large Models

Representation-Aligned Multi-Scale Personalization for Federated Learning

Few-for-Many Personalized Federated Learning