RegionFed: Federated Learning for Personalized Query Understanding in Heterogeneous Retail Environments

Quoc H. Nguyen; Ali Lafzi; Abhijeet Phatak; Siddharth Pratap Singh; Rohit Upadhyay; Yogananda Domlur Seetharama; Chittaranjan Tripathy

cs.LGSep 4, 2026

RegionFed: Federated Learning for Personalized Query Understanding in Heterogeneous Retail Environments

Authors: Quoc H. Nguyen, Ali Lafzi, Abhijeet Phatak, Siddharth Pratap Singh, Rohit Upadhyay, Yogananda Domlur Seetharama, Chittaranjan Tripathy

Organizations: Walmart Global Tech Sunnyvale California, USA

Abstract

Retail search systems serve diverse geographic regions with distinct query patterns, vocabularies, and product preferences, creating significant data heterogeneity that challenges both privacy-preserving training and model personalization. Federated learning offers a natural solution for privacy, but standard FL methods produce global models that sacrifice regional performance, while existing personalized FL approaches operate at the parameter level and catastrophically collapse on modern transformers (below 10% accuracy on T5) due to tied embeddings and LayerNorm interactions. We introduce RegionFed, an \textit{architecture-robust} federated learning framework that sidesteps this failure by operating entirely at the gradient level. RegionFed uses the $\ell_2$ conflict between regional and global gradients as a unified signal that (i) diagnoses heterogeneity, (ii) routes each region to the cheapest sufficient personalization strategy, and (iii) adaptively controls personalization strength. Because it treats models as differentiable black boxes, RegionFed deploys on T5-Small, T5-3B, RoBERTa, and CNN with zero code changes, providing large gains on transformers (where parameter-level methods collapse) and consistent improvements on CNNs. Across three public datasets (Amazon ESCI, Amazon Reviews, LEAF-FEMNIST) and four architectures, RegionFed-Meta achieves 92.27%, closing the gap to the privacy-violating centralized upper bound (Centralized + Regional Weighting: 92.04%, $Δ$ =0.23pp, within 1 $σ$ ) while providing $(ε{\approx}0.60)$ -differential privacy and $\mathcal{O}(1/\sqrt{T})$ convergence.

Figures & tables

Panel of Figure 1 — Figure 1: RegionFed architecture: three-layer hierarchy with global server, regional coordinators ( $M$ regions), and local clients. Blue bidirectional arrows show gradient/model exchange; right panel shows adaptive components (strategy selection, golden section $\rho$ optimization, adaptive $\alpha_{r}$ weights).

Category	Method	Overall (%) $\uparrow$	Intent (%) $\uparrow$	Spell (%) $\uparrow$	NER (%) $\uparrow$	RRS $\uparrow$
Panel A: Amazon ESCI, T5-Small (primary benchmark, per-task breakdown)
Upper Bound	Centralized	91.72 $\pm$ 0.28	96.35 $\pm$ 0.15	78.89 $\pm$ 0.34	99.93 $\pm$ 0.02	–
(no privacy)	Cent. + Reg. Weighting	92.04 $\pm$ 0.25	96.72 $\pm$ 0.18	79.84 $\pm$ 0.31	99.55 $\pm$ 0.08	91.54
Standard FL	FedAvg	80.18 $\pm$ 0.31	78.79 $\pm$ 0.44	75.94 $\pm$ 0.38	85.81 $\pm$ 0.27	80.04
	FedProx ( $\mu{=}0.01$ )	67.18 $\pm$ 0.42	59.03 $\pm$ 0.51	75.47 $\pm$ 0.35	67.05 $\pm$ 0.48	67.13
	Reg-Aware FedAvg	83.47 $\pm$ 0.29	81.52 $\pm$ 0.37	76.89 $\pm$ 0.33	92.01 $\pm$ 0.21	83.21

Table 1: Overall performance across datasets and architectures (50 rounds, 5 seeds). Bold : best federated; underline : second-best. Red : collapsed (

<

10%). RF-Meta closes the gap to the centralized upper bound (

\Delta

=0.23pp) while parameter-level methods collapse on transformers but not CNN.

Method	1.0 (Low)	0.3 (Med)	0.1 (High)	Deg. $\downarrow$	Reg. SD $\downarrow$	RRS $\uparrow$
	RRS by $\alpha_{D}$
FedAvg	89.1	80.04	67.5	21.6	2.98	80.04
FedProx	89.5	67.13	69.2	20.3	3.40	67.13
Reg-Aware FedAvg	89.8	83.21	71.3	18.5	2.54	83.21
Local FT	90.2	79.53	78.3	11.9	3.21	79.53
FedBABU	89.8	84.12	72.4	17.4	2.18	84.12

Table 2: Robustness to heterogeneity and regional equity

Region	Query	Task	Predicted	Best	Global	Gain
Electronics	apple tv	Intent	brand_lookup	92.3%	84.5%	+7.8%
Fashion	butter avlable	Spell	butter available	91.3%	83.6%	+7.7%
Home	tv cost	NER	PRODUCT O	91.4%	85.8%	+5.6%
Sports	need yoga mat	Intent	product_search	91.9%	84.1%	+7.8%
Beauty	supplements on sale	Intent	price_comparison	93.4%	87.2%	+6.2%

Table 3: Query-level case studies: RegionFed produces correct personalized predictions with 5–8% gain over the global (non-personalized) baseline across regions and tasks.

Figure 2: Convergence over 50 rounds. Full y-axis (0–96%) shows SCAFFOLD’s collapse (

\sim

9%); inset zooms into top methods (83–93%) to differentiate converged performance. RegionFed plateaus above 91% by round 5; FedAvg reaches

\sim

82% by round 20. The persistent gap confirms additional rounds cannot compensate for lacking regional personalization.

Appendix figures & tables23 assets

Supplementary material from the paper’s appendix.

Appendix

Metric	Value
Total / Train / Test Samples	20,000 / 16,000 / 4,000
Regions / Clients per Region	8 / 10 (80 total)
Vocabulary Size / Max Query Length	5,000 words / 20 tokens
Intent Classes / Entity Types	8 / 6 (5 + O tag)
Product Categories	24
Dirichlet $\alpha_{D}$ / Regional Bias	0.3 / 70%

Appendix

Table 4: Amazon ESCI Benchmark Statistics

ID	Region Type	Focus	Top Categories
0	Urban Tech	Tech	Electronics, Gaming
1	Young Demo.	Fashion	Fashion, Apparel
2	High-Income	Premium	Fashion, Electronics
3	Price-Conscious	Value	Grocery, Food
4	Active Lifestyle	Outdoor	Sports, Fitness
5	Business Prof.	Prof.	Apparel, Books

Appendix

Table 5: Regional Shopping Preferences and Product Focus

Figure 3: Intent Distribution Across 80 Clients. Heatmap showing the proportion of each intent category across all 80 clients organized into 8 regions (blue dashed lines). Darker colors indicate higher specialization, demonstrating non-IID distribution from Dirichlet $\alpha{=}0.3$ . Bottom panel shows samples per region.

Figure 4: Regional Product Category Distribution. Top: client-level category heatmap (80 clients

\times

24 categories). Bottom: regional preference matrix validating the 70% bias formulation. Each region concentrates on designated categories (

C_{r}

) while maintaining 30% cross-regional diversity.

Strategy	Acc.	RRS	Cost	Best For
Meta	92.3%	91.6%	High	Max accuracy
Interp	92.1%	86.1%	Med	Perfect NER
Grad	91.9%	91.5%	Low	Edge devices
Dynamic	91.9%	90.2%	Var.	Automation

Appendix

Table 6: Strategy Trade-offs Summary

Strategy	Compute	Memory	Comm. Overhead	Best For
Grad	$O(\|\theta\|)$	$O(\|\theta\|)$	Same as FedAvg	Edge/mobile devices; latency-critical deployments
Interp	$O(E\cdot\|\theta\|)$	$2\times O(\|\theta\|)$	$<$ 3%	High local compute; smooth regional blending
Meta	$O(2\cdot\|\theta\|)$	$O(\|\theta\|+\|\nabla\|)$	$<$ 3%	Max accuracy (92.27%); RRS-critical applications
Dynamic	Variable	Variable	$<$ 5%	Automated strategy selection; heterogeneous fleets

Appendix

Table 7: Strategy comparison: complexity, memory, and use cases

Region	FedAvg	FedTP	Grad	Interp	Meta	Dyn.
Electronics	82.1	92.0	91.9	84.6	92.3	91.9
Fashion	78.5	91.0	91.3	83.6	91.2	90.8
Home	81.3	91.8	91.2	85.8	91.4	91.1
Grocery	77.2	91.5	91.1	85.5	91.0	78.2
Sports	80.8	91.2	91.8	84.1	91.8	91.9
Books	82.4	90.5	90.4	85.4	90.3	90.3

Appendix

Table 8: Regional Performance Breakdown by Strategy (%)

Figure 5: Regional Performance Comparison Across Strategies. Line plot showing accuracy trends for Grad (pink), Interp (yellow), Meta (green), and Dynamic (cyan) across eight product category regions. Interp consistently underperforms, while Grad/Meta maintain competitive performance with region-specific advantages. Dynamic’s anomalous drop in Grocery reflects the challenge of automated strategy selection.

Figure 6: Optimal Adaptation Intensity by Strategy and Region. Line plot showing

\rho

values for Grad (pink), Interp (yellow), and Meta (green) across regions. Grad and Meta converge to constant

\rho=0.0344

(optimized for efficiency), while Interp exhibits variable

\rho

(0.10-0.34) reflecting its sensitivity to regional data characteristics.

Region	Query	Task	Predicted	Grad	Meta	Interp	Dynamic	Gain
Electronics	apple tv	Intent	brand_lookup	91.9	92.3	84.5	91.9	+7.8
Fashion	butter avlable	Spell	butter available	91.3	91.2	83.6	90.8	+7.7
Home	tv cost	NER	PRODUCT O	91.2	91.4	85.8	91.1	+5.6
Grocery	find laptop deals	Intent	brand_lookup	91.1	91.0	85.5	78.2	+5.6
Sports	need yoga mat	Intent	product_search	91.8	91.8	84.1	91.9	+7.8
Books	wrench discount	Intent	price_comparison	90.4	90.3	85.4	90.3	+5.0

Appendix

Table 9: Query-Level Case Studies: Multi-Task Performance with Actual Predictions

Round	Strategy	Accuracy	Diagnosis
1	grad	91.1%	$\checkmark$ Correct selection
2	interp	86.6%	$\triangle$ Performance degraded
3	interp	78.2%	$\times$ Catastrophic drop
4	interp	86.2%	$\triangle$ Partial recovery
5	interp	78.0%	$\times$ Final collapse

Appendix

Table 10: Dynamic Strategy Selection for Grocery Region Across Rounds

	Intent			Spell			NER
Method	Low	Med	High	Low	Med	High	Low	Med	High
FedAvg	95.2	90.1	78.2	76.3	68.3	49.1	95.8	88.6	75.0
FedProx	95.3	90.7	80.1	76.9	69.2	51.3	96.1	89.5	76.2
Reg-Aware FedAvg	95.6	91.3	81.5	77.1	69.8	53.2	96.3	90.1	79.3
Local FT	96.1	93.2	85.7	77.8	72.1	62.4	96.8	94.1	86.8
FedTP	96.5	93.8	87.1	78.5	73.8	65.2	98.5	97.5	94.0

Appendix

Table 11: Per-Task Performance Under Data Heterogeneity (Accuracy %)

Figure 7: Performance gap heatmap: RF-Meta vs other methods. The gap widens from Low (0.6–2.9%) to High (3.4–18.0%) heterogeneity.

Figure 8: Loss landscape under high heterogeneity (

\alpha{=}0.1

). (a) Regional loss contours show conflicting optima. (b) FedAvg’s aggregated loss shows gradient cancellation; regional optima (

\triangle

\nabla

) are suboptimal globally, forcing a compromise (

\star

). (c) RegionFed finds region-specific optima (

\star

) independently.

Method	4 Reg.	8 Reg.	16 Reg.	Deg. $\downarrow$
FedAvg	83.41 $\pm$ 0.38 %	80.18 $\pm$ 0.31 %	74.92 $\pm$ 0.52 %	8.49%
FedProx	70.53 $\pm$ 0.47 %	67.18 $\pm$ 0.42 %	60.84 $\pm$ 0.61 %	9.69%
Reg-Aware FedAvg	85.27 $\pm$ 0.33 %	83.47 $\pm$ 0.29 %	79.13 $\pm$ 0.44 %	6.14%
FedTP	92.08 $\pm$ 0.21 %	91.54 $\pm$ 0.22 %	89.41 $\pm$ 0.35 %	2.67%
RF-Grad	92.14 $\pm$ 0.24 %	91.92 $\pm$ 0.25 %	89.73 $\pm$ 0.31 %	2.41%
RF-Meta	92.58 $\pm$ 0.22 %	92.27 $\pm$ 0.31 %	90.14 $\pm$ 0.38 %	2.44%

Appendix

Table 12: Scalability Analysis: Performance with Increasing Region Count (all experimental)

Region Assignment	Acc.	$\Delta$	Description
Semantic (default)	92.3%	–	Category-coherent
Geographic proxy	91.1%	–1.2%	By location, mixed categories
Random assignment	88.1%	–4.2%	Uniform random
Adversarial	86.7%	–5.6%	Maximally heterogeneous
FedAvg (no regions)	80.2%	–12.1%	Baseline

Appendix

Table 13: Region Misspecification Analysis (RegionFed-Meta, 8 Regions)

Configuration	Acc.	$\Delta$
Baseline (Regional Only)	88.5%	–
Reg-Aware FedAvg (no adaptive $\alpha$ )	83.5%	–
+ Hierarchical Personalization	89.4%	+0.9%
+ Golden Section $\rho$	88.9%	+0.5%
+ Adaptive $\alpha$	89.6%	+1.1%
+ Hier. + $\rho$	89.8%	+1.4%

Appendix

Table 14: Ablation Study: Component Contributions

$\tau$	Overall Acc.	RRS	Reg. $\sigma$
0.5	91.1%	90.8%	1.21
0.75	91.8%	91.3%	1.05
1.0	92.3%	91.6%	0.97
1.5	91.9%	91.2%	1.08
2.0	91.0%	90.5%	1.34

Appendix

Table 15: Temperature Sensitivity (

\tau_{r}=\tau_{u}

)

$\mu_{r}$ Setting	Overall Acc.	$\alpha_{r}$ Range	$\Delta$
$0.5\times\mu_{r}^{cal}$	91.5%	[0.68, 0.95]	–0.8%
$0.75\times\mu_{r}^{cal}$	91.9%	[0.61, 0.92]	–0.4%
$\mu_{r}^{cal}$ (calibrated)	92.3%	[0.53, 0.88]	–
$1.25\times\mu_{r}^{cal}$	92.0%	[0.51, 0.82]	–0.3%
$1.5\times\mu_{r}^{cal}$	91.6%	[0.50, 0.76]	–0.7%
Fixed $\mu_{r}=0$	90.1%	[0.82, 0.97]	–2.2%

Appendix

Table 16: Sensitivity to

\mu_{r}

Perturbation (RegionFed-Meta)

$\alpha$ Range	Overall Acc.	$\Delta$
$[0.0,1.0]$	89.1%	-3.2%
$[0.25,1.0]$	90.8%	-1.5%
$[0.5,1.0]$	92.3%	–
$[0.75,1.0]$	91.4%	-0.9%

Appendix

Table 17: Effect of Minimum Personalization Weight

\alpha_{min}

$C$	$\epsilon$	Overall Acc.	RRS	$\alpha_{r}$ Range	Effect
0.1	0.60	89.14%	88.52%	[0.50, 0.54]	Severe clipping bias
0.5	0.60	91.48%	90.87%	[0.51, 0.72]	Moderate clipping
1.0	0.60	92.27%	91.62%	[0.53, 0.88]	Balanced (default)
5.0	0.60	91.82%	91.15%	[0.50, 0.94]	High noise variance
$\infty$	$\infty$	92.53%	91.85%	[0.50, 0.96]	No DP (upper bound)

Appendix

Table 18: DP Clipping Threshold

C

Sensitivity (RegionFed-Meta,

\sigma_{dp}{=}4.0

)

Figure 9: Per-region

\alpha_{r}

trajectories across training rounds. High-conflict regions (Electronics, Grocery) maintain

\alpha_{r}\approx 0.85

, while low-conflict regions (Books, Health) settle near

\alpha_{r}\approx 0.55

. All regions stabilize after round 2.

$\sigma_{dp}$	1.0	2.0	4.0	8.0	16.0
$\epsilon$ (approx.)	2.40	1.20	0.60	0.30	0.15
FedAvg	80.85	80.52	80.18	78.93	74.21
RF-Meta	92.54	92.41	92.27	91.48	88.15
$\Delta$ (RF-Meta $-$ FedAvg)	+11.69	+11.89	+12.09	+12.55	+13.94

Appendix

Table 19: Privacy-utility tradeoff: accuracy (%) at varying privacy budgets

Explore similar work

Global Federated Learning Strategies for Building Efficient Personalized Models
Aug 15, 2026Seongyoon KimFederated Learning Heterogeneous Federated Learning
한국과학기술원 Korea Advanced Institute of Science and Technology 산업및시스템공학과
FedLAFP: Low-Rank Aggregation Meets Full-Rank Personalization in Federated Fine-Tuning
Sep 29, 2026Mengjun Yi, Huaian Gu, Yinghao Ai +2Low-Rank Adaptation Federated Fine-Tuning
State Key Laboratory for Novel Software Technology · School of Artificial Intelligence · School of Computer Science +1
Fine-Tuning Impairs the Balancedness of Foundation Models in Long-tailed Personalized Federated Learning
May 4, 2026Shihao Hou, Chikai Shang, Zhiheng Yang +5Class-Imbalanced Learning Fine-Tuning
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, Xiamen, China · Department of Statistical Science, University College London, London, United Kingdom · School of Computer Science and Technology, Guangdong University of Technology, Guangzhou, China

Aug 15, 2026, cs.LGJ/K move · Enter open · S save

Global Federated Learning Strategies for Building Efficient Personalized Models

Seongyoon Kim

한국과학기술원 Korea Advanced Institute of Science and Technology 산업및시스템공학과

Federated Learning Heterogeneous Federated Learning Federated Fine-Tuning Reward Modeling

Open paper AskPDF arXiv

Federated learning (FL) is a practical framework that can train models on distributed user data while guaranteeing data privacy; however, due to heterogeneity in which each user has a different data distribution, problems frequently arise where both global and personalization performance deteriorate simultaneously. This dissertation presents methodologies for building efficient personalized models by identifying which strategies are effective in the global training stage and by showing how to preserve global knowledge while securing user-specific performance during local adaptation. First, we show that as data heterogeneity increases, the collapse of feature vectors is a more fundamental bottleneck than classifier weights, and propose a method that directly mitigates the discrepancy in representation magnitude between local and global models. Second, we analyze that a training approach that strengthens local alignment can induce forgetting of global knowledge (e.g., categories not observed locally), and propose a method that achieves both local alignment and global knowledge preservation by combining feature distillation based on the global model's feature vectors. Third, in federated personalized reward model learning with preference heterogeneity, we empirically verify the conventional belief that "increasing the number of global models yields better initialization," and we show that when sufficient local fine-tuning is allowed, a single global initialization can instead provide stronger personalization performance. This study redefines the role of global initialization under data and preference heterogeneity and provides practical training strategies that simultaneously satisfy global knowledge preservation and personalization.

Abstract

Explore similar work

Global Federated Learning Strategies for Building Efficient Personalized Models

FedLAFP: Low-Rank Aggregation Meets Full-Rank Personalization in Federated Fine-Tuning

Fine-Tuning Impairs the Balancedness of Foundation Models in Long-tailed Personalized Federated Learning