GUIDE-FBO: Guidance via Uncertainty Intervention and Distributional Exchange for Federated Bayesian Optimization

Jintao Wei; Chenxi Li; Songhao Wang

stat.MLSep 28, 2026

GUIDE-FBO: Guidance via Uncertainty Intervention and Distributional Exchange for Federated Bayesian Optimization

Authors: Jintao Wei, Chenxi Li, Songhao Wang

Organizations: College of Business, Southern University of Science and Technology Shenzhen, China

Abstract

Federated Bayesian Optimization (FBO) enables distributed agents to collaboratively optimize expensive black-box objectives without sharing raw local observations. However, effective knowledge transfer remains challenging under communication constraints and task heterogeneity. We propose GUIDE-FBO, in which agents exchange compact distributions over the locations of their respective optima inferred from local Gaussian process (GP) posteriors, rather than raw observations, query points, or surrogate parameters. The server merges and reweights these distributional components before returning a subset to each agent. Each agent then constructs a Federated Interventional GP (FI-GP), which preserves the local posterior mean and spatially rescales its covariance for local decision making. For the upper confidence bound (UCB) instantiation, GUIDE-UCB, we prove that any bounded FI-GP uncertainty intervention preserves the leading-order cumulative regret rate of standard GP-UCB. When the transferred distributions place greater support near an optimum than in a suboptimal region, selecting the latter requires greater local posterior uncertainty. Experiments on 12 synthetic benchmarks and three real-world optimization tasks show that GUIDE-FBO remains effective across settings ranging from homogeneous to severely heterogeneous. Ablation results highlight the importance of spatially localized uncertainty intervention, while the communication analysis shows that GUIDE-FBO exchanges only compact distributional messages.

Figures & tables

Heterogeneity	TS	UCB	NEI	FTS	FTS-DE	FMTBO	CGP-TS	CGP-UCB	CGP-NEI	GUIDE-TS	GUIDE-UCB	GUIDE-NEI
Level 1: $(0,0)$	11.33	7.00	5.50	10.42	9.83	5.75	10.17	4.25	2.75	7.25	1.33	2.42
Level 2: $(0.05,0.1)$	10.67	6.17	5.33	10.25	10.75	5.58	10.25	4.67	3.08	7.50	1.58	2.17
Level 3: $(0.3,1.0)$	9.17	4.83	5.42	11.08	10.33	3.67	8.33	4.25	6.75	8.17	2.58	3.42

Table 1: Average rank across 12 synthetic benchmarks under three heterogeneity levels, computed from the final average simple regret over 10 independent runs. Lower is better; best and second-best results are shown in bold and underlined.

Task	TS	UCB	NEI	FTS	FTS-DE	FMTBO	CGP-TS	CGP-UCB	CGP-NEI	GUIDE-TS	GUIDE-UCB	GUIDE-NEI
Landmine Detection (AUC)	0.803987	0.805593	0.805508	0.804052	0.801349	0.805970	0.805247	0.806501	0.805218	0.805001	0.806604	0.806452
Activity Recognition (Acc.)	0.987787	0.987821	0.987804	0.987800	0.987739	0.987891	0.987701	0.987959	0.987990	0.987754	0.987938	0.987994
FedHPO (Acc.)	0.850968	0.850973	0.851145	0.850439	0.851098	0.851049	0.850766	0.851262	0.851068	0.850853	0.851178	0.851271
Average rank	9.67	6.33	5.67	10.00	9.33	5.33	10.00	2.33	5.33	9.67	2.67	1.67

Table 2: Results on three real-world federated optimization benchmarks, averaged over 10 independent runs. Higher task performance and lower average rank are better; best and second-best results are shown in bold and underlined.

Benchmark	Vanilla UCB	Single-Gaussian belief	w/o value-aware reweighting	w/o component merging	w/o agent-specific sampling	Uniform uncertainty scaling	Full GUIDE-UCB
Ackley	19.4350	8.6510	8.4753	8.5349	8.4855	19.1366	8.4785
Rastrigin	80.4771	60.2355	59.1892	59.9796	60.0268	80.3952	58.3082
Zakharov	49.8750	25.5947	22.9307	25.3701	21.5431	52.0749	22.1717
Michalewicz	6.1248	5.9494	5.9814	5.9650	5.8771	6.1078	5.8629
Average rank	6.75	4.50	2.75	3.75	2.50	6.25	1.50

Table 3: Ablation study of GUIDE-UCB on four synthetic benchmarks under Level 2 heterogeneity. Final average simple regret over 10 runs is reported; lower is better. The best and second-best results in each row are shown in bold and underlined, respectively.

	FTS	FTS-DE	FMTBO	CGP	GUIDE
Uplink	500	2000	12	12	22
Downlink	500	2000	2	200	$\leq 105$
Total	1000	4000	14	212	$\leq 127$

Table 4: Communication cost per agent per round for

d=10

. GUIDE-FBO uses diagonal covariance and downloads at most

P=5

components per round.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Function	Dim.	Domain	Maximum	Landscape
Ackley	10	$[-32.768,32.768]^{10}$	$0$	Multimodal
Levy	10	$[-10,10]^{10}$	$0$	Multimodal
Griewank	10	$[-600,600]^{10}$	$0$	Multimodal
Rastrigin	10	$[-5.12,5.12]^{10}$	$0$	Highly multimodal
Weierstrass	10	$[-0.5,0.5]^{10}$	$0$	Irregular multimodal
Ellipsoid	10	$[-5.12,5.12]^{10}$	$0$	Ill-conditioned

Appendix

Table 5: Synthetic benchmark functions. All objectives are evaluated as maximization problems.

Benchmark	Level 1	Level 2	Level 3
Ackley	1.000	0.485	0.027
Levy	1.000	0.450	0.076
Griewank	1.000	0.861	0.155
Rastrigin	1.000	0.489	0.136
Weierstrass	1.000	0.464	0.005
Ellipsoid	1.000	0.740	0.120

Appendix

Table 6: Mean pairwise Spearman correlation between the 16 agent objectives. Each benchmark entry is averaged over 10 task realizations, and Overall additionally averages across the 12 benchmarks. Higher values indicate greater agreement in candidate rankings.

Parameter	Setting
Posterior-optimum samples $M$	$500$
ORF features $D_{\mathrm{ORF}}$	$500$
DPGMM covariance	Diagonal
LCB coefficient $\kappa$	$1.0$
Merge threshold $\delta_{\mathrm{merge}}$	$0.05$
Downlink packet size $P$	$5$

Appendix

Table 7: Default configuration of GUIDE-FBO. The same settings are used across benchmarks unless otherwise stated.

Benchmark	TS	UCB	NEI	FTS	FTS-DE	FMTBO	CGP-TS	CGP-UCB	CGP-NEI	GUIDE-TS	GUIDE-UCB	GUIDE-NEI
Ackley	19.4789	19.3082	18.4994	19.4870	19.6941	19.0814	19.5242	8.7013	6.5675	18.6182	4.4896	5.2015
Levy	24.9042	11.9439	10.5159	23.0893	23.5361	9.3873	22.9011	3.7128	2.6747	11.6750	1.5865	1.9122
Griewank	47.1321	14.1117	13.3126	43.9996	15.4063	11.3637	43.2063	6.3414	4.9622	23.2003	4.9517	6.3052
Rastrigin	105.6477	80.4791	79.9759	102.8319	103.4912	81.3840	102.9572	62.0303	59.7812	90.3331	53.4890	54.6122
Weierstrass	13.5724	12.8962	11.8821	13.5131	13.5052	12.2728	13.5183	8.0722	8.0706	12.4086	3.2670	3.7779
Ellipsoid	146.4074	117.5092	87.9959	138.8153	137.5396	81.6766	140.1417	26.8736	24.0600	98.2097	1.7311	3.0529

Appendix

Table 8: Complete synthetic results under Level 1 (Homogeneous). Entries report the final mean average simple regret over 10 independent runs. Lower is better. The best and second-best results in each row are shown in bold and underlined, respectively.

Benchmark	TS	UCB	NEI	FTS	FTS-DE	FMTBO	CGP-TS	CGP-UCB	CGP-NEI	GUIDE-TS	GUIDE-UCB	GUIDE-NEI
Ackley	19.5136	19.4350	18.7628	19.5511	19.7171	18.9928	19.5347	10.1460	10.9368	18.8906	8.4785	9.5379
Levy	24.3585	11.0529	10.1363	23.4060	25.8641	9.7791	24.2127	4.3481	3.5741	12.7145	3.2816	3.3482
Griewank	44.4283	13.5953	11.9247	44.1421	26.2381	11.3470	43.6216	7.1888	7.1002	24.8356	6.3000	7.8444
Rastrigin	101.5417	80.4771	78.9030	101.4711	101.7745	80.9836	103.0048	64.6460	62.5459	88.0868	58.3082	60.1472
Weierstrass	13.4680	12.9344	12.1369	13.4517	13.6576	12.5151	13.5531	8.9999	9.3717	12.7340	6.6966	6.8316
Ellipsoid	139.8685	113.6023	86.7074	139.5091	130.8015	78.4957	136.6720	37.2115	29.1808	91.6564	13.1331	13.3723

Appendix

Table 9: Complete synthetic results under Level 2 (Mild heterogeneity). Entries report the final mean average simple regret over 10 independent runs. Lower is better. The best and second-best results in each row are shown in bold and underlined, respectively.

Benchmark	TS	UCB	NEI	FTS	FTS-DE	FMTBO	CGP-TS	CGP-UCB	CGP-NEI	GUIDE-TS	GUIDE-UCB	GUIDE-NEI
Ackley	20.4047	20.5073	20.4203	20.4080	20.3972	20.2383	20.3699	20.3380	20.4331	20.3552	20.1739	20.1161
Levy	37.5643	24.3700	25.5279	41.1159	40.0354	25.0598	34.5967	24.6258	30.4253	34.2904	20.8306	23.6984
Griewank	67.8144	28.8836	26.9351	72.7054	74.5664	27.0838	65.7857	21.7611	30.0882	68.9316	21.2957	29.7074
Rastrigin	106.3844	95.6377	96.0487	110.2650	111.8623	95.4028	108.3831	96.2275	100.1082	104.5270	90.8849	94.0912
Weierstrass	13.6349	13.1439	12.7244	13.6075	13.6386	12.8031	13.4763	12.9093	12.8169	13.4107	12.8443	12.4937
Ellipsoid	157.2353	162.9886	138.9518	172.5919	167.3659	120.5134	156.5689	134.2802	151.8904	149.4864	127.6623	127.9452

Appendix

Table 10: Complete synthetic results under Level 3 (Severe heterogeneity). Entries report the final mean average simple regret over 10 independent runs. Lower is better. The best and second-best results in each row are shown in bold and underlined, respectively.

Benchmark	Vanilla UCB	Single-Gaussian belief	w/o value-aware reweighting	w/o component merging	w/o agent-specific sampling	Uniform uncertainty scaling	Full GUIDE-UCB
Ackley	20.5073	20.2321	20.1612	20.2473	20.2122	20.4661	20.1739
Rastrigin	95.6377	92.1619	94.3347	92.5862	90.7210	90.2925	90.8849
Zakharov	101.9986	95.9903	109.8422	96.5643	98.2310	103.0493	92.9444
Michalewicz	7.3490	7.3529	7.5093	7.3660	7.4075	7.4026	7.3793
Average rank	5.00	3.00	5.25	4.00	3.75	4.50	2.50

Appendix

Table 11: Ablation study of GUIDE-UCB on four representative synthetic benchmarks under Level 3 heterogeneity. Entries report the final mean average simple regret over 10 independent runs; lower is better. The last row reports the average rank across the four benchmarks. The best and second-best results are shown in bold and underlined, respectively.

Parameter	Tested values	Default
posterior-optimum samples $M$	$\{100,250,\mathbf{500},750,1000\}$	500
Downlink packet size $P$	$\{1,3,\mathbf{5},8,10\}$	5
Maximum guidance $\lambda_{\max}$	$\{0.25,0.5,\mathbf{1.0},2.0\}$	1.0
Merging threshold $\delta_{\mathrm{merge}}$	$\{0.01,0.02,0.03,0.04,\mathbf{0.05}\}$	0.05

Appendix

Table 12: Hyperparameter grids used in the Level 3 sensitivity analysis. Bold entries denote the default settings.

Parameter	Value	Rastrigin	Michalewicz
$M$	100	92.6392	7.3885
	250	91.9958	7.4102
	500	90.8849	7.3793
	750	91.3377	7.3489
	1000	93.1664	7.3473
$P$	1	92.5324	7.3935

Appendix

Table 13: Sensitivity results of GUIDE-UCB on Rastrigin and Michalewicz under Level 3 heterogeneity. Entries report the final mean average simple regret over 10 independent runs; lower is better.

Benchmark	TS	UCB	NEI	FTS	FTS-DE	FMTBO	CGP-TS	CGP-UCB	CGP-NEI	GUIDE-TS	GUIDE-UCB	GUIDE-NEI
Ackley	129.51	143.48	269.34	141.88	148.32	160.56	252.10	411.90	1811.38	267.19	275.87	402.82
Levy	111.27	125.76	262.50	133.12	124.19	256.59	237.84	263.90	765.01	215.37	217.87	499.63
Griewank	85.13	99.67	279.07	118.16	113.43	242.71	246.30	253.81	773.83	179.68	185.57	344.68
Rastrigin	120.42	163.72	290.74	128.78	121.39	297.36	265.75	359.93	1221.39	227.67	262.98	555.44
Weierstrass	126.20	140.59	250.92	141.79	149.75	171.29	246.86	347.38	1050.71	266.59	267.94	677.94
Ellipsoid	175.70	164.10	306.68	161.63	153.78	246.67	304.80	326.98	892.53	249.66	253.14	624.32

Appendix

Table 14: Mean wall-clock time per independent run on the synthetic benchmarks under Level 2 heterogeneity, averaged over 10 runs. Unit: seconds.

Explore similar work

Federated Agent Optimization
Oct 1, 2026Qiang Yang, Zhiqiang Kou, Xueyi Zhang +6Federated Learning Agentic Optimization
The Hong Kong Polytechnic University, Hong Kong SAR, China · National University of Singapore, Singapore · The University of Tokyo, Japan +2
Federated Bilevel Performative Prediction
Jun 18, 2026Liangxin Qian, Chang Liu, Xuanyu Cao +2Bilevel Optimization Distribution Shifts
College of Computing and Data Science, Nanyang Technological University, 639798, Singapore · Innovation and Management Center, School of Software Technology (Ningbo), Zhejiang University, 315100, China · School of Electrical Engineering & Computer Science, Washington State University, Pullman, WA 99164, USA.
No-Regret Bayesian Optimization with Finite-Library Input-Warped Kernels
Sep 2, 2026Edvin Ketabati Augustinsson, Robert A. BridgesBayesian Optimization Gaussian Process
AI Sweden, Gothenburg, Sweden

Oct 1, 2026, cs.AIJ/K move · Enter open · S save

Federated Agent Optimization

Qiang Yang, Zhiqiang Kou, Xueyi Zhang, Dong-Dong Wu+5

The Hong Kong Polytechnic University, Hong Kong SAR, China · National University of Singapore, Singapore · The University of Tokyo, Japan+2

Federated Learning Agentic Optimization

Open paper AskPDF arXiv

Large language model (LLM) agents increasingly operate in private environments and accumulate valuable experience from task execution, tool use, feedback, and local knowledge. Yet such experience is distributed across organizations and cannot be directly shared because of privacy and proprietary constraints. Conventional federated learning is insufficient for this setting, as agent capabilities extend beyond model parameters to memory, tools, rewards, skills, and structured knowledge. In this paper, we formulate \textbf{Federated Agent Optimization (FAO)}, which studies how distributed agents can collaboratively improve through controlled information exchange while keeping raw data, complete trajectories, and private knowledge local. We define FAO as a multi-objective problem balancing agent utility, privacy leakage, and communication cost, and organize its optimization space across policy, memory, tool use, reward, and structured knowledge and skills. We further characterize how private experience can be abstracted, protected, aggregated, and adapted into transferable capabilities, providing a unified view of how agents can benefit from one another without direct experience sharing. Finally, we identify the key challenges of FAO and outline several promising directions for future research toward trustworthy federated agent systems.

Abstract

Explore similar work

Federated Agent Optimization

Federated Bilevel Performative Prediction

No-Regret Bayesian Optimization with Finite-Library Input-Warped Kernels