This paper studies model-free federated learning of a common LQR controller for a fleet of similar systems that communicate with a server over an uplink subject to limited bandwidth, packet dropout, and additive noise. We propose ResiFedLQR, a scalar communication scheme in which each agent sends one real number per packet, with redundancy, instead of full gradient estimates. The scalars can be gradient coordinates, projections of the gradient onto random directions reproduced at the server from a shared seed, or the rollout costs themselves. For the projection encoding, we prove that, with high probability, every iterate stabilizes all agents and the fleet-average cost converges linearly to a neighborhood of the optimum whose radius separates heterogeneity, sampling, and communication effects, and we derive a rule for choosing the number of transmissions from the channel parameters and the fleet size. We then compare the encodings with full-gradient transmission under the same uplink budget and identify the signal-to-noise regime and power constraint under which each is preferable. Numerical experiments on a heterogeneous fleet evaluate stability and convergence under varying noise levels and dropout probabilities.
Figures & tables
Fig. 1 : Federated LQR over an unreliable uplink. The server broadcasts the common gain Kt over a reliable downlink. Each agent runs rollouts on its own unknown system, forms a local gradient estimate, and returns messages over an uplink subject to packet dropout and additive noise.
GL
OL
CL
Scalar sent
v⊤g~t,n
rollout cost Jt,s(n)
[g~t,n]i
Packets/round
Nsc (design choice)
ns (= rollouts)
dNF
Local computation
ZO estimate
none
ZO estimate
Encoding error
(d−1)∥g~t,n∥2/Nt,nrec
none
none
Channel-noise 2nd moment
dσch2/Nt,nrec
d2σch2/(r2Nt,nrec)
≈d2σch2/Nt,nrec
A lost packet removes
one projection
one rollout
one copy of one coordinate
TABLE I : Comparison of the scalar encodings of Definition 1 .
Fig. 2 : Fraction of realizations losing closed-loop stability versus channel noise for gradient-level (GL) and oracle-level (OL) encoding, at equal rollouts ( ns=5 ) and equal packets ( Nsc=ns per agent and round), in the setting of Section VI ( d=9 , r=0.1 , pdrop=0.2 , 20 realizations per noise level).
Fig. 3 : Empirical resilience regions ( 52 ) over stepsize and dropout probability for σch∈{1,2,3,4} at (NF,Nsc)=(8,72) .
σch
1
2
3
4
FedLQR
52%
12%
0%
0%
ResiFedLQR
100%
79%
52%
38%
TABLE II : Fraction of the 10×10(η,pdrop) grid in the empirical resilience region ( 52 ).
Fig. 4 : Empirical failure probability pfail over stepsize and dropout probability. Top: FedLQR ; bottom: ResiFedLQR . Darker colors indicate smaller pfail ; solid and dashed curves mark pfail=0 and 0.5 .
Fig. 5 : Normalized optimality gap ( 53 ) versus round under increasing uplink channel noise, at equal uplink budget Nsc=dNF and pdrop=0.2 . Lines are medians over 75 realizations and shaded bands are interquartile ranges, truncated at the panel edges; the dotted line marks G0 .
σch=1
σch=2
(NF,Nsc)
F
R
F
R
(1,9)
0.73 (15)
0.60 (0)
0.75 (44)
0.58 (0)
(2,18)
0.60 (1)
0.62 (0)
0.62 (30)
0.64 (0)
(4,36)
0.67 (0)
0.59 (0)
0.60 (11)
0.59 (0)
(6,54)
0.60 (0)
0.60 (0)
0.63 (1)
0.60 (0)
(8,72)
0.59 (0)
0.59 (0)
0.57 (1)
0.60 (0)
TABLE III : Median of GT over 75 realizations at pdrop=0.2 ( G0=0.93 ); in parentheses, the number of realizations with maxt≤TGt>5 . F: FedLQR ; R: ResiFedLQR .
Fig. 6 : Normalized optimality gap ( 53 ) versus cumulative uplink bits for σch=1 and pdrop=0.2 . Lines are medians over 75 realizations; the vertical line marks the budget of 5.76×106 bits and the dotted line marks G0 .
Fig. 7 : Normalized mean-square reconstruction error of one transmission round versus normalized channel noise, for d=9 , NF=2 , and pdrop=0.2 at equal budget. Lines: ( 47 ) and ( 48 ); markers: Monte Carlo estimates from 4×104 realizations. The dashed line marks the crossover.
Fig. 8 : Fraction of realizations losing closed-loop stability versus channel noise for bundled full-gradient transmission ( FedLQR ), CL encoding (element-wise) without and with a transmit gain of d , and GL encoding ( ResiFedLQR ), at (NF,Nsc)=(18,162) , pdrop=0.2 , η=10−6 , and T=2000 , with 20 realizations per noise level. (a) Fleet of Section VI-A ( M=10 ). (b) The same fleet extended to M=100 agents.
pdrop=0.1
0.3
0.5
0.7
η
F
G
F
G
F
G
F
G
5×10−7
0
0
3
0
2
0
0
0
1×10−6
5
0
4
0
5
0
8
1
2×10−6
7
0
6
2
7
2
8
3
4×10−6
8
2
8
2
8
6
8
6
TABLE IV : Number of realizations, out of 8 , losing closed-loop stability for M=100 , (NF,Nsc)=(18,162) , and σch=16 . F: bundled full-gradient transmission; G: GL encoding.
Fig. 9 : (a) Gap after T=2×104 rounds versus heterogeneity level γ : median and interquartile range over 20 realizations; dashed: gain minimizing Javg . (b) Fraction of realizations losing stability versus channel noise for d∈{9,36,81} at equal budget.
Problem
M , d
number of agents; policy dimension d=nunx
J(n) , Javg
agent and fleet-average costs ( 2 ), ( 4 )
S , Sc
gains stabilizing all agents ( 5 ); sublevel set of K0 ( 27 )
ϵ1,ϵ2
heterogeneity bounds ( 3 )
Ft , Fˉ
optimality gap Javg(Kt)−Javg⋆ ; certified radius ( 42 )