Rapid Mean-Field Formation Transport with a Shared-Representation Neural Fokker-Planck Controller
Authors: Zejian Zhou, Hao Xu
Organizations: Department of Electrical Engineering and Computer Science, University of Wyoming, Laramie, WY 82071, USA. · Department of Electrical and Biomedical Engineering, University of Nevada, Reno, Reno, NV 89557, USA.
This paper develops a mean-field formation transport method with low online computational cost for large-scale multi-agent systems. Classical mean-field planners often require repeated solutions of coupled forward and backward partial differential equations, making online replanning computationally demanding. Density and control are therefore approximated by a common parameterized nonlinear map with separate scalar density and vector control outputs. Parameters are optimized using Fokker--Planck and initial-condition residuals, formation-matching costs on numerically simulated closed-loop particle trajectories, and control effort. Numerical computations are restricted to a bounded region of the state space, with an additional penalty on probability flux across its boundary. For the periodic-domain model, we establish well-posedness of the induced density evolution and bound the density-approximation error in terms of the dynamics and initial-condition residuals. Parameter optimization precedes online operation, during which controller coefficients remain fixed. Experiments quantify the accuracy--runtime trade-off relative to a mean-field planner, show lower transport error than classical feedback baselines, and demonstrate the benefits of the common parameterization.
Figures & tables
Fig. 1: Evolution of the 2-Wasserstein distance to the target distribution on the 1D benchmark. The plotted reference ρt⋆ is the fixed target density.
Method
W2
Energy
Time (s)
SR-NFPC
0.077
3.626
0.101
Mean-field plan [ 10 ]
0.025
3.886
0.307
Centralized sampled-goal
0.040
5.077
0.044
TABLE I: 1D benchmark terminal metrics and total runtime.
Fig. 2: Terminal Wasserstein distance across the five formation tasks. SR-NFPC achieves the lowest error among the compared methods on all five tasks.
Method
Avg. W2
Avg. energy
Best W2 count
SR-NFPC
0.428
1.412
5/5
Potential field [ 2 ]
0.719
1.182
0/5
Consensus [ 17 , 3 ]
1.159
1.581
0/5
No control
0.931
0.000
0/5
TABLE II: Average performance over five formation tasks.
Metric
Shared map
Separate maps
Terminal W2
0.0394
0.0542
Control energy
4.158
4.756
Runtime (s)
0.0621
0.0862
Parameter count
2594
17154
TABLE III: Shared representation versus separate density/control networks on the transition task.
We develop a model-free policy gradient method for discrete-time mean-field control (MFC). In MFC, the policy affects the objective both through the controlled dynamics and through the population distribution. Standard REINFORCE estimators capture the first effect but not the second. We introduce Transport REINFORCE, a transport map-based approach that perturbs a suitable transformation of the population distribution to estimate this missing mean-field contribution. The method applies to both finite and continuous state spaces. In finite state spaces, we perturb the population distribution directly on the probability simplex through a convex combination of the current population weights and random weights. In continuous state spaces, we project the population distribution onto the manifold of Gaussian mixtures, and then randomize it via a transport map that ensures the perturbed law remains within this manifold. We prove consistency of the perturbed objective and gradient as the perturbation vanishes, and derive bias and mean-square error bounds for the resulting sample-based gradient estimator. Numerical experiments on several MFC benchmarks show that Transport REINFORCE improves over standard REINFORCE.
Stochastic mean-field control (MFC) provides a fundamental framework for coordinating large populations of interacting agents under uncertainty, with a wide range of applications. Existing numerical and deep-learning methods solve one MFC problem instance at a time and must be re-optimized whenever the task changes. In this work, we formulate stochastic MFC as an operator-learning problem and develop, to the best of our knowledge, the first mesh-free, self-supervised neural operator for stochastic MFC. The main challenge is that the diffusion term in the controlled Fokker--Planck equation precludes deterministic transport-map representations. We address this challenge by combining the probability-flow ODE with an invertible normalizing-flow-based transformer, which recasts the dynamics as a deterministic continuity equation and enables closed-form score evaluation through the exact inverse and analytical log-determinant of the normalizing flow, with O(d) cost per particle for networks of fixed size. Through transformer-based in-context learning, task prompts, represented by compact distribution parameters or raw particle clouds, condition the transport map, enabling a single pretrained operator to solve unseen tasks in one forward pass. The resulting Normalizing Flow Invertible Solution Transformer (NFIST) is trained end-to-end by minimizing the stochastic control objective directly, requiring no precomputed numerical solutions for training. We further prove the consistency of the proposed operator-learning formulation with task-by-task optimization. Numerical experiments on stochastic optimal control, Schrödinger bridge, systemic-risk control, and obstacle-avoiding path planning demonstrate effective zero-shot generalization while substantially reducing the computational cost of solving large families of stochastic MFC problems.
Suyi Gao, Mo Zhou, Rongjie Lai
Department of Mathematics, Purdue University, West Lafayette, USA · Department of Mathematics, University of California, Los Angeles, USA
Diffusion-based planning has achieved strong results in single-agent offline reinforcement learning, yet scaling to many-agent systems remains intractable due to the curse of dimensionality in the joint trajectory space. We introduce MF-Diffuser, a framework that lifts trajectory planning to the Wasserstein space of trajectory distributions, where the propagation of chaos ensures a small representative subset of agents captures the full population dynamics. Our approach features a value-weighted chaotic entropy objective that reconciles generative fidelity with return maximization, and a hierarchical coarse-to-fine strategy that progressively grows the agent population during denoising. We establish end-to-end suboptimality bounds with four interpretable terms, revealing that mean-field approximation error scales as O(H2/N) while offline distribution shift provably does not grow with population size N, and prove the generated policy is an approximate mean-field Nash equilibrium with explicit convergence guarantees. Experiments on three mean-field RL benchmarks -- spanning stage games, sequential dynamics, and adversarial team competition -- show MF-Diffuser achieves the best return in the majority of settings, with the largest gains on suboptimal offline data and at extreme scales (N≥103).