Toward Generalist Neural Motion Planners for Robotic Manipulators: Challenges and Opportunities
Authors: Davood Soleymanzadeh, Ivan Lopez-Sanchez, Hao Su, Yunzhu Li, Xiao Liang, Minghui Zheng
Organizations: J. Mike Walker ’66 Department of Mechanical Engineering, Texas A&M University, College Station, TX, USA · Lab of Biomechatronics and Intelligent Robotics, Department of Biomedical Engineering, Tandon School of Engineering, New York University, NY, USA · Department of Computer Science, Columbia University, NY, USA · Zachry Department of Civil and Environmental Engineering, Texas A&M University, College Station, TX, USA
State-of-the-art generalist manipulation policies have enabled the deployment of robotic manipulators in unstructured human environments. However, these frameworks struggle in cluttered environments primarily because they utilize auxiliary modules for low-level motion planning and control. Motion planning remains challenging due to the high dimensionality of the robot's configuration space and the presence of workspace obstacles. Neural motion planners have enhanced motion planning efficiency by offering fast inference and effectively handling the inherent multi-modality of the motion planning problem. Despite such benefits, current neural motion planners often struggle to generalize to unseen, out-of-distribution planning settings. This paper reviews and analyzes the state-of-the-art neural motion planners, highlighting both their benefits and limitations. It also outlines a path toward establishing generalist neural motion planners capable of handling domain-specific challenges. For a list of the reviewed papers, please refer to https://davoodsz.github.io/planning-manip-survey.github.io/.
Figures & tables
Fig. 1: Deep learning for robotic manipulator motion planning. (a, b) Point cloud networks (PCNets) for End-to-end (E2E) planning [ 1 , 2 ] (Section V-A ). (c) Variational autoencoders (VAEs) [ 3 ] , and (d) normalizing flows [ 4 ] for informed sampling (Section V-C ). (e) Multilayer perceptions (MLPs) for trajectory optimization [ 5 ] (Section V-D ). (f) Convolutional neural networks (CNNs) for E2E planning [ 6 ] (Section V-A ). (g) Generative adversarial networks (GANs) for constraint manifold learning [ 7 ] (Section V-C ). (h) MLPs for trajectory optimization [ 8 ] (Section V-D ). (i) Graph neural networks (GNNs) for collision checking [ 9 ] (Section V-E ). (j) MLPs for steering [ 10 ] (Section V-B2 ). (k) VAEs for trajectory optimization [ 11 ] (Section V-D ). (l) CNNs for collision checking [ 12 ] (Section V-E ).
Fig. 2: Overview of this survey paper. Figures are adopted from Carvalho et al. [ 19 ] , Qureshi et al. [ 20 ] , Bency et al. [ 21 ] , and Song et al. [ 22 ]
Reference
Robot Types
Scope and Focus
Notes
Wang et al. , 2021 [ 24 ]
Not specific to manipulators
Machine learning for improving/replacing classical planning algorithms
Up to 2021
McMahon et al. , 2022 [ 18 ]
Not specific to manipulators
Machine learning for improving classical sampling-based planning algorithms
Not covering other planning algorithms
Noroozi et al. , 2023 [ 23 ]
Not specific to manipulators
All types of planning algorithms (including both classical and learning-based)
Not specific to deep learning
Tamizi et al. , 2023 [ 15 ]
Robotic manipulators
All types of planning algorithms (classical, learning-based)
Not specific to deep learning
Carvalho et al. , 2025 [ 32 ]
Not specific to manipulators
Data-driven planning algorithms
Not specific to robotic manipulators
Ours
Robotic manipulators
Focused on deep learning for improving/replacing classical planning algorithms
Since 2018
TABLE I: Overview of existing survey papers on robotic motion planning, their scope, focus, and limitations compared to this paper.
Fig. 3: An example of a robotic manipulator’s autonomy stack.
Term
Definition
Configuration space
An N-dimensional space spanned by robotic manipulator joint values, where N is the number of DOF.
Workspace
The 3-dimensional space in which the robotic manipulator’s end-effector operates.
Planning constraints
Include geometric constraints; kinematic constraints; and dynamic constraints.
Global planning
Planning algorithms that first plan a path from a start to a goal configuration, and then execute it.
TABLE II: Robotic manipulator planning terminologies, and their definitions used in this manuscript.
Symbol
Definition
Symbol
Definition
C
Configuration space
Cfree
Free configuration space
Cobs
Obstacle configuration space
X
Workspace
Xfree
Free workspace
Xobs
Obstacle workspace
q
Configuration
x
Workspace pose
f(.)
Forward kinematics
qinit
Initial configuration
qgoal
Goal configuration
σ
Path
TABLE III: Mathematical notations and definitions for motion planning of robotic manipulators utilized throughout this paper.
Fig. 4: An example of a 2-DOF planar manipulator: workspace vs. configuration space [ 39 ] .
Geometric constraints
Kinematics constraints
Dynamic constraints
Velocity evolution
Path planning
✓
✗
✗
✗
Motion planning
✓
✓
✓
✗
Trajectory planning
✓
✓
✓
✓
TABLE IV: Three main approaches to solve the planning problem for robotic manipulators, their distinguishing characteristics.
∙ Struggle to scale up to high-dimensional configuration spaces. ∙ Non-smooth paths ∙ Low convergence rate ∙ Hardly applicable to dynamic environments ∙ Inefficient sampling primitive
TABLE V: The two main categories of sampling-based planning algorithms for motion planning of robotic manipulators, their advanced variants, algorithmic primitives, and limitations.
Fig. 5: Algorithmic primitives (I. Sampling, II. Steering, and III. Collision Checking) of sampling-based planning algorithms.
∙ Refining an initial path subject to kinematic and task-specific constraints.
∙ Getting stuck in local minima. ∙ Computationally complex. ∙ Hand-crafted cost functions. ∙ Requires collision gradient. ∙ Dependent on the initial guess
TABLE VI: The two main categories of trajectory optimization planning algorithms for robotic manipulators, their constituent algorithms, characteristics, and limitations.
Fig. 6: An abstract illustration demonstrating how global trajectory optimization techniques rely on the initial trajectory to warm-start the optimization process. This highlights that even slightly different initializations can lead to distinct final trajectories.
TABLE VII: Schematic of basic Deep learning frameworks, their characteristics, and their potential for improving various components of classical planning algorithms for robotic manipulators.
TABLE VIII: Schematic of various deep generative models, their characteristics, and their applications in improving various components of classical planning algorithms for robotic manipulators. The flow matching figure is adopted from [ 199 ] .
Fig. 9: A comparison of neural motion planners with benchmark planners across various planning complexities for out-of-distribution planning scenarios. (a) Planning metrics were used to quantify the performance of motion planners. (b) Environment complexity: SIMPNet [ 247 ] performance compared with benchmark planners (Bi-RRT [ 50 ] , IRRT* [ 61 ] , MPNet [ 20 ] , KG-Planner [ 248 ] ) across different scenarios. (c) Degree of freedom: VQ-MPT [ 3 ] performance compared with benchmark planners (RRT*, IRRT* [ 61 ] , BIT* [ 65 ] , MPNet [ 20 ] ) across different scenarios. (d) Obstacle type: MPD [ 19 ] performance compared with benchmark planners (Bi-RRT [ 50 ] , GPMP [ 91 ] ) across different scenarios.
Paper
Benchmarks & Metrics
Contributions
T [s]↓
C ↓
S [%]↑
Pandy et al. [ 240 ]
95ms
PL-W : 25.2
73
∙ A planning-specific cost (loss) function. ∙ MLPs plus highway layers [ 241 ] as neural motion planner.
100 ms (RRT*)
PL-W : 16.98 (RRT*)
1.45 (RRT*)
Ota et al. [ 242 ]
-
-
-
∙ MLP-based neural waypoints generation for RL-based reactive planning.
P-NTFields [ 246 ]
0.03
PL-C : 0.43
92
∙ Utilizes ResNet-style deep network for configuration encoding. ∙ Utilizes 3D CNNs for workspace encoding.
0.05 (NTFields [ 243 ] )
PL-C : 0.38 (NTFields [ 243 ] )
84 (NTFields [ 243 ] )
TABLE IX: Overview of the state-of-the-art of utilizing deep learning for end-to-end planning, including the planning metrics reported relative to evaluated benchmark methods, and each approach’s primary contribution to robotic manipulator motion planning.
TABLE X: State-of-the-art literature on utilizing various deep learning frameworks to improve various components of sampling-based motion planning algorithms for robotic manipulator motion planning.
Paper
Benchmarks & Metrics
Contributions
T [s]↓
C ↓
S [%]↑
MPNet [ 267 ]
0.81
PL-C : 6.98
78.6
∙ MLP framework as informed sampler. ∙ Dropout [ 269 ] for encoding planning multi-modality.
9.2 (BIT*)
10.78 (BIT*)
56 (BIT*)
PCCNet [ 270 ]
0.1
PL-C : 5.88
90
∙ MLP-based informed sampler and collision-checker for bin-picking tasks.
0.38 (MPNet [ 267 ] )
7.3 (MPNet [ 267 ] )
94.4 (MPNet [ 267 ] )
Hou et al. [ 281 ]
0.6
-
-
∙ MLP-based informed sampling distribution.
TABLE XI: Overview of the state-of-the-art of utilizing deep learning frameworks for informed sampling, including the planning metrics reported relative to evaluated benchmark methods, and each approach’s primary contribution to robotic manipulator motion planning.
Paper
Benchmarks & Metrics
Contributions
T [s]↓
C ↓
S [%]↑
CBF-INC [ 303 ]
345.8
#V : 162.5
76.1
∙ Utilizes a neural control barrier function for effective steering.
287.6 (RRT)
#V: 252.5 (RRT)
62.5 (RRT)
Chiang et al. [ 10 ]
-
PL - L : 90
80
∙ Deep neural network for implicit swept volume estimation for efficient steering.
-
PL - L: 100 (PRM)
40 (PRM)
TABLE XII: Overview of the state-of-the-art of deep learning for improving the steering primitive, including the planning metrics reported relative to evaluated benchmark methods, and each approach’s primary contribution to robotic manipulator motion planning.
Paper
Benchmarks & Metrics
Contributions
T [s]↓
C ↓
S [%]↑
CoMPNet [ 315 ]
4.92
-
-
∙ Utilizes MPNet’s sampler for informed sampling. ∙ Projects samples on constraint manifolds via projection operator.
54.81 (CBiRRT [ 266 ] )
-
-
CoMPNetX [ 316 ]
15.05
-
-
∙ Utilizes a deterministic neural projector to project informed samples on the constraint manifolds.
19.77 (CoMPNet [ 315 ] )
-
-
LAC-RRT [ 318 ]
5.37
#V 203.11
100
∙ Utilizes an encoder-decoder structure for configuration encoding. ∙ Equality constraints are enforced within the latent space.
TABLE XIII: Overview of the state-of-the-art of deep learning for improving constrained sampling-based planning, including the planning metrics reported relative to evaluated benchmark methods, and each approach’s primary contribution to robotic manipulator motion planning.
Paper
Benchmarks & Metrics
Contributions
T [s]↓
C ↓
S [%]↑
DJ-GOMP [ 5 ]
0.75
-
-
∙ Utilizes an MLP framework to estimate the initial trajectory for warm-starting trajectory optimization.
1.25 (TrajOpt [ 90 ] )
-
-
CNP-B [ 8 ]
8 ms
-
100
∙ Parametrizes the initial trajectory via B-spline. ∙ utilizes an MLP to predict the control points of the B-spline.
60 ms (BIT*)
-
100 (BIT*)
MPSM [ 11 ]
0.21
-
-
∙ Encodes solution manifolds with a customized VAE structure. ∙ Utilizes CHOMP [ 79 ] for fine-tuning the generated trajectories.
TABLE XIV: Overview of the state-of-the-art of deep learning for improving trajectory optimization, including the planning metrics reported relative to evaluated benchmark methods, and each approach’s primary contribution to robotic manipulator motion planning.
Paper
Benchmarks & Metrics
Contributions
T [s]↓
C ↓
S [%]↑
SLIP+ ∗ [ 346 ]
0.13
-
96
∙ Utilizes an MLP framework to determine the collision status of robotic configurations.
1.02 (PRM)
-
77 (PRM)
Krawczyk et al. [ 347 ]
0.16 ms
-
-
∙ Utilizes an MLP framework to determine the collision status of robotic configurations.
0.31 ms (FCL [ 147 ] )
-
-
CN-RRT ∗ [ 348 ]
2.0
#V: 964
97
∙ Utilizes an MLP framework to determine the separation distance between the manipulator and the surrounding environment.
TABLE XV: Overview of the state-of-the-art of deep learning for improving collision checking, including the metrics reported relative to evaluated benchmark methods, and each approach’s primary contribution to robotic manipulator motion planning.
Fig. 10: Data representation and training objectives of neural motion planners for robotic manipulators. The left column illustrates the input and output modalities (data representation) and the right column demonstrates the corresponding loss function (training objective) for training and deploying neural motion planners. Classical motion planners such as cuRobo [ 138 ] and planners from OMPL [ 360 ] , along with collision checkers from FCL [ 147 ] , are mainly used to generate oracle datasets for training neural motion planners.
TABLE XVI: A compact and concise overview of state-of-the-art literature that leverages various deep learning frameworks to improve various components of classical planning algorithms for robotic manipulator motion planning. less than 5 papers , 5∼10 papers , more than 10 papers .
Fig. 11: Possible methods to improve the generalizability of neural motion planners: (a) LLMs for end-to-end planning [ 261 ] . (b) Large-scale synthetic dataset generation [ 384 ] .
Fig. 12: Possible methods to enhance the safety of neural motion planners: (a) Constraint-aware neural motion planning [ 267 ] . (b) Safety filters [ 343 ] . (c) Digital twins [ 389 ] .
Planners
Advantages
Limitations
SBMP
∙ Probabilistic completeness
∙ Non-smooth paths
Global Optimization
∙ Smooth paths. ∙ Incorporates equality and non-equality constraints
∙ Susceptible to local minima ∙ Computationally complex
Neural Planning
∙ Fast inference ∙ Encoding high-dimensional, complex distributions ∙ Leverages past experience for new problems
∙ Data scarcity ∙ Safety and reliability
TABLE XVII: A qualitative comparison between neural motion planning and classical motion planning for robotic manipulators.
Open-loop end-to-end neural motion planners have recently been proposed to improve motion planning for robotic manipulators. These methods enable planning directly from sensor observations without relying on a privileged collision checker during motion planning. However, existing planners produce a single path for a given planning problem and cannot exploit their open-loop nature to propose multiple motion plans. To address this limitation, we introduce Flow Motion Policy, an open-loop neural motion planner that uses flow matching to generate a batch of motion plan proposals by learning a distribution over motion plans conditioned on the planning observation. At inference time, it samples multiple candidate motion plans to enable efficient best-of-N inference while avoiding iterative collision checking during planning. We benchmark the Flow Motion Policy against representative sampling-based, optimization-based and neural motion planning methods. Evaluation results demonstrate that Flow Motion Policy improves planning success and efficiency, highlighting the effectiveness of stochastic generative policies for end-to-end motion planning and best-of-N sampling. Project website: https://davoodsz.github.io/FlowMotionPolicy.github.io/
Davood Soleymanzadeh, Xiao Liang, Minghui Zheng
J. Mike Walker ’66 Department of Mechanical Engineering, Texas A&M University, College Station, TX 77843, USA · Zachry Department of Civil and Environmental Engineering, Texas A&M University, College Station, TX 77843 USA
Recent end-to-end neural motion planners generate trajectories from raw sensor observations, avoiding the privileged geometric models required by classical planners. However, collision-free planning in cluttered environments remains challenging. We present NeurRAFT, a generative planning framework based on anchor-level flow matching and clearance-aware preference tuning. Unlike prior neural planners that model dense waypoint sequences and spend capacity on redundant local details and smoothness, NeurRAFT operates on compact anchor waypoints. We train the planner using a Jacobian-weighted loss that accounts for the task-space impact of each anchor. At inference, the anchors are generated in two integration steps, followed by cubic-spline interpolation to recover a smooth, full-resolution trajectory. Since imitation learning from positive demonstrations cannot distinguish collision-free from near-collision trajectories, collision-prone behaviors persist at test time. Rather than relying on post-hoc corrections, we directly reshape the pretrained planner's distribution toward safer solutions without augmenting inference. Specifically, Direct Preference Optimization shifts probability mass toward trajectories with larger obstacle clearance, with the resulting improvement directly absorbed into the planner parameters. Experiments show substantial improvements over state-of-the-art planners, while real-world experiments demonstrate zero-shot transfer to a Franka robot under noisy and partially occluded depth observations. Video results available at https://neurraft.github.io/.
Sibo Tian, Chang Liu, Minghui Zheng +1
J. Mike Walker ’66 Department of Mechanical Engineering, Texas A&M University, College Station, TX 77843, USA
Rigid-body motion planning is fundamental to robotic transport and manipulation, where both object position and orientation determine whether a body can move through cluttered environments. This coupling makes planning chal- lenging because translation and rotation can be constrained very differently by surrounding geometry. We propose Shape- Aware Neural Time Fields, a physics-informed neural motion planning framework that learns a reusable travel-time, or cost- to-go, representation for rigid-body motion while explicitly accounting for these distinct geometric restrictions. The learned field captures local motion difficulty in translation and rotation and is paired with a sampling-based planner that balances progress toward the goal with the cost of reaching nearby poses. We evaluate the framework across 2D and 3D rigid- body planning tasks and non-prehensile manipulation through object pushing in diverse environments. Results show substantial improvements in planning success and inference efficiency over physics-informed neural planning baselines, while remaining competitive with classical sampling-based planners at significantly lower planning times. Ablation studies further validate the contributions of both the learned representation and the proposed planning strategy. Finally, closed-loop pushing experiments under execution deviations and external disturbances demonstrate that fast replanning enables reactive and adaptive task execution.
Yanming Zhou, Yuchen Liu, Ahmed H. Qureshi
Department of Computer Science, Purdue University, West Lafayette, IN 47907, USA