An Introduction to Zero-Order Optimization Techniques for Robotics

Armand Jordana; Jianghan Zhang; Joseph Amigo; Ludovic Righetti

cs.ROJun 27, 2025

An Introduction to Zero-Order Optimization Techniques for Robotics

Authors: Armand Jordana, Jianghan Zhang, Joseph Amigo, Ludovic Righetti

Organizations: Machines in Motion Laboratory, New York University, USA

Abstract

Zero-order optimization techniques are becoming increasingly popular in robotics due to their ability to handle non-differentiable functions and escape local minima. These advantages make them particularly useful for trajectory optimization and policy optimization. In this work, we propose a mathematical tutorial on random search. It offers a simple and unifying perspective for understanding a wide range of algorithms commonly used in robotics. Leveraging this viewpoint, we classify many trajectory optimization methods under a common framework and derive novel competitive RL algorithms.

Figures & tables

Figure 1: Randomized smoothing (RS) compared to log-sum-exp (LSE) smoothing (with

\Sigma=I

	Weights
MPPI-CMA	Exponential Average (Eq. 19 )
CMA-ES ⋆	Ordering ( Hansen and Ostermeier, 2001 )
CEM ⋆⋆	Elitist (Remark 7 )

Table 1: Variations of Algorithm 6 . ⋆ We refer to CMA-ES with no evolutionary path. ⋆⋆ Note that CEM updates the mean first.

Panel of Figure 2 — Figure 2: Cost according to the number of iterations for different test problems. MPPI outperform RS but needs a task-dependent learning-rate.

Panel of Figure 3 — Figure 3: Cost according to the number of iterations for different test problems. The CMA update allows to use the same learning rate parameters across multiple tass. The full and block-diagonal covariance update perform similarly.

Figure 4: Fraction of runs with a normalized score above a given threshold (

\tau

). Higher is better.

Algorithm	Analytical update
MPPI (Alg. 5 )	Eq. ( 23 )
MPPI-CMA (Alg. 6 )	Eq. ( 31 )
DDPG	Eq. ( 36 )
RS-DDPG (Alg. 7 )	Eq. ( 38 )
LSE-DDPG (Alg. 8 )	Eq. ( 39 )

Table 2: Summary of all the algorithm covered as well as their corresponding analytical update (i.e. the update before performing Monte Carlo approximation)

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Risk-seeking and Risk-averse transformation

Panel of Figure 6 — Figure 6: Median cost according to the number of iterations for MPPI-CMA and CEM with full covariance update. The median is taken over 6 seeds. Covariance adaptation needs two distinct learning rates: for both MPPI-CMA and CEM, $(\alpha_{m},\alpha_{\Sigma})=(1,0.1)$ outperforms a single shared rate $(1,1)$ or $(0.1,0.1)$ on both Go2Walk and HumanoidStandup.

Panel of Figure 7 — Figure 7: Median cost according to the number of iterations for different test problems. The median is taken over 6 seeds. MPPI-CMA and CEM perform the best: CEM sometimes reaches lower minima while MPPI-CMA sometimes decreases the cost faster during the first few iterations.

Figure 8: Episodic return for each environment.

Zeroth-Order Optimization

Zeroth-order optimization uses function values or comparisons, rather than gradients, to optimize objectives when differentiation is unavailable or costly.

See all 85 papers

Monte Carlo Tree Search with Tensor Factorization for Optimization Problems in Robotics
Jul 7, 2025Teng Xue, Yan Zhang, Amirreza Razmjoo +1Monte Carlo Tree Search TT Decomposition
Idiap Research Institute, Martigny, Switzerland · École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland
Pareto-Optimal Entropy-Regularized Trajectory Optimization
Oct 6, 2026Dimitrios S. Georgiou, Augustinos D. Saravanos, Evangelos A. TheodorouTrajectory Optimization Trajectory Generation
Daniel Guggenheim School of Aerospace Engineering, Georgia Institute of Technology, Atlanta, GA, USA. · Department of Aeronautics and Astronautics, Massachusetts Institute of Technology, Cambridge, MA, USA.
RoboRouter: Training-Free Policy Routing for Robotic Manipulation
Mar 9, 2026Yiteng Chen, Zhe Cao, Hongjia Ren +9Robot Policy Learning Robotic Manipulation
South China University of Technology · Nanjing University · University of New South Wales +5

Jul 7, 2025J/K move · Enter open · S save

Monte Carlo Tree Search with Tensor Factorization for Optimization Problems in Robotics

Teng Xue, Yan Zhang, Amirreza Razmjoo, Sylvain Calinon

Idiap Research Institute, Martigny, Switzerland · École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland

Monte Carlo Tree Search TT Decomposition Robotic Manipulation Robotic Optimization Robot Motion Planning

Open paper AskPDF arXiv

Many robotic tasks, such as inverse kinematics, motion planning, and contact-rich manipulation, can be formulated as optimization problems. Solving these problems requires addressing inherent nonlinear kinematics, complex contact dynamics, long-horizon correlations, and multi-modal optimization landscapes, each posing distinct challenges for state-of-the-art optimizers. While existing methods tackle these issues through problem-specific strategies, such specialization inherently limits cross-task generalization, requires heavy engineering effort in problem reformulation, and hinders multi-task autonomy. Monte Carlo Tree Search (MCTS) offers a compelling framework that generalizes across diverse robotic tasks via strategic exploration of the solution space. However, it typically suffers from combinatorial complexity when applied naively, resulting in slow convergence and excessive storage space in high-dimensional domains. To address this limitation, we propose Tensor Train Tree Search (TTTS), which leverages tensor factorization to exploit implicit correlations among different branches within the decision tree. By utilizing the resulting compact, linear-complexity representation, TTTS significantly reduces both computation and storage overhead, thereby enabling highly efficient global decision making. Experimental results across inverse kinematics, motion planning around obstacles, legged robot manipulation, multi-stage motion planning, and bimanual whole-body manipulation demonstrate the efficiency of TTTS for generalized robot optimization over a diverse set of tasks.

Abstract

Zeroth-Order Optimization

Explore similar work

Monte Carlo Tree Search with Tensor Factorization for Optimization Problems in Robotics

Pareto-Optimal Entropy-Regularized Trajectory Optimization

RoboRouter: Training-Free Policy Routing for Robotic Manipulation