Gradient-Free Optimization

Latest papers

14 papers1 paper in the last four weeks, with none the four weeks before. 0.0% of all new papers.

Approaching Backpropagation Accuracy with Weight Perturbation on MNIST
Oct 15, 2026Patrick InoueGradient-Free Optimization Neural Network Optimization
The MNIST experiments originate from the author’s master’s thesis, submitted under the name Patrick Stricker at Albstadt-Sigmaringen University in March 2023 [4].
Initial condition recovery in nonlinear damped viscous photoacoustic tomography using a convolutional neural network-guided gradient-free optimization framework
Oct 1, 2026Madhu Gupta, Anwesa Dey, Prapti Tala +1Medical Imaging Gradient-Free Optimization
Department of Mathematics, Indian Institute of Technology, Gandhinagar, Gujarat, India. · Department of Mathematics, University of Utah, Salt Lake City, Utah, UT, USA. · Department of Mathematics, The University of Texas at Arlington, Arlington, TX, USA.
GRACE: Gradient-Free Robot Action Generation via Combined Diffusion-MPPI Posterior Mean Estimation
Jul 22, 2026Leesai Park, Jiho HOng, Sanghyun KimGradient-Free Optimization Diffusion Guidance
School of Mechanical Engineering, Kyung Hee University, Yongin, Republic of Korea. · Advanced Institute of Convergence Technology (AICT), Suwon, Republic of Korea.
Beyond Backpropagation: Monte Carlo Method Can Train Deep Neural Networks
Jul 9, 2026Hong ZhaoDeep Learning Optimization Gradient-Free Optimization
Department of Physics, Xiamen University, Xiamen 361005, China · Lanzhou Center for Theoretical Physics, Lanzhou University, Lanzhou 730000, China
Gradient-Free Warm-Start Library Recovery: an Amortized-Regret Separation
Jun 19, 2026Jianwei LouGradient-Free Optimization Continual Learning
RailMind Systems, Neuss, Germany
GRZO: Group-Relative Zeroth-Order Optimization for Large Language Model Fine-Tuning
Jun 1, 2026Liyan Tan, Yequan Zhao, Yifan Yang +3Zeroth-Order Optimization Memory-Efficient Training
University of California, Santa Barbara
DAGGER: Gradient-Free Construction of Transiently Amplifying Networks under Hard Connectivity Constraints
May 31, 2026James C. FergusonDynamical Systems Gradient-Free Optimization
The African Institute for Mathematical Sciences (Muizenberg, South Africa) · the Institute of Science and Technology Austria (Klosterneuburg, Austria)
PatchWorld: Gradient-Free Optimization of Executable World Models
May 29, 2026Jiaxin Bai, Yue Guo, Yifei Dong +13Model-Based Planning Gradient-Free Optimization
1Hong Kong Baptist University · 2Independent Researcher · 3HKUST +4
Gradient-Free Training of Spiking Neural Networks via Low-Rank Evolution Strategies
May 14, 2026Dhruv Patankar, Sachit Ramesha GowdaGradient-Free Optimization Evolutionary Optimization
Shunya Research
Gradient-Free Noise Optimization for Reward Alignment in Generative Models
May 12, 2026Jeongsol Kim, Hongeun Kim, Jian Wang +1Diffusion Model Alignment Stochastic Optimal Control
KAIST AI1 · Snap Inc.2
DC-Ada: Reward-Only Decentralized Sensor Adaptation for Heterogeneous Multi-Robot Teams
Apr 5, 2026Saad AlqithamiGradient-Free Optimization Robot Policy Adaptation
Department of Computer Science, Al-Baha University, Albaha 65779 Saudi Arabia
Matching Accuracy, Different Geometry: Evolution Strategies vs GRPO in LLM Post-Training
Apr 2, 2026William Hoy, Binxu Wang, Xu PanEvolution Strategies RL Post-Training
Department of Industrial and Systems Engineering, University of Miami · Kempner Institute, Harvard University · Harvard University
From Gradients to Riccati Geometry: Kalman World Models for Single-Pass Learning
Mar 13, 2026Andrew KirulutaGradient-Free Optimization State Space Models
UC Berkeley School of Information
DeepAFL: Deep Analytic Federated Learning
Feb 28, 2026Jianheng Tang, Yajiang Huang, Kejia Fan +8Gradient-Free Optimization Federated Learning
Peking University, PRC · Key Lab of High Confidence Software Technologies (PKU), Ministry of Education, PRC · Central South University, PRC +4

Oct 15, 2026J/K move · Enter open · S save

Approaching Backpropagation Accuracy with Weight Perturbation on MNIST

Patrick Inoue

The MNIST experiments originate from the author’s master’s thesis, submitted under the name Patrick Stricker at Albstadt-Sigmaringen University in March 2023 [4].

Gradient-Free Optimization Neural Network Optimization Image Classification

Open paper AskPDF arXiv

Weight perturbation (WP) estimates learning directions from random parameter perturbations and a scalar change in loss, without backpropagating derivatives. We examine how batch size, training duration, and estimator stabilization affect its attainable accuracy in fully connected MNIST networks. In the shallow configuration, large-batch training reaches approximately 97% test accuracy, compared with an early backpropagation (BP) reference near 97.5--98%. The same comparison exposes a substantial efficiency gap: under a ten-epoch budget, shallow WP reaches 90.2%, whereas the reported BP reference reaches 97.4%. Epoch-indexed curves show diminishing returns from increasing batch size beyond 4,000. Learning-rate reduction and averaging independent perturbation-based updates also stabilize deeper networks, although their recorded results do not establish BP-level accuracy. We distinguish these empirical findings from a finite-horizon approximation argument: under stated regularity conditions, population WP can approximate a fixed gradient-descent trajectory arbitrarily closely at any fixed finite depth. Positive classification margins then suffice to preserve the reference predictions on a finite test set with arbitrarily high probability. Together, the results identify near-BP accuracy as a configuration- and budget-dependent possibility, rather than evidence of equal efficiency or an unconditional accuracy guarantee across depth.