Overview

Zeroth-order optimization studies how to improve an objective using only its evaluations, making it useful when gradients are unavailable, expensive, or incompatible with black-box systems. Common approaches estimate directions with finite differences or random perturbations, then reduce cost and noise through variance reduction, adaptive sampling, low-dimensional subspaces, or structured updates. The methods are used in settings such as language-model fine-tuning, robotics, test-time adaptation, and optimization under privacy or communication limits, alongside theoretical work on convergence and query complexity. A central tension is that avoiding backpropagation can save memory and enable broader access, but noisy estimates and many required evaluations can make optimization slower or less reliable than first-order methods.

Latest papers

CardsList

99 papers13 papers in the last four weeks, down 7% on the four weeks before. 0.1% of all new papers.

  1. Event-triggered Implicit Perturbation for Zeroth-Order Fine-Tuning of Spiking Transformers

    Aug 21, 2026Tengteng Lei, Prabodh Katti, Rashi Dutt +5Zeroth-Order OptimizationSpiking Neural Networks

  2. ZoAQ: Adaptive Zeroth-Order Querying via Query-Reuse Coupling

    Aug 19, 2026Yangyang Feng, Yao ShuZeroth-Order OptimizationBlack-Box Optimization

  3. SubZero+: Memory-Efficient Adaptive Zeroth-Order LLM Fine-Tuning in Random Subspaces

    Aug 16, 2026Ziming Yu, Shuyao Xiao, Xingyu Zhao +6Memory-Efficient Fine-TuningZeroth-Order Optimization

  4. Curvature-Aware Zeroth-Order Optimization for Memory-Efficient Test-Time Adaptation

    Aug 12, 2026Junming Zhang, Shuyu Yin, Peilin Liu +2On-Device TrainingTest-Time Adaptation

  5. Beyond the Capability Boundary: Zeroth-Order Optimization for Self-Evolving LLM Agents

    Aug 10, 2026Bingzhen Liu, Xiaomeng Fan, Yuwei Wu +4Zeroth-Order OptimizationLLM Agent Training

  6. ZOMP: Zeroth-Order Multi-Modal Prompt Tuning for Vision-Language Models

    Aug 8, 2026Sajjad Ghiasvand, Yifan Yang, Mahnoosh Alizadeh +1VLM AdaptationZeroth-Order Optimization

  7. Noise-Aware Shrinkage for Differentially Private Zeroth-Order Fine-Tuning of Large Language Models

    Aug 4, 2026Lele Zheng, Weifeng Kong, Xinyi Zhang +3Memory-Efficient Fine-TuningPrivacy-Preserving Language Models

  8. Not the Dimension, the Norm: What Matters in Gradient-Free Weight Perturbation of Language Models

    Aug 3, 2026Taeyeong Kim, Ahhyun Kim, TaeHyeon Kim +1Zeroth-Order OptimizationLLM Fine-Tuning

  9. Overcoming the Weakest-Link Effect in LLM-Driven Program Optimization via Heterogeneous Edit Recombination

    Jul 31, 2026Jingwen Fu, Zhen Liu, Yuhan Liu +2Zeroth-Order OptimizationLarge Language Model-Guided Optimization

  10. QScheduler: Adaptive Gradient Sampling for Zeroth-Order On-Device Training on INT8 NPUs

    Jul 21, 2026Victor Felipe Domingues Do Amaral, Pierre Demaj, Erwan Libessart +3On-Device TrainingZeroth-Order Optimization

  11. SkillOpt-Lite: Better and Faster Agent Self-evolution via One Line of Vibe

    Jul 3, 2026Yifei Shen, Bo Li, Xinjie ZhangAI Coding AgentsZeroth-Order Optimization

  12. ZO-Act: Efficient Zeroth-Order Fine-Tuning via One-Shot Activation-Informed Low-Rank Subspaces

    Jul 1, 2026Xun Dong, Yibo Xu, Naigang Wang +3Zeroth-Order OptimizationLLM Fine-Tuning

  13. Finding Stationary Points by Comparisons

    Jun 25, 2026Helin Wang, Chenyi Zhang, Xiwen Tao +2Black-Box OptimizationZeroth-Order Optimization

  14. \chisao{}: A GPU-Native Parallel Optimizer for Multimodal Black-Box Functions via Convergence-Anticonvergence Oscillation

    Jun 24, 2026Ira WolfsonGPU AccelerationBlack-Box Optimization

  15. Zero-order Parameter-free Optimization for LMO-based Methods: Novel Approach for Efficient Fine-tuning

    Jun 12, 2026Dmitriy Bystrov, Daniil Medyakov, Dmitry Bylinkin +1Gradient DescentZeroth-Order Optimization

  16. Dominant-Layer ZO: A Single Layer Dominates Zeroth-Order Fine-Tuning of LLMs

    Jun 3, 2026Wanhao Yu, Ziyan Wang, Zheng Wang +7Fine-TuningMemory-Efficient Fine-Tuning

  17. GRZO: Group-Relative Zeroth-Order Optimization for Large Language Model Fine-Tuning

    Jun 1, 2026Liyan Tan, Yequan Zhao, Yifan Yang +3Zeroth-Order OptimizationMemory-Efficient Training

  18. Wall-Clock Complexity for Zeroth-Order Optimization with Tunable Oracle Fidelity

    May 29, 2026Alexandra Suvorikova, Igor Pavlov, Artem Vasin +4Stochastic OptimizationOracle Complexity

  19. Revisiting Zeroth-Order Hessian Approximation: A Single-Step Policy Optimization Lens

    May 29, 2026Junbin Qiu, Zhaowei Hong, Renzhe Xu +1Zeroth-Order OptimizationSecond-Order Optimization

  20. Aligned but Fragile: Enhancing LLM Safety Robustness via Zeroth-Order Optimization

    May 28, 2026Zhihao Liu, Yifan Wu, Jian Lou +3LLM AlignmentZeroth-Order Optimization

  21. Inference-Native Zeroth-Order Optimization for LLMs

    May 27, 2026Zelin Li, Caiwen DingLLM ServingZeroth-Order Optimization

  22. Convergence of Consensus-Based Particle Methods for Nonconvex Bi-Level Optimization

    May 19, 2026Yutong Chao, Xudong Sun, Konstantin Riedl +2Dynamical SystemsZeroth-Order Optimization

  23. New Insight of Variance reduce in Zero-Order Hard-Thresholding: Mitigating Gradient Error and Expansivity Contradictions

    May 18, 2026Xinzhe Yuan, William de Vazelhes, Bin Gu +1Zeroth-Order OptimizationConstrained Optimization

  24. EVA-0: Test-Time Model Evolution with Only Two Forward Passes per Sample

    May 15, 2026Guohao Chen, Shuaicheng Niu, Geng Li +4Efficient Neural Network InferenceTest-Time Adaptation

  25. Position: Zeroth-Order Optimization in Deep Learning Is Underexplored, Not Underpowered

    May 15, 2026Sijia Liu, Yicheng Lang, Soumyadeep Pal +6Deep Learning OptimizationZeroth-Order Optimization