Overview

Trust-region optimization studies how to choose steps using a local approximation of an objective, accepting steps that improve the objective and adjusting the region when predictions are inaccurate. Methods include quadratic-model and sequential quadratic programming approaches, stochastic and derivative-free updates, and Bayesian optimization with local Gaussian-process models. These methods are used for constrained and noisy problems, expensive black-box search, and fine-tuning policies or machine-learning models; evaluations often include CUTEst problems, synthetic optimization benchmarks, and application-specific tasks. A central challenge is balancing reliable local progress and convergence guarantees against the need to scale, handle noisy evaluations, and explore beyond the current region.

Latest papers

31 papers4 papers in the last four weeks, down 33% on the four weeks before. 0.0% of all new papers.

Feb 4, 2026

QUATRO: Query-Adaptive Trust Region Policy Optimization for LLM Fine-tuning

GRPO-style reinforcement learning (RL)-based LLM fine-tuning algorithms have recently gained popularity. Relying on heuristic trust-region approximations, however, they can lead to brittle optimization behavior, as global importance-ratio clipping and group-wise normalization fail to regulate samples whose importance ratios fall outside the clipping range. We propose Query-Adaptive Trust-Region policy Optimization (QUATRO), which directly enforces trust-region constraints through a principled optimization. This yields a clear and interpretable objective that enables explicit control over policy updates and stable, entropy-controlled optimization, with a stabilizer terms arising intrinsically from the exact trust-region formulation. Empirically verified on diverse mathematical reasoning benchmarks, QUATRO shows stable training under increased policy staleness and aggressive learning rates, maintaining well-controlled entropy throughout training.