Importance Sampling

Follow this topic Ask about this topic

Importance sampling studies how to estimate expectations, probabilities, or gradients under a target distribution by drawing samples from another distribution and correcting them with weights. Adaptive and defensive proposals, sequential Monte Carlo, and self-normalized or tempered weights are used in settings ranging from rare-event estimation and Bayesian inference to off-policy evaluation and language-model policy optimization. Applications include estimating rare failures, reweighting data under distribution shift, and making learning or inference more computationally efficient. A central tension is that better correction can reduce bias but high-variance weights destabilize estimates, so current work develops clipping, calibration, effective-sample-size diagnostics, and guarantees for when reweighting is reliable.

Latest papers

74 papers22 papers in the last four weeks, up 340% on the four weeks before. 0.1% of all new papers.

Beyond Importance Sampling: Rejection-Gated Policy Optimization
Apr 16, 2026Ziwu Sun, Zhen Gao, Jiyong Zhang +1Importance Sampling Policy Optimization
Adaptive Learning via Off-Model Training and Importance Sampling for Fully Non-Markovian Optimal Stochastic Control. Complete version
Apr 14, 2026Dorival Leão, Alberto Ohashi, Simone Scotti +1Stochastic Optimal Control Adaptive Learning
GRASS: Gradient-based Adaptive Layer-wise Importance Sampling for Memory-efficient Large Language Model Fine-tuning
Apr 9, 2026Kaiyuan Tian, Yu Tang, Gongqingjian Jiang +5LLM Fine-Tuning Fine-Tuning
National University of Defense Technology · Information Support Force Engineering University
Structured SIR: Efficient and Expressive Importance-Weighted Inference for High-Dimensional Image Registration
Mar 18, 2026Ivor J. A. Simpson, Neill D. F. CampbellUncertainty Quantification Importance Sampling
Department of Informatics University of Sussex, UK · Department of Computer Science University College London, UK · Department of Computer Science University of Bath, UK
Excited Pfaffians: Generalized Neural Wave Functions Across Structure and State
Mar 15, 2026Nicholas Gao, Till Grutschus, Frank Noé +1Variational Monte Carlo Neural Quantum States
Department of Computer Science & Munich Data Science Institute, Technical University of Munich · CuspAI · Free University of Berlin +2
Improving Search Agent with One Line of Code
Mar 10, 2026Jian Li, Dongsheng Chen, Zhenhua Xu +5RL for Tool Use Group Relative Policy Optimization
Nanjing University · Tencent YoutuLab
Data-Aware Random Feature Kernel for Transformers
Mar 4, 2026Amirhossein Farzam, Hossein Mobahi, Nolan Andrew Miller +1Transformer Attention Attention Mechanisms
Google DeepMind
Multimodal-Prior-Guided Importance Sampling for Hierarchical Gaussian Splatting in Sparse-View Novel View Synthesis
Mar 3, 2026Kaiqiang Xiong, Zhanke Wang, Ronggang WangNovel View Synthesis 3D Gaussian Splatting
Guangdong Provincial Key Laboratory of Ultra High Definition Immersive Media Technology, Shenzhen Graduate School, Peking University, China · Peng Cheng Laboratory, China · Migu Culture Technology Co., Ltd., China
Partition Function Estimation under Bounded f-Divergence
Feb 26, 2026Adam Block, Abhishek ShettyImportance Sampling
Department of Computer Science, Columbia University · Department of Electrical Engineering, Columbia University · MIT
Soft Sequence Policy Optimization
Feb 22, 2026Svetlana Glazyrina, Maksim Kryzhanovskiy, Roman IschenkoLLM Alignment RL Post-Training
Lomonosov Moscow State University, Moscow, Russia · Institute for Artificial Intelligence, Lomonosov Moscow State University, Moscow, Russia
Additive Control Variates Dominate Self-Normalisation in Off-Policy Evaluation
Feb 16, 2026Olivier Jeunen, Shashank GuptaOff-Policy Evaluation Variance Reduction
aampe Antwerp, Belgium · Microsoft AI Bangalore, India
Stabilizing Off-Policy Training for Long-Horizon LLM Agent via Turn-Level Importance Sampling and Clipping-Triggered Normalization
Nov 25, 2025Chenliang Li, Adel Elmahdy, Alex Boyd +7RL for Language Model Reasoning Importance Sampling
Texas A&M University · GE HealthCare · Independent Researcher +1
Score-Regularized Joint Sampling with Importance Weights for Flow Matching
Nov 21, 2025Xinshuang Liu, Runfa Blark Li, Shaoxiu Wei +1Flow Matching Importance Sampling
University of California, San Diego · San Diego, CA, USA
Interacting Annealed Importance Sampling via Population-Based MCMC Kernels
Jan 28, 2024Haoxuan Chen, Lexing YingImportance Sampling Markov Chain Monte Carlo
Institute for Computational and Mathematical Engineering (ICME), Stanford University, Stanford, CA, 94305, USA · Department of Mathematics and Institute for Computational and Mathematical Engineering (ICME), Stanford University, Stanford, CA, 94305, USA

Apr 16, 2026J/K move · Enter open · S save

Beyond Importance Sampling: Rejection-Gated Policy Optimization

Ziwu Sun, Zhen Gao, Jiyong Zhang, Jiaheng Li

Importance Sampling Policy Optimization Policy Gradient Methods RL Fine-Tuning Policy Learning Importance Sampling for RL

Open paper AskPDF arXiv

We propose a new perspective on policy optimization: rather than reweighting all samples by their importance ratios, an optimizer should select which samples are trustworthy enough to drive a policy update. Building on this view, we introduce Rejection-Gated Policy Optimization (RGPO), which replaces the importance sampling ratio r_theta = pi_theta / pi_old with a smooth, differentiable acceptance gate alpha_theta(s, a) = g(r_theta(s, a)) in the range [0, 1]. Unlike prior work that applies rejection sampling as a data-level heuristic before training, RGPO elevates rejection to an optimization principle: the gate participates directly in gradient computation and is implicitly updated alongside the policy. RGPO provides a unified framework: the policy gradients of TRPO, PPO, and REINFORCE all correspond to specific choices of the effective gradient weight w(r) = g'(r) * r. We prove that RGPO guarantees finite, bounded gradient variance even when importance sampling ratios are heavy-tailed (where IS variance diverges). We further show that RGPO incurs only a bounded, controllable bias and provides an approximate monotonic policy improvement guarantee analogous to TRPO. RGPO matches PPO in computational cost, requires no second-order optimization, and extends naturally to RLHF-style preference alignment. In online preference fine-tuning of Qwen2.5-1.5B-Instruct on Anthropic HH-RLHF (n = 3 seeds), RGPO uses a dual-ratio gate that anchors learning to both the previous policy and the reference model, achieving a Pareto-dominant outcome: the highest reward among online RL methods (+14.8% vs. PPO-RLHF) and the lowest KL divergence to the reference model (-16.0% vs. PPO-RLHF, -53.1% vs. GRPO).