Kullback-Leibler Divergence

Ask about this topic

Momentum

27 papers in the last four weeks, up 238% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 138

All topics
CardsList

  1. FERPO: Forward Entropy-Regularized Policy Optimization

    Oct 1, 2026Sebastian Sanokowski, Alireza Sarmadi, Majid KhadivEntropy Regularized Reinforcement LearningOffline Reinforcement Learning

    Applied and Theoretical Aspects of Robot Intelligence (ATARI) Lab, Munich Institute of Robotics and Machine Intelligence (MIRMI), Technical University of Munich

  2. How Divergence Becomes Decision Flips in Compressed Language Models

    Sep 30, 2026Beatriz Almeida FelicioKullback-Leibler DivergenceLarge Language Model Decoding

    Instituto de Informática (INF) Universidade Federal de Goiás (UFG)

  3. Probabilistic Adversarial Training

    Sep 30, 2026Andi Zhang, Xingyu Zhao, Siddartha KhastgirAdversarial TrainingKullback-Leibler Divergence

    WMG, University of Warwick · Wuhan University

  4. Learning Beyond Full Imitation: Task-Preserving Knowledge Distillation

    Sep 30, 2026Qianfeng Yuan, Wenbing TaoKnowledge DistillationKullback-Leibler Divergence

    Huazhong University of Science and Technology

  5. ReTaCo: Residual-Target Control for On-Policy Distillation

    Sep 30, 2026Zixiang Ni, Zhuo Hu, Renjie Cao +8Efficient On-Policy DistillationOn-Policy

    Xi’an Jiaotong University · Zhejiang University · Georgia Institute of Technology +6

  6. Understanding Off- vs On-Policy Distillation: A Tale of Distinct Training Objectives

    Sep 29, 2026Qiwei Di, Xuheng Li, Kaixuan Ji +3Kullback-Leibler DivergenceFeedback

    Department of Computer Science, University of California, Los Angeles, CA 90095, USA

  7. Unifying Distributional Training for One-Step Visual Generation

    Sep 28, 2026Chi Zhang, Shi Haoyang, Yueyi Liu +9Generative Flow NetworksVisual Generation

    Tsinghua University · Fudan University · Xi’an Jiaotong University +4

  8. An RL View of OPD: Least Square Policy Distillation for Sample-Efficient LLM Reasoning

    Sep 28, 2026Shangzhe Li, Yuxiao Yang, Tianrun Yu +4LLM Reasoning StrategiesOffline Reinforcement Learning

    University of North Carolina at Chapel Hill · Brigham Young University · NVIDIA

  9. On-Policy or Off-Policy Learning? A Systematic Study of Distillation Dynamics

    Sep 28, 2026Julianna Piskorz, Antonin Berthon, Mihaela van der SchaarOff-Policy LearningOn-Policy

    University of Cambridge Cambridge, UK

  10. Unbiased Top-kk Estimation for On-Policy Distillation

    Sep 28, 2026Linjian Meng, Siyuan Gan, YuHan Li +5Efficient On-Policy DistillationKullback-Leibler Divergence

    Shanghai Artificial Intelligence Laboratory, Shanghai, China · State Key Laboratory of Novel Software Technology, Nanjing University, Nanjing, China

  11. Do We Really Need KL Divergence for On-Policy Distillation of Large Language Models?

    Sep 27, 2026Wenze Lin, Jiyuan Long, Jiale Zhao +11Kullback-Leibler DivergenceResidual Distillation

    LeapLab, Tsinghua University · Qiuzhen College, Tsinghua University · Beihang University +3

  12. Non-Adaptive Learning of Sparse Erdős--Rényi Graphs via Affine Splitting

    Sep 27, 2026Hoang TaO(\Bar{K}\Log N)$Inhomogeneous Random Graphs

    Department of Computer Science, Hanoi University of Science and Technology, Vietnam

  13. The cost of useful natural gradient updates

    Sep 27, 2026Subhransu S. Bhattacharjee, Dylan Campbell, Rahul ShomeFisher Information MatrixGradient

    School of Computing, The Australian National University

  14. Sharp training-conditional coverage for conformal prediction under covariate shift

    Sep 27, 2026Mehrdad PournaderiOnline Conformal PredictionCovariate Shift

    Mofid Securities

  15. Minimally Invasive Steering of Language Models

    Sep 24, 2026Taha Entesari, Jingyu Zhang, Daniel Khashabi +1SteeringFrozen Language Model

    Johns Hopkins University

  16. Not Every Token Is Worth Distilling: Selective Supervision for Direct-OPD

    Sep 24, 2026Yibo Zhao, Zixuan Yang, Yunshi Lan +1Efficient On-Policy DistillationProcess-Level Supervision

    School of Data Science and Engineering East China Normal University · Hugging Face

  17. Marginal Log-Likelihood Increments under Dirichlet-Smoothed Markov Estimation

    Sep 22, 2026Levin David SchwabKullback-Leibler DivergenceAgentic Workflow Design

  18. Variational objectives for amortized Bayesian inference in inverse problems: The role of posterior conditioning

    Sep 21, 2026Abhishek Srivastava, Arijit Hazra, Rajesh DubbakuVariationalBayesian Inference

    Department of Mechanical Engineering, Indian Institute of Technology Palakkad · Department of Mathematics, Indian Institute of Technology Palakkad

  19. Beyond Token-Local Imitation: Reward-Compatible Temporal Credit Assignment for On-Policy Distillation

    Sep 15, 2026Shiqi Liu, Zeyu He, Letian Tao +9Credit AssignmentKullback-Leibler Divergence

    School of Vehicle and Mobility & College of AI, Tsinghua University · 2Didi Voyager Labs, DiDi Autonomous Driving

  20. Local Robustness Quantification for Naive Bayes Classifiers and Generative Forests: a General Approach

    Sep 11, 2026Adrián Detavernier, Jasper De BockRobustness VerificationRandom Forest

    Foundations Lab for imprecise probabilities Ghent University Ghent, Belgium

  21. How Wrong Can a Good Predictor Be? Diverging Updates with Vanishing Predictive KL

    Sep 11, 2026Qifu Wen, Shuaijun Liu, Zihan Zhou +2Posterior Predictive DistributionKullback-Leibler Divergence

    Boston University, Boston, MA, USA · Shanghai Jiao Tong University, Shanghai, China · The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China

  22. Likelihood-free inference with nuisance parameters through normalizing flows

    Sep 9, 2026Phil AsshetonNormalizing FlowsSummary Statistics

    statsadvice.com, Berlin, Germany.

  23. Dynamical Non-compensatory Multidimensional IRT Model Using Variational Approximation

    Sep 9, 2026Hiroshi Tamano, Daichi MochihashiItem Response TheoryKullback-Leibler Divergence

    the graduate university for advanced studies, sokendai · the institute of statistical mathematics

  24. FlowCPO: A Unified Divergence View of Preference Alignment for Flow Models

    Sep 9, 2026Yansen Han, Shengyi Liao, Peng Sun +4Flow ModelsPreference Alignment

    Westlake University · Zhejiang University · Kling Team, Kuaishou Technology

  25. Exact Degeneracy Under Balanced k-Shot Sampling:Consequences for Small-Sample Discriminant Analysis on LLM Embeddings

    Sep 9, 2026Lingxiao QuDiscriminative ModelsPrincipal Component Analysis

    The University of Aizu, Aizuwakamatsu, Fukushima, Japan

  26. Mode Coverage in Normalizing Flow Boltzmann Generators via Log-Ratio Variation

    Sep 8, 2026Qi Feng, Rongjie Lai, Di Qi +1Normalizing FlowsKullback-Leibler Divergence

    Department of Mathematics, Florida State University, Tallahassee, FL 32306 · Department of Mathematics, Purdue University, West Lafayette, IN 47907

  27. Mask Forcing: Improving Autoregressive Video Diffusion Distillation via Dual-Noise Masking Rollout

    Sep 8, 2026Zhuoran Zhao, Shengju Qian, Tongtong Liang +7Autoregressive Video Diffusion ModelsVideo Diffusion Models

    1HKUST(GZ) · 2HKUST · 3LIGHTSPEED +3

  28. Knowledge Distillation During Mid-Training Favors Reasoning over Factual Recall

    Sep 1, 2026Jacqueline He, Howard Yen, Shuyue Stella Li +9Knowledge DistillationTeacher

    Meta AI · University of Washington · Princeton University

  29. Language-Statistical Analysis of Neural Audio Codec Tokens Across Architectures, Corpora, and Noise Conditions

    Aug 31, 2026Joonyong Park, Shinnosuke Takamichi, David M. Chan +3Neural Audio CodecsResidual Vector Quantization

    Grad. School of Information Science and Technology, The University of Tokyo, Tokyo, Japan · Faculty of Science and Technology, Keio University, Tokyo, Japan · Berkeley Artificial Intelligence Research Lab (BAIR), University of California, Berkeley, CA, USA

  30. D3^3-MOPD: Dynamic Domain ScheDuling for Efficient Multi-Teacher Distillation

    Aug 25, 2026Zechen Sun, Zhiwei Zhang, Fei Zhao +7Aware Heterogeneous Multi-Teacher Multimodal On-Policy DistillationTeacher-Student Distillation

    AllSpark Team

  31. WARP: Wasserstein-Aligned RAG for Population Opinions

    Aug 24, 2026Aman Singh Thakur, Aditya Agrawal, Alwarappan Nakkiran +1Top-KPopulations

  32. SoftWater: Class-Aware Rate Allocation for Softmax Quantization

    Aug 12, 2026Joao V. Cavalcanti, Ashia C. WilsonGumbel-Softmax RelaxationKullback-Leibler Divergence

    Department of Electrical Engineering and Computer Science · Massachusetts Institute of Technology · Cambridge, MA 02139, USA

  33. A Local Sinkhorn Framework for Conditional Distribution Reconstruction of Multidimensional Random Fields

    Aug 12, 2026Mingtao Xia, Qijing ShenFused Sinkhorn-Localized SimilarityConditional Distribution

    School of Mathematics, University of Birmingham, Watson Building, Edgbaston, Birmingham, B15 2TT, United Kingdom · Department of Mathematics, University of Houston, 3551 Cullen Blvd, Houston, 77204, Texas, United States · Nuffield Department of Medicine, University of Oxford, United Kingdom, Old Road Campus, Oxford, OX3 7BN, United Kingdom

  34. PAC-Bayes Beyond Parameter Space: Behavioral Equivalence, Z-Information, and Exact Complexity Decomposition

    Aug 11, 2026Vasant G. Honavar, Satish Kumar Keshri, Neil Ashtekar +1Pac-Bayesian TheoryKullback-Leibler Divergence

    Artificial Intelligence Research Laboratory Department of Informatics and Intelligent Systems The Pennsylvania State University · Artificial Intelligence Research Laboratory Department of Computer Science and Engineering The Pennsylvania State University

  35. SR-OPSD: Self-Referenced On-Policy Self-Distillation

    Aug 10, 2026Zhuo Sun, Entong Li, Yanlong Zhao +9Unsupervised On-Policy Self-DistillationSelf-Teacher

    Shanghai University of Finance and Economics · Imperial College London · Independent Researcher +6

  36. Exact Rank-Space KL Projection for Shared-Marginal Low-Rank Factors: Application to Doubly Stochastic Clustering

    Aug 9, 2026Enliang HuLow-Rank StructureFactor

  37. Causal State-Space Model for Causal Inference: Estimating Longitudinal Individual Treatment Effects

    Aug 8, 2026Abisoye Abidakun, Mingjun Zhong, Georgios LeontidisCausal InferencesHeterogeneous Treatment Effects

    Department of Computing Science, University of Aberdeen, Aberdeen, UK · UiT The Arctic University of Norway

  38. Finite Constant Frontiers and Auditable Regret Certificates for Average-Reward Reinforcement Learning

    Aug 7, 2026Ibne Farabi Shihab, Abu Sa-Adat Mohamed Moon-Im Al Ahsan, Md Najmus SwaqeebOptimal LearnerMarkov Decision Processes

    Department of Computer Science, Iowa State University · Department of Computer Science & Engineering, BRAC University

  39. Efficient Knowledge Distillation for LLMs: Offline Top-K Logits and a Fused Chunked KL Loss

    Aug 4, 2026Bakbergen Ryskulov, Iker García-Ferrero, David Montero +5Knowledge DistillationKullback-Leibler Divergence

    Multiverse Computing

  40. Flux-OPD: On-Policy Distillation with Evolving Contexts

    Jul 30, 2026Yuran Wang, Zekun Wang, Bohan Zeng +10Self-TeacherUni-Opd

    Peking University · Kling Team · Tsinghua University +3

  41. Trajectory-Regularized Stochastic Optimal Control via KL Divergence

    Jul 24, 2026Mintae Kim, Koushil SreenathStochastic Optimal ControlStochastic Optimization

    Hybrid Robotics, UC Berkeley, CA 94720, USA

  42. Total Variation Distance Estimation in Autoregressive Models

    Jul 21, 2026Eric Price, Kevin Tian, Zhiyang Xun +1Autoregressive Language ModelsKullback-Leibler Divergence

    University of Texas at Austin

  43. Distilled Reinforcement Learning for LLM Post-training

    Jul 19, 2026Chen Wang, Zhaochun Li, Jionghao Bai +4Post-TrainingOffline Reinforcement Learning

    College of Elite Engineers, Nankai University · Zhongguancun Academy · Beijing Institute of Technology +4

  44. Approximate Relative Entropy Constraints for Nonlinear Covariance Steering Under Distribution Ambiguity

    Jul 18, 2026Trevor N. Wolf, Jay W. McMahonIntent-Aligned Autonomous Spacecraft GuidanceCovariance

    *Postdoctoral Associate, Department of Aerospace Engineering Sciences, University of Colorado Boulder, AAS Member. · †Associate Professor, University of Colorado Boulder, Department of Aerospace Engineering Sciences, and AAS Fellow.

  45. Feedback Attribution and Representation Geometry: Metrics for Comparing Individual and Shared Rewards in MARL

    Jul 17, 2026Tasha Pais, Richard HigginsMulti-Agent Reinforcement LearningReward Signal

    Softmax, San Francisco, CA, USA.

  46. An Efficient Newton Algorithm for Nonnegative Matrix Factorization with the Kullback-Leibler Divergence

    Jul 15, 2026Damien Lesens, Jérémy E. Cohen, Bora UçarNonnegative Matrix FactorizationKullback-Leibler Divergence

    ENS de Lyon · Univ. Lyon, INSA-Lyon, UCBL, UJM, CNRS, Inserm, CREATIS UMR 5220, U1294, F-69621, Lyon, France. · CNRS and LIP (UMR5668, Université de Lyon - ENS de Lyon - UCBL - CNRS - Inria), 46 allée d'Italie, Lyon, France.

  47. Can a Language Model Learn Facts Continually in Its Weights?

    Jul 13, 2026Charles O'Neill, Max Kirkby, Jonathon Liu +1Large Language Model MemoryFacts

    Baseten

  48. A Convex Approximation Framework for Neural Likelihood-Based Bayesian Inverse Problems

    Jul 7, 2026Fabian Schneider, Tapio Helin, Leila TaghizadehBayesian Inverse ProblemsNeural Approximations

    Institute of Analysis and Scientific Computing, TU Wien, Vienna, Austria · School of Engineering Sciences, LUT University, Lappeenranta, Finland

  49. Lean-Quantum: Toward AI-Assisted Formalization of Quantum Information

    Jul 6, 2026Kazumi Kasaura, Kei Tsukamoto, Kento Mori +6FormalizationKullback-Leibler Divergence

    1OMRON SINIC X Corporation · 2RIKEN · 3The University of Tokyo +3

  50. What Does a Discrete Diffusion Model Learn?

    Jul 6, 2026Rodrigo Casado Noguerales, Bernhard Schölkopf, Thomas Hofmann +1Diffusion ModelsKullback-Leibler Divergence

    1ETH Zurich · 2Max Planck Institute for Intelligent Systems, Tübingen · 3ELLIS Institute Tübingen

  51. Tightening the Score Matching Gap for Diffusion Models

    Jul 5, 2026Benjamin Dupuis, Tyler Farghly, Maxime Haddouche +2Score-Based Diffusion ModelKullback-Leibler Divergence

    INRIA - CNRS - D´epartement d’Informatique de l’Ecole Normale Sup´erieure - PSL Research University, France · Ecole Polytechnique - CMAP, IP Paris, Palaiseau, France

  52. CouCE: A Unified Causal Framework for Debiased Deep Metric Learning

    Jun 29, 2026Xin Yuan, Zhenyang Niu, Meiqi Wan +3Uncoupled LearningAdversarial Training

    Wuhan University of Science and Technology Wuhan, China · Harbin Institute of Technology Harbin, China

  53. ARKD: Adaptive Reinforcement Learning-Guided Bidirectional KL Divergence Distillation for Text Generation

    Jun 29, 2026Zilong Liu, Xuewen Zhang, Jinrui Xing +3Probe-Logit DistillationKullback-Leibler Divergence

    Li Auto, Beijing, China · School of Software and Microelectronics, Peking University, China

  54. KbSD: Knowledge Boundary aware Self-Distillation for Behavioral Calibration in Agentic Search

    Jun 29, 2026Tao Feng, Xinke Jiang, Chao WuAgentic Reinforcement LearningAgentic Search

    Zhejiang University, Hangzhou, China · Peking University, Beijing, China

  55. A Diagnostic Framework and Multi-Evaluator Audit of Evaluator-Driven Preference Dynamics in Self-Adapting LLM Agents

    Jun 29, 2026Liu ZewenSelf-EvaluationLarge Language Model Agents

    Qilu Institute of Technology, School of Software Engineering · Tai’an, Shandong, China

  56. Beyond Global Divergences: A Local-Mass Perspective on Bayesian Inference

    Jun 25, 2026Hanli Xu, Fengxiang He, Sarat MokaBayesian InferenceKullback-Leibler Divergence

    UNSW Sydney · University of Edinburgh

  57. AsyncOPD: How Stale Can On-Policy Distillation Be?

    Jun 23, 2026Wonjun Kang, Kevin Galim, Seunghyuk Oh +9Efficient On-Policy DistillationAsynchronous Execution

    FuriosaAI · Ajou University · UC Berkeley +3

  58. KLip-PPO: A per-sample KL perspective on PPO-Clip

    Jun 22, 2026Riccardo Colletti, Robin HolzingerProximal Policy OptimizationKullback-Leibler Divergence

    University of California, Berkeley

  59. StreamKL: Fast and Memory-Efficient KL Divergence for Boosting Attention Distillation

    Jun 18, 2026Guangda Liu, Yiquan Wang, Chengwei Li +6Dataset DistillationKullback-Leibler Divergence

    Shanghai Jiao Tong University · Huawei · Fudan University

  60. QMaxCal: Path-Space Regularization for Open Quantum Control via Girsanov's Theorem

    Jun 18, 2026Merijn Moody, Zier Mensch, Miranda C. N. Cheng +2Kullback-Leibler RegularizationKullback-Leibler Divergence

    Dutch Institute for Emergent Phenomena, University of Amsterdam, Netherlands · Institute of Physics, University of Amsterdam, Netherlands · Korteweg-de Vries Institute for Mathematics, University of Amsterdam, Netherlands +4