Exponential Moving Average

Follow this topic Ask about this topic

Momentum

2 papers in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 21

All topics

GLaS-JEPA: Gaussian-Regularized Speech SSL without Engineered Prediction Targets
Sep 29, 2026Gaspard Botté, Séverin Baroudi, Samir Sadok +5Discrete Speech Representations Speech Encoder
Concordia University · Mila – Qu´ebec AI Institute · Univ Toulon, Aix Marseille Univ, CNRS, LIS +4
EMA: Elastic and Performance Transparent Memory Across GPUs
Sep 22, 2026Yi Xu, Tian Xia, Ion StoicaGraphics Processing Unit Resources Peak Memory
UC Berkeley
DESA-TTA: Dynamic EMA and Source Anchoring for Test-Time Adaptation
Sep 1, 2026Atif Belal, Lilian Hollard, Marco Pedersoli +1Stable Test-Time Adaptation Test-Time Adaptation
LIVIA, ILLS, Dept. of Systems Engineering, ETS Montréal, Canada
Constant Individual Regret in General Games
Aug 31, 2026Mingyang Liu, Gabriele Farina, Asuman OzdaglarImperfect-Information Games Plan Regret
LIDS, EECS, Massachusetts Institute of Technology
SR-JEPA: Learning Predictive Latent State in 3D Scenes
Aug 6, 2026Zihan Zhou, Qifu Wen, Xi ZengNext-State Prediction Latent Variable
Boston University, Boston, MA, USA · Shanghai Jiao Tong University, Shanghai, China
Stable Global Weighting of Flow Mixtures using Simplex Exponential Moving Average
Jul 4, 2026Benjamin Wiriyapong, Oktay Karakus, Can Eyupoglu +1Normalizing Flows Variational Inference
School of Computer Science and Informatics, Cardiff University, Abacws, Cardiff, CF24 4AG, U.K. · Department of Computer Engineering, Turkish Air Force Academy, National Defence University, Istanbul, Türkiye.
Heterogeneous Knowledge Distillation via Geometry Decoupling and Momentum-Aware Gradient Regulation
Jun 23, 2026Wuming Yang, Xiang Zhang, Hongmin ZhaoKnowledge Distillation Exponential Moving Average
School of Computer and Mathematics, Central South University of Forestry and Technology, Changsha, Hunan, China · College of Computer Science and Technology, and the Laboratory of Digitizing Software for Frontier Equipment, National University of Defense Technology, Changsha, Hunan, China
EPM-JEPA: Operator-Side Experience Modulation in JEPA-Family World Models
Jun 11, 2026Vedant PandyaS-Jepa Joint-Embedding Predictive Architectures
School of Artificial Intelligence and Data Engineering (SAIDE) Indian Institute of Technology Jodhpur
TT-DAC-PS: Twin-Target Deterministic Actor-Critic with Policy Smoothing for Optimal Trade Execution
Jun 7, 2026Ilia Zaznov, Atta Badii, Julian Kunkel +1Soft Actor-Critic Exponential Moving Average
Department of Computer Science University of Reading Reading, UK · Department of Computer Science/GWDG University of Göttingen Goettingen, Germany · ICMA Centre, Henley Business School University of Reading Reading, UK
CF-JEPA: Mask-free forward prediction with asymmetric encoder utilization for time-series representation learning
Jun 5, 2026Jaehoon Lee, Sunghyun SimJoint-Embedding Predictive Architectures Time Series
aGraduate School of Artificial Intelligence Convergence Engineering, Changwon National University, Changwon, South Korea · bDepartment of Artificial Intelligence Engineering, Changwon National University, Changwon, South Korea
Steady-Forcing: Balancing Spatial Persistence and Motion Continuity in Long-Horizon Nature Video Diffusion
Jun 2, 2026Matiur Rahman Minar, Seunghun Oh, GangHyeon Jeong +1Autoregressive Video Diffusion Models Optical Flow
Department of Computer Science and Engineering Sogang University Seoul, Korea · Department of Artifical Intelligence Sogang University Seoul, Korea
Automated Erythrocyte Detection and Tracking for Retinal Blood Flow Quantification in Erythrocyte-Mediated Angiography
May 31, 2026Chiao-Yi Wang, Havish S Gadde, Yi-Ting Shen +3Coronary Angiography Acute Myeloid Leukemia
Department of Bioengineering, University of Maryland, College Park, MD 20742, USA · Department of Ophthalmology and Visual Sciences, University of Maryland School of Medicine, Baltimore, MD 21201, USA · Department of Electrical and Computer Engineering, University of Maryland, College Park, MD 20742, USA
ADAPT: A Self-Calibrating Proactive Autoscaler for Container Orchestration
May 15, 2026Himanshu Singh BaghelRate Scaling Post-Deployment Adaptation
Department of Computer Engineering [J.C. Bose University of Science and Technology] Faridabad, India
EMA: Efficient Model Adaptation for Learning-based Systems
May 13, 2026Daiyang Yu, Xinyu Chen, Yihan Zhang +3Parameter-Efficient Adaptation Rapid Adaptation
University of Illinois Urbana-Champaign USA · The Hong Kong University of Science and Technology Hong Kong, China
ConRetroBert: EMA Stabilized Dual Encoders for Template-Based Single-Step Retrosynthesis
May 12, 2026Mohammad Jahid Ibna Basher, Ali Khodabandeh Yalabadi, Ivan Garibay +1Reaction Bert-Based Models
Department of Industrial Engineering University of Central Florida Orlando, FL 32816
Improving Generative Adversarial Networks with Self-Distillation
May 9, 2026Antoni Nowinowski, Krzysztof KrawiecConditional Gans Generative Adversarial Network
Poznan University of Technology
AdaTKG: Adaptive Memory for Temporal Knowledge Graph Reasoning
May 8, 2026Seunghan Lee, Jun Seo, Jaehoon Lee +7Temporal Knowledge Graphs Graph-Structured Memories
LG AI Research Seoul, South Korea
UniSD: Towards a Unified Self-Distillation Framework for Large Language Models
May 7, 2026Yiqiao Jin, Yiyang Wang, Lucheng Fu +7Self-Distillation Framework Large Language Model Adaptation
Georgia Institute of Technology · University of California, Los Angeles · Carnegie Mellon University +1
On-Orbit Real-Time Wildfire Detection Under On-Board Constraints
May 7, 2026Matthias Rötzer, Veronika Pörtge, Martin Ickerott +6Wildfire Detection Rgb-Thermal
OroraTech GmbH, St.-Martin-Str. 112, 81669 Munich, Germany · Ludwig-Maximilians-Universität München, Munich, Germany
ILDR: Geometric Early Detection of Grokking
Apr 22, 2026Shreel GolwalaGrokking Intra-Class
Virginia Tech
EMAG: Self-Rectifying Diffusion Sampling with Exponential Moving Average Guidance
Dec 19, 2025Ankit Yadav, Ta Duc Huy, Lingqiao LiuClassifier-Free Guidance Diffusion Sampling
Australian Institute for Machine Learning, Adelaide University, Australia

Sep 29, 2026, eess.ASJ/K move · Enter open · S save

GLaS-JEPA: Gaussian-Regularized Speech SSL without Engineered Prediction Targets

Gaspard Botté, Séverin Baroudi, Samir Sadok, Francesco Paissan+4

Concordia University · Mila – Qu´ebec AI Institute · Univ Toulon, Aix Marseille Univ, CNRS, LIS+4

Discrete Speech Representations Speech Encoder Self-Supervised Learning Exponential Moving Average Representation Collapse

Open paper AskPDF arXiv

Speech self-supervised learning aims to learn general-purpose representations for downstream speech tasks. However, current approaches rely on complex, carefully designed prediction targets. We challenge this necessity with GLaS-JEPA, a framework that directly predicts the current encoder's continuous representations at masked positions, without contrastive learning, discrete targets, or separate EMA target encoders. We prevent representation collapse using SIGReg representation-space regularization, eliminating the need for engineered target-generation mechanisms. Pretrained on 960 hours of LibriSpeech, our 57M-parameter model achieves a 6.89% WER on frozen-encoder SUPERB ASR and a 25.87% CER on slot filling, outperforming the best non-distilled sub-90M baselines by 43.1% and 22.0%, respectively. These results demonstrate that highly competitive speech representations can emerge from a radically simplified training recipe.

Momentum

Related topics

Latest papers 21