Elastic Neural Networks

Follow this topic Ask about this topic

Latest papers

11 papers6 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

EDWARD-NMT: Elastic Dimensions With Adaptive Routing and Dual-consistency for Neural Machine Translation
Oct 23, 2026Edward Nguyen, Thai Tran, Kiet Anh Hoang +4Machine Translation Mixture-of-Experts Models
UNEY, Switzerland
HydraLLM: Adaptive Width Slicing for Elastic Text and Vision-Language Models
Oct 16, 2026Oscar Kemme, Janek Haberer, Ali Hojjat +1Elastic Neural Networks Efficient VLM Inference
Hamburg University of Technology (TUHH), Germany · Kiel University, Germany · UNU-INWEH, Germany
Your Transformer Is Secretly Elastic
Oct 14, 2026Walter Simoncini, Dheeraj Varghese, Gertjan J. Burghouts +1Elastic Neural Networks Transformer Pruning
University of Amsterdam · TNO, Intelligent Imaging
ShellElastic: Scalable Elastic Pretraining with Simultaneous Maximal Updates and Hyperparameter Transfer
Oct 10, 2026Can Jin, Yan Kang, Yifan Gong +5Elastic Neural Networks Transformer
Rutgers University · Adobe Research
EV-JEPA: Elastic V-JEPA for Any Device and Cross-Device Inference
Oct 9, 2026Anxhelo Diko, Jieyu Lin, Stefano Ambrogio +5Elastic Neural Networks Edge-Cloud Collaborative Inference
Meta Reality Labs Research · FAIR at Meta
Stepped MoE: Segment-Level Routing with Configurable Inference Complexity
Oct 5, 2026Arnav Kundu, Zhaoyang Xu, Bairu Hou +3On-Device Language Model Inference Dynamic Neural Networks
Apple
Elastoformer: Enabling Dynamic Adaptivity via Elastic Model Transformation
Sep 9, 2026Sudaksh Kalra, Dolly SapraEfficient Neural Network Inference Dynamic Neural Networks
University of Amsterdam The Netherlands
FEAST: Federated Shared-Space Training for Resource-Heterogeneous Clients
Aug 10, 2026Bostan Khan, Masoud DaneshtalabDynamic Neural Networks Communication-Efficient Distributed Training
Mälardalen University, Västerås, Sweden
Tevatron-Elastic: A Unified Abstraction for Training Elastic Retrievers and Rerankers
Aug 9, 2026Yu Wang, Shengyao Zhuang, Xueguang Ma +4Model Compression Dynamic Neural Networks
University of Utah · 2The University of Queensland · University of Waterloo +1
Plasticity of Growing and Elastic Neural Networks in Online Continual Learning
Aug 2, 2026Jeong Min Kong, Richard S. SuttonLoss of Plasticity Dynamic Neural Networks
Department of Electrical and Computer Engineering University of California, Los Angeles (UCLA) · Department of Computing Science University of Alberta Alberta Machine Intelligence Institute (Amii)
Elastic Spectral State Space Models for Train-Once Budgeted Inference
Jan 30, 2026Dachuan Song, Junyu Yin, Zechen Hu +1Sequence Modeling State Space Models
Department of Electrical and Computer Engineering, George Mason University, Fairfax, VA, USA · Department of Computer Science, George Mason University, Fairfax, VA, USA

Oct 23, 2026J/K move · Enter open · S save

EDWARD-NMT: Elastic Dimensions With Adaptive Routing and Dual-consistency for Neural Machine Translation

Edward Nguyen, Thai Tran, Kiet Anh Hoang, Duy Hoang+3

UNEY, Switzerland

Machine Translation Mixture-of-Experts Models Elastic Neural Networks

Open paper AskPDF arXiv

Sparse Mixture-of-Experts (MoE) is a powerful paradigm for scaling Neural Machine Translation (NMT) and managing language heterogeneity. However, practical limitations hinder its deployment on resource-constrained devices: the entire expert pool must remain in memory, and the per-token compute budget is statically fixed during training by rigid top-k routing. We introduce EDWARD-NMT (Elastic Dimensions With Adaptive Routing and Dual-consistency for NMT), a novel MoE encoder-decoder architecture designed for a train once, deploy anywhere paradigm that exposes a family of deployable sub-networks at inference time. EDWARD-NMT integrates three key innovations: (i) Matryoshka-style elastic dimensions to flexibly reduce the memory footprint; (ii) an adaptive sigmoid router with a learned, task-conditioned threshold, allowing each token to dynamically determine its active expert count; and (iii) a dual-consistency objective to maximize the performance of the extracted sub-networks. Extensive experiments on PhoMT (En-Vi), WMT-14 (En-De), multilingual OPUS-16, and a multi-domain De-En benchmark validate our approach. The full-capacity architecture matches or exceeds strong MoE baselines with improved parameter efficiency. Furthermore, the sub-networks seamlessly extracted from a single checkpoint consistently outperform comparably sized dense Transformers without any retraining.