Efficient Sequence Mixers

Follow this topic Ask about this topic

Momentum

3 papers in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

Jul 6Week of Sep 21

Latest papers 15

All topics

CrossDistill: Balancing Quality and Diversity via Trajectory-Level Hybrid Few-Step Distillation
Sep 13, 2026Yuxi Liu, Haoyu Li, Yixiang Cai +8Few-Step Distillation Text-To-Video Diffusion Models
Peking University, Melon Group · Alibaba Group · Tsinghua University
Isotropic Embedding Perturbations for Robust Vision Language Encoders
Sep 9, 2026Hyesong Choi, Daeun Kim, Song Park +5Data Augmentation Single-Cell Foundation Models
Soongsil Univ. · NAVER AI Lab · Ewha W. Univ.
No-Regret Mixing of LRU and LFU with Optimal Switching Cost
Sep 7, 2026Younes Ben Mazziane, Xinying ZouEfficient Sequence Mixers
Linköping University
Higher-Dimensional Rotary Position Embedding
Aug 30, 2026Yixing Li, Ruobing Xie, Yudong Zhang +3Rotary Position Embedding Transformer Architectures
The Chinese University of Hong Kong · Tencent Hunyuan · Tsinghua University +1
Diff2Mix: Controllable Music Mixing via Diffusion Models and Differentiable Audio Effects
Aug 5, 2026Yisu Zong, Jinjie Shi, Joshua ReissAudio Editing Mix
Centre for Digital Music, Queen Mary University of London, UK
Passing Coarse Marginal Checks Can Be Cheap: Persona Mixtures and Imprecise Treatment-Response Estimates in an LLM Persona Panel
Aug 2, 2026Yohei NakajimaArtificial Intelligence Personas Efficient Sequence Mixers
Untapped Capital
Mixing-Free and Signal-Optimal Learning of Gaussian Graphical Models from Glauber Dynamics
Jul 20, 2026Vignesh Tirukkonda, Gautam DasarathyProbabilistic Graphical Models Optimal Learner
Arizona State University
Stringological sequence prediction II: Right-to-left automaticity and related complexity measures
Jul 19, 2026Vanessa KosoySequence Modeling Predictability
Faculty of Mathematics, Technion, Haifa, Israel · Computational Rational Agents Laboratory, Delaware, USA
An Integrable Token Mixing Layer from the Generalized Yang Baxter Equation
Jun 13, 2026Snigdha Chandan KhilarAsymmetric Grouped-Attention Variate Mixer Efficient Sequence Mixers
Independent Researcher
FastMix: Fast Data Mixture Optimization via Gradient Descent
Jun 12, 2026Haoru Tan, Sitong Wu, Yanfeng Chen +5Pretraining Efficient Sequence Mixers
University of Hong Kong · 2Hunyuan LLM, Tencent · 3Chinese University of Hong Kong
The Bridge-Garden Dilemma in LLM Distillation: Why Mixing Hard and Soft Labels Works
May 25, 2026Guanghui Wang, Kaiwen Lv Kacuila, Zhiyong Yang +5Knowledge Distillation Dataset Distillation
School of Computer Science and Technology, University of Chinese Academy of Sciences, Beijing, China · Alibaba Group, Hangzhou, China · State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China +2
Caracal: Causal Architecture via Spectral Mixing
Apr 30, 2026Bingzheng Gan, Tianyi Zhang, Yusu Li +4Causal Attention Sequence Modeling
Huawei Technologies Co., Ltd., China.
AfriqueLLM: How Data Mixing and Model Architecture Impact Continued Pre-training for African Languages
Jan 10, 2026Hao Yu, Tianyi Xu, Michael A. Hedderich +3Large Language Model Pretraining African Languages
McGill University, Canada · Mila-Quebec AI Institute, Canada · LMU Munich & Munich Center for Machine Learning, Germany +2
Controllably Efficient Language Models
Nov 7, 2025Jatin Prakash, Aahlad Puli, Rajesh RanganathTransformer Architectures Linear Attention
New York University
Sanity Checking Causal Representation Learning on a Simple Real-World System
Feb 27, 2025Juan L. Gamella, Simon Bing, Jakob RungeCausal Representation Learning Causal
Seminar for Statistics, ETH Zurich · Technische Universität Berlin · Department of Computer Science, University of Potsdam +1

Sep 13, 2026, cs.CVJ/K move · Enter open · S save

CrossDistill: Balancing Quality and Diversity via Trajectory-Level Hybrid Few-Step Distillation

Yuxi Liu, Haoyu Li, Yixiang Cai, Tengxu Sun+7

Peking University, Melon Group · Alibaba Group · Tsinghua University

Few-Step Distillation Text-To-Video Diffusion Models Diversity Efficient Sequence Mixers Visual Fidelity

Open paper AskPDF arXiv

Few-step distillation accelerates diffusion models but must balance diversity and fidelity: trajectory-based distillation preserves mode coverage, while distribution matching sharpens samples but can reduce diversity. We show that this tension can be exploited in a noise-regime-dependent way: high-noise steps largely determine global modes, whereas low-noise steps refine local details. We propose CrossDistill, a trajectory-level hybrid distillation framework that splits the sampling trajectory at a crossover point, applies a trajectory-preserving objective on the high-noise interval and a distribution-matching objective on the low-noise interval, and couples the two stages through the crossover state. In contrast to loss-level mixing, and complementarily to training-time two-stage recipes, CrossDistill explicitly assigns complementary objectives along the noise axis, so that global branching is preserved before local statistics are sharpened. CrossDistill is a noise-level scheduling policy: PCM and DMD are plug-in instantiations, while the noise partition, crossover coupling, and objective ordering are the key design elements. Experiments on text-to-video diffusion models and qualitative image-to-video results show that CrossDistill expands the few-step quality-diversity frontier, retaining seed-level variation while achieving competitive visual fidelity.

Momentum

Related topics

Latest papers 15