Non-Autoregressive Text Generation

Ask about this topic

Overview

Non-autoregressive text generation studies how to generate sequences without choosing every next token in left-to-right order, aiming for faster inference and better use of parallel computation. Common approaches include masked diffusion with iterative remasking, discrete and continuous flow matching, latent-variable models, insertion-based generation, and distillation from autoregressive teachers. Methods are tested on language modeling, open-ended generation, summarization, and reasoning, with designs ranging from many refinement steps to one-step decoding. The main tension is speed versus coherence: parallel updates can miss token dependencies, and likelihood-based metrics may not reliably capture whether generated text is fluent or distributionally representative.

Latest papers

CardsList

31 papers7 papers in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

  1. Rectified Language Flow

    Oct 24, 2026Baiyu Su, Runlong Liao, Lizhang Chen +1Language ModelingNon-Autoregressive Text Generation

    The University of Texas at Austin

  2. Next-Concept Modeling: Fast, Parallel Coarse-to-Fine Text Generation

    Oct 9, 2026Hermann Kumbong, Ishan S. Khare, Iddah Mlauzi +6Text GenerationNon-Autoregressive Text Generation

    Department of Computer Science, Stanford University · Department of Electrical Engineering, Stanford University · Together AI

  3. One-Step Text Generation by Seq-Drifting

    Oct 9, 2026Jiayu Liu, Ruizhi Zhao, Tianyun Ji +3Text GenerationOne-Step Generative Modeling

    Hong Kong Institute of AI for Science, City University of Hong Kong · Shenzhen University · State Key Laboratory of Cognitive Intelligence, University of Science and Technology of China +2

  4. Isotropic Yet Undecodable: The Sequential Content-Sufficiency Gap in Latent-Predictive Text Representations

    Oct 6, 2026K. P. Santoso, N. Z. Fadil, F. P. Harsanti +2Non-Autoregressive Text GenerationPredictive Representation Learning

    School of Computing, National University of Singapore · Department of Information Technology, Institut Teknologi Sepuluh Nopember · Avalon AI +1

  5. SOL: Measuring Gaps between Text Distributions by Double Sliced Wasserstein Metrics

    Oct 5, 2026Gregor Kornhardt, Moritz Piening, Jannis Chemseddine +1Non-Autoregressive Text GenerationLanguage Model Generation Evaluation

    Institute of Mathematics, TU Berlin, Berlin, Germany · Department of Computer Science, University of Göttingen, Göttingen, Germany

  6. Nash Equilibrium Text: A Game-Theoretic Decoding Framework for Text Generation

    Oct 5, 2026Alireza Jafari, Arman Adibi, Mohammad Ghavamzadeh +1Nash EquilibriumNon-Autoregressive Text Generation

    Department of Computer Science University of Virginia Charlottesville, VA, USA · School of Computer and Cyber Sciences Augusta University Augusta, GA, USA · Qualcomm AI Research San Diego, CA, USA

  7. Gumbel Straight Flow: Distilling Autoregressive Models into One-step Flow Maps

    Sep 30, 2026Yeongmin Kim, Arnaud Doucet, Andrew Campbell +3Non-Autoregressive Text GenerationOne-Step Generative Modeling

    Google DeepMind · Work done as a Student Researcher at Google DeepMind Amsterdam

  8. DA-DLM: Explicitly Modeling Token Dependencies in Diffusion Language Models

    Sep 14, 2026Pengyu Ji, Zichen Zhang, Xiang Hu +1Non-Autoregressive Text GenerationDiffusion Language Model Decoding

    ‡School of Information Science and Technology, ShanghaiTech University · §Shanghai Engineering Research Center of Intelligent Vision and Imaging · ¶Tencent

  9. Representation-based Masked Diffusion Model

    Sep 11, 2026Yangrong Hu, Ding Huang, Xueyu Zhou +1Non-Autoregressive Text GenerationMasked Diffusion Models

    Department of Data Science and Artificial Intelligence The Hong Kong Polytechnic University, Hong Kong SAR, China · Department of Applied Mathematics The Hong Kong Polytechnic University, Hong Kong SAR, China · ByteDance Seed, Beijing, China

  10. SENSESHIFT: Continuous Sentiment-Controlled Text Generation via Encoder-based Mask Infilling

    Aug 25, 2026Shahed Masoudian, Markus Frohmann, Emmanouil Karystinaios +2Controllable Text GenerationNon-Autoregressive Text Generation

    Johannes Kepler University Linz · Thomson Reuters Labs · University of Toronto, Vector Institute

  11. Luna-TTS Family Technical Report

    Aug 12, 2026Feng Yin, Shuai Shi, Junjie Zheng +19Non-Autoregressive Text GenerationTTS Synthesis

    VUI Labs Research

  12. Context-weighted Discrete Flow Matching

    Jul 23, 2026Daniil Cherniavskii, Daniel Severo, Karen UllrichNon-Autoregressive Text GenerationAdaptive Loss Weighting

    University of Amsterdam · Work done during an internship at Meta FAIR. · Meta FAIR

  13. FreyaTTS Technical Report

    Jul 10, 2026Ahmet Erdem Pamuk, Ömer Yentür, Ahmet Tunga Bayrak +2Non-Autoregressive Text GenerationTTS Synthesis

    Freya Team

  14. Posterior Refinement: Fast Language Generation via Any-Order Flow Maps

    Jun 23, 2026Manan Agarwal, Sheel Shah, Chanhyuk Lee +6Non-Autoregressive Text GenerationGenerative Flow Networks

    Carnegie Mellon University · KAIST

  15. Non-Autoregressive Minimum Bayes' Risk Decoding for Fast Speech Recognition

    Jun 16, 2026Hiroyuki Deguchi, Takatomo Kano, Katsuki Chousa +1Non-Autoregressive Text GenerationMBR Decoding

    NTT, Inc., Japan

  16. Hacking Generative Perplexity: Why Unconditional Text Evaluation Needs New Metrics

    Jun 7, 2026Antonio Franca, Alexander TongNon-Autoregressive Text GenerationLLM Evaluation

    AITHYRA, Vienna, Austria

  17. NAVIRA: Decoupled Stochastic Remasking for Masked Diffusion Language Models

    Jun 4, 2026Andrey Fomenko, Maksim Kryzhanovskiy, Svetlana Glazyrina +1Non-Autoregressive Text GenerationMasked Diffusion Models

    Lomonosov Moscow State University, Moscow, Russia · Institute for Artificial Intelligence, Lomonosov Moscow State University, Moscow, Russia

  18. Variational Learning for Insertion-based Generation

    Jun 1, 2026Yangtian Zhang, Zhe Wang, Arthur Gretton +4Non-Autoregressive Text GenerationGoal-Conditioned Planning

    1Yale University · 2Google DeepMind · University College London +1

  19. When Confidence Misleads: Suffix Anchoring and Anchor-Proximity Confidence Modulation for Diffusion Language Models

    May 27, 2026Jungwon Park, Jimyeong Kim, Jungmin Ko +2Non-Autoregressive Text GenerationLanguage Model Decoding

    1RICS · 2AIIS · 3IPAI +2

  20. When Latent Geometry Is Not Enough: Draft-Conditioned Latent Refinement for Non-Autoregressive Text Generation

    May 15, 2026De Shuai ZhangNon-Autoregressive Text GenerationDiffusion Language Models

    Beijing Wuzi University

  21. Discrete Stochastic Localization for Non-autoregressive Generation

    May 13, 2026Yunshu Wu, Jiayi Cheng, Longxuan Yu +4Non-Autoregressive Text GenerationMasked Diffusion Models

    University of California Riverside · New York University

  22. How to Train Your Latent Diffusion Language Model Jointly With the Latent Space

    May 8, 2026Viacheslav Meshchaninov, Alexander Shabalin, Egor Chimbulatov +4Non-Autoregressive Text GenerationLatent Diffusion Models

    Constructor University, Bremen, Germany · HSE University, Moscow, Russia · Applied AI Institute, Moscow, Russia +2

  23. Continuous Latent Diffusion Language Model

    May 7, 2026Hongcan Guo, Qinyu Zhao, Yian Zhao +8Non-Autoregressive Text GenerationDiffusion Transformer

    ByteDance Seed · The Australian National University · Peking University +2

  24. Towards Faster Language Model Inference Using Mixture-of-Experts Flow Matching

    Apr 16, 2026Aihua LiNon-Autoregressive Text GenerationFlow Matching

    Duke University

  25. Early Decisions Matter: Proximity Bias and Initial Trajectory Shaping in Non-Autoregressive Diffusion Language Models

    Apr 12, 2026Jiyeon Kim, Sungik Choi, Yongrae Jo +2Non-Autoregressive Text GenerationDiffusion Language Model Decoding

    KAIST AI · LG AI Research · University of Illinois Chicago

  26. Discrete Flow Maps

    Apr 10, 2026Peter Potaptchik, Jason Yim, Adhi Saravanan +3Flow Map LearningNon-Autoregressive Text Generation

    Harvard University · University of Oxford · Independent +3

  27. DiffVC: A Non-autoregressive Framework Based on Diffusion Model for Video Captioning

    Apr 9, 2026Junbo Wang, Liangyu Fu, Yuke Li +4Video CaptioningNon-Autoregressive Text Generation

    School of Software, Northwestern Polytechnical University, Xi’an 710129, China · School of Computer Science, Northwestern Polytechnical University, Xi’an 710129, China · Beijing University Of Technology, Beijing 100124, China +1

  28. Gumbel Distillation for Parallel Text Generation

    Mar 23, 2026Chi Zhang, Xixi Hu, Bo Liu +1Non-Autoregressive Text GenerationLanguage Model Distillation

    Department of Computer Science The University of Texas at Austin

  29. Why Diffusion Language Models Struggle with Truly Parallel (Non-Autoregressive) Decoding?

    Feb 26, 2026Pengxiang Li, Dilxat Muhtar, Tianlong Chen +2Parallel Decoding for Diffusion Language ModelsDiffusion Language Model Decoding

    The Hong Kong Polytechnic University, Hong Kong, China · ELLIS Institute Tübingen, Tübingen, Germany · Max Planck Institute for Intelligent Systems, Tübingen, Germany +3

  30. On the Semantic and Syntactic Information Encoded in Proto-Tokens for One-Step Text Reconstruction

    Feb 20, 2026Ivan Bondarenko, Egor Palkin, Fedor TikunovNon-Autoregressive Text GenerationRepresentation Geometry in Language Models