Single-Token Output Distributions

Follow this topic Ask about this topic

Momentum

3 papers in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

Jul 6Week of Sep 21

Latest papers 29

All topics

Distribution Matching Distillation for Continuous Diffusion Language Models
Sep 30, 2026Paul Le Van Kiem, Dario Shariatian, Umut Simsekli +1Diffusion Language Models Diffusion Models
Inria, PSL Research University · Cohere · CMAP, Ecole Polytechnique
Referential Uncertainty in Human--AI Collaboration
Sep 30, 2026Christian Poelitz, Finale Doshi-Velez, Siân LindleyHuman-Ai Collaboration Human-Robot Collaboration
Microsoft Research, Cambridge, UK · Harvard University, Cambridge, MA, USA
Probability is Not Enough: Exploring and Counting Divergent Tokens for Reasoning Uncertainty Quantification in LLMs
Sep 29, 2026Feiyang Li, Shengjing Liu, Qi Zhan +7Large Language Model Uncertainty Confidence Calibration
College of Computer Science and Software Engineering, Shenzhen University · Tsinghua University · Behavioral and Spatial AI Lab, Peking University & Tongji University
XU-RS: Explaining Credal Width in Random-Set Language Models
Sep 29, 2026David Achara, Maryam Sultana, Alexander D. Rast +1Large Language Model Uncertainty Uncertainty Score
Institute for AI, Data Analysis and Systems (AIDAS) Oxford Brookes University
Divergence Timing and Cumulative Disagreement under KV-Cache Eviction
Sep 15, 2026Xinyue Luo, Fei YuDivergence Single-Token Output Distributions
Wealth AI Lab, Ant Group
Predictive Likelihood Ratios for Language Model Watermark Detection
Sep 14, 2026Li MaLarge Language Model Watermarks Two-Sample Testing
Department of Statistics and Data Science Institute, University of Chicago
Train What You Deploy:Token-Faithful Post-Training of a Production Coding
Sep 7, 2026Cheng Li, Jiexiong Liu, Yixuan Chen +1Post-Training Single-Token Output Distributions
KunlunMeta
Privileged Likelihood Is Not Automatically Value: Three Checks for Token Credit in On-Policy Self-Distillation
Aug 10, 2026Xuan-Phi Nguyen, Shrey Pandit, Yiran Zhao +3Credit Assignment Single-Token Output Distributions
Salesforce AI Research
Stochastic Autoregressive Learning
Aug 7, 2026Ilan Doron-Arad, Idan Mehalel, Elchanan MosselAutoregressive Model Single-Token Output Distributions
1MIT · 2The Hebrew University
Guarantees on Dynamical System Distinguishability for LLM Token Generation
Jul 25, 2026Mohamed Akrout, Dan WilsonSingle-Token Output Distributions Dynamical Systems
Department of Electrical Engineering and Computer Science University of Tennessee, Knoxville, TN, USA
Production and Perception in LLMs: A Token Probability Approach
Jul 13, 2026Anna Marklová, Jiří Milička, Martina Vokáčová +1Single-Token Output Distributions Linguistics
Faculty of Arts, Charles University, Prague · Faculty of Mathematics and Physics, Charles University, Prague
One Token Is Enough: Fingerprinting and Verifying Large Language Models from Single-Token Output Distributions
Jul 11, 2026Tomas BrucknerSingle-Token Output Distributions Fingerprint
Faculty of Informatics and Statistics, Prague University of Economics and Business, Prague, Czech Republic
Telescope: Improving Zero Shot Detection of LLM Generated Content By Measuring Token Repetition Probability
Jul 5, 2026Christopher Nassif, Josh F. CooperMachine-Generated Text Detection Single-Token Output Distributions
Virginia Tech Department of Electrical and Computer Engineering · Virginia Tech Department of Industrial and Systems Engineering
Sentence-Level Contextual Entrainment in Large Language Models
Jun 23, 2026Yang Liu, Chenhui ChuSingle-Token Output Distributions
Kyoto University
WaveDetect: Robust Framework for Machine-Generated Text Detection via Wavelet Transform
Jun 22, 2026Zhichen Liu, Kaitong Qin, Linhan He +1Machine-Generated Text Detection Attacker Large Language Model
Dept. of Computer Science and Engineering, Southern University of Science and Technology
Hacking Generative Perplexity: Why Unconditional Text Evaluation Needs Distributional Metrics
Jun 7, 2026Antonio Franca, Alexander TongLanguage Model Perplexity Single-Token Output Distributions
AITHYRA Institute, Vienna, Austria.
ARCA: Adapter-Residual Credit Assignment When Token Signals Degenerate
May 29, 2026Rodney Lafuente-MercadoCredit Assignment Large Language Model Reinforcement Learning
Scale AI
The Fragility of Chain-of-Thought Monitoring Across Typologically Diverse Languages
May 27, 2026Eric Onyame, Runtao Zhou, Kowshik Thopalli +2Chain-of-Thought Reasoning Single-Token Output Distributions
University of Virginia · Lawrence Livermore National Laboratory
Cultural Value Alignment Via Latent Activation Steering in Large Language Models
May 25, 2026Trung Duc Anh Dang, Sarah MasudCultural Evolution Large Language Model Alignment
University of Copenhagen
X-Token: Projection-Guided Cross-Tokenizer Knowledge Distillation
May 20, 2026Sharath Turuvekere Sreenivas, Adithyakrishna Venkatesh Hanasoge, Mingyu Yang +4Teacher-Student Distillation Single-Token Output Distributions
GEM: GPU-Variability-Aware Expert to GPU Mapping for MoE Systems
May 19, 2026Sourish Wawdhane, Avinash Kumar, Poulami DasMixture-Of-Expert Inference Mixture-Of-Experts
The University of Texas at Austin
Computer-Aided Design Generation by Cascaded Discrete Diffusion Model
May 6, 2026Honghu Pan, Xiaoling Luo, Yongyong Chen +2Diffusion Models Diffusion Dynamics
School of Artificial Intelligence and Robotics, Hunan University · College of Computer Science and Software Engineer, Shenzhen University · School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen +1
A geometric relation of the error introduced by sampling a language model's output distribution to its internal state
May 6, 2026Albert F. ModenbachSingle-Token Output Distributions Chess
Department of Mathematics, King’s College London, United Kingdom.
Prompt-Induced Score Variance in Zero-Shot Binary Vision-Language Safety Classification
May 1, 2026Charles Weng, Dingwen Li, Alexander MartinLarge Language Model Safety Zero-Shot Robustness
Johns Hopkins University · Independent
Reward Models Are Secretly Value Functions: Temporally Coherent Reward Modeling
Apr 24, 2026Alex NikulkovSingle-Token Output Distributions Value Functions
AI at Meta
Stochasticity in Tokenisation Improves Robustness
Apr 17, 2026Sophie Steger, Rui Li, Sofiane Ennadir +4Single-Token Output Distributions Stochastic
Institute of Signal Processing and Speech Communication, Graz University of Technology, Graz, Austria · ELLIS Institute Finland & Aalto University, Espoo, Finland · King AI Labs, Microsoft Gaming +2
The Truncation Blind Spot: How Decoding Strategies Systematically Exclude Human-Like Token Choices
Mar 19, 2026Esteban Garces Arias, Nurzhan Sapargali, Christian Heumann +1Machine-Generated Text Detection Single-Token Output Distributions
Department of Statistics, LMU Munich · Munich Center for Machine Learning (MCML)
Evaluating Generative Models via One-Dimensional Code Distributions
Mar 9, 2026Zexi Jia, Pengcheng Luo, Yijia Zhong +2Generative Models Long Visual-Token Sequences
WeChat AI, Tencent Inc., China · School of Intelligence Science and Technology, Peking University · College of Computer Science and Artificial Intelligence, Fudan University
Tokens, the oft-overlooked appetizer: Large language models, the distributional hypothesis, and meaning
Dec 14, 2024Julia Witte Zimmerman, Denis Hudon, Kathryn Cramer +9Single-Token Output Distributions Distributional Information
Computational Story Lab, Vermont Complex Systems Institute, University of Vermont, Burlington, VT 05405, USA · Vermont Complex Systems Institute, University of Vermont, Burlington, VT 05405, USA · Department of Computer Science, St. Michael’s College, Colchester, VT 05439, USA +3

Sep 30, 2026, cs.LGJ/K move · Enter open · S save

Distribution Matching Distillation for Continuous Diffusion Language Models

Paul Le Van Kiem, Dario Shariatian, Umut Simsekli, Alain Durmus

Inria, PSL Research University · Cohere · CMAP, Ecole Polytechnique

Diffusion Language Models Diffusion Models Dataset Distillation Distribution Matching Distillation Single-Token Output Distributions Simplex

Open paper AskPDF arXiv

Continuous diffusion language models generate all tokens in parallel, yet high-quality generation can still require hundreds of network evaluations (NFEs). We study how distributional distillation can reduce this cost by exploiting the student's probabilistic token outputs. Our unified formulation connects the student's output parameterization to the resulting gradient estimators and yields two methods with the same student architecture and reverse-KL matching objective: Simplex-DMD uses continuous token relaxations and pathwise gradients, while Reinforce-DMD uses categorical sampling and REINFORCE with a learned density ratio. We develop both methods for multi-step generation and investigate the training and sampling choices associated with each parameterization. On OpenWebText, for sequences of 1,024 tokens, Simplex-DMD achieves a generative perplexity of 45.6 at a unigram entropy of 5.44 nats in just 4 NFEs, a 49% reduction relative to the strongest evaluated diffusion baseline at matched entropy and sampling budget. Reinforce-DMD improves the frontier at larger budgets, reaching a generative perplexity of 14.9 at an entropy of 5.00 nats with 256 NFEs, a 20% reduction under the same comparison protocol.

Momentum

Related topics

Latest papers 29