Sequence Prediction

Follow this topic Ask about this topic

Latest papers

31 papers8 papers in the last four weeks, up 167% on the four weeks before. 0.0% of all new papers.

Next Token or Vector? Learning How Far to Decode
Oct 16, 2026Sai Sreenivas Chintha, Chidera Biringa, Rishab Sharma +6LLM Inference Acceleration Multi-Token Prediction
Microsoft · University of Massachusetts Amherst
SteerCast: Retrieval-Based Latent Steering for Decoder-Only Time Series Forecasting
Oct 8, 2026Van Dai Do, Huu Hiep Nguyen, Minh Hoang Nguyen +1Multivariate Time Series Forecasting Sequence Prediction
Deakin’s Applied Artificial Intelligence Initiative Deakin University, Geelong, Australia
LightMTP: Lightweight Latent Multi-Token Prediction
Oct 5, 2026Tamara Czinczoll, Julie Kallini, Gerard de Melo +1Language Model Pretraining Multi-Token Prediction
Hasso Plattner Institute / University of Potsdam · Stanford University · Tel-Aviv University
How Far Back Should a Transformer Look? Repetition and Copying in Music Sequence Models
Oct 2, 2026Amir FathiLong-Context Modeling Sequence Modeling
Department of Computing Science University of Alberta Edmonton, AB, Canada
OpenTSLM TeeMoE: A Unified Time-Series Language Model for Forecasting, Contextual Prediction, and Reasoning
Sep 30, 2026Tony Chen, Timo Stoffregen, Maxwell Xu +8Temporal Reasoning in Language Models LLM-Based Time Series Forecasting
Columbia University, USA · Stanford University, USA · Aionic Labs, Switzerland +4
Information-Theoretic Analysis of Next-Token Prediction under Markovian Data
Sep 28, 2026Masoud Kavian, Abdellatif Zaidi, Milad SefidgaranInformation-Theoretic Generalization Bounds Next-Token Prediction
Mathematical and Algorithmic Science Laboratory, Huawei Paris Research Center, 92100 Boulogne-Billancourt, France · Laboratoire d’Informatique Gaspard Monge, Université Gustave Eiffel, 77420 Champs-sur-Marne, France
Decoupling Token Roles in Autoregressive Pretraining
Sep 27, 2026Suqin Yuan, Runqi Lin, Kevin Qinghong Lin +4Language Model Pretraining Autoregressive Language Modeling
University of Sydney · University of Oxford · Southeast University
The Cost of Long Memory: State, Context, and Stability Complexity in Sequence Models
Sep 24, 2026Yuheng SongSequence Prediction Time Series Forecasting
School of Management, Huazhong University of Science and Technology
Stringological sequence prediction III: layered ziplines and a tradeoff between efficiency and expressivity
Sep 17, 2026Vanessa KosoySequence Modeling Sequence Prediction
Faculty of Mathematics, Technion, Haifa, Israel · Computational Rational Agents Laboratory, Delaware, USA
SIDE: Sensor Impersonation Detection at the Edge via Sequence Prediction
Sep 5, 2026Nahom BirhanInternet of Things Time-Series Anomaly Detection
Department of Electrical and Computer Engineering Old Dominion University Virginia, USA
Fast Weight Attention for Continual Learning
Aug 27, 2026Yifan Zhang, Steve Ta, Jasper Zhang +8Recurrent Language Models Memory-Augmented Language Models
Bytedance Seed · Princeton University · Hyperbolic Labs +1
Truthful Calibration Measures for Sequential Prediction
Aug 21, 2026Anagha Gokul, Jason Hartline, Lunjia Hu +2Online Calibration Sequence Prediction
Khoury College of Computer Sciences · Department of Computer Science · Microsoft Research New England
How Far Should Tokenization Go? Predictive Effectiveness and Relational Losslessness
Aug 18, 2026Yi WangTokenization Token Efficiency
Department of Electronic Engineering, Tsinghua University
Sequence prediction under a lying oracle
Aug 14, 2026Puspabeethi Samanta, Nikhil Karamchandani, Jayakrishnan NairRegret Minimization Sequence Prediction
Department of Electrical Engineering IIT Bombay
Hierarchical Latent Prediction for Language Models
Aug 6, 2026Chang Shi, Tim Pearce, Manan Tomar +2Language Model Pretraining Multi-Token Prediction
University of Texas at Austin · Microsoft Research
Hierarchical Solomonoff Induction: An Unbounded Machine Learning Model
Aug 2, 2026Nathan YoungHierarchical Bayesian Modeling Statistical Learning Theory
Strong AI Lab, University of Auckland
Stringological sequence prediction II: Right-to-left automaticity and related complexity measures
Jul 19, 2026Vanessa KosoySequence Prediction Sequence Modeling
Faculty of Mathematics, Technion, Haifa, Israel · Computational Rational Agents Laboratory, Delaware, USA
ReDiTT: Retrieval Augmented Conditional Diffusion Transformers for Asynchronous Time Series
Jul 14, 2026Saiyue Lyu, Zhitian Zhang, Ruizhi Deng +1Irregular Time-Series Modeling Event Time Prediction
Department of Computer Science, University of British Columbia · Work done during an internship at RBC Borealis. · RBC Borealis
Pretraining Recurrent Networks without Recurrence
Jun 4, 2026Akarsh Kumar, Phillip IsolaRecurrent Neural Networks Recurrent Language Models
MIT
Prediction Under Imperfect Compression: A Theory of Approximate MDL
Jun 3, 2026Qian Li, Xinyu Mao, Shang-Hua Teng +1Minimum Description Length Approximation Algorithms
Shenzhen Research Institute of Big Data · University of Southern California
Time Series as Language: A Universal Tokenizer for General-Purpose Time Series Foundation Models
May 31, 2026Yunhao Zhang, Ruiying Qi, Jiale Zheng +3Time Series Tokenization Time Series Generation
Shanghai Jiao Tong University · Huawei Noah’s Ark Lab
EviTrack: Selection over Sampling for Delayed Disambiguation
May 19, 2026Omer HaqSequence Prediction Sequential Hypothesis Testing
Independent Researcher · Baltimore, MD, USA
Conditional Attribute Estimation with Autoregressive Sequence Models
May 13, 2026Erica Stutz, Giacomo Marino, Daniella Meeker +2Next-Token Prediction Controllable Text Generation
Department of Biomedical Informatics and Data Science Yale University New Haven, CT 06510 · Department of Biostatistics Yale University New Haven, CT 06510 · Department of Biomedical Informatics and Data Science, Department of Pediatrics Yale University New Haven, CT 06510
The Power of Second Order Methods for Sequence Preconditioning
May 8, 2026Annie Marsden, Elad HazanDynamical Systems Sequence Modeling
Google DeepMind · Princeton University
Sessa: Selective State Space Attention
Apr 20, 2026Liubomyr HorbatkoState Space Models Long-Context Language Modeling
Stringological sequence prediction I: efficient algorithms for predicting highly repetitive sequences
Mar 27, 2026Vanessa KosoySequence Prediction Sequence Modeling
Faculty of Mathematics, Technion – Israel Institute of Technology, Haifa, Israel 3200003. · Computational Rational Agents Laboratory (coral-research.org), 2810 N Church St PMB 49028, Wilmington, DE 19802.
Thinking into the Future: Latent Lookahead Training for Transformers
Mar 3, 2026Lorenzo Noci, Gregor Bachmann, Seyed-Mohsen Moosavi-Dezfooli +1Autoregressive Language Modeling Sequence Prediction
Apple
Online Prediction of Stochastic Sequences with High Probability Regret Bounds
Feb 18, 2026Matthias Frey, Jonathan H. Manton, Jingge ZhuSequence Prediction Sequence Modeling
Department of Electrical and Electronic Engineering, The University of Melbourne
Quantum Attention by Overlap Interference: Predicting Classical and Many-Body Quantum Sequences
Feb 6, 2026Alessio Pecilli, Matteo RosatiSelf-Attention Transformer Attention
Dipartimento di Ingegneria Civile, Informatica e delle Tecnologie Aeronautiche, Universit`a degli Studi Roma Tre, Via della Vasca Navale 79, 00146 Rome, Italy
Training LLMs Beyond Next Token Prediction - Filling the Mutual Information Gap
Oct 31, 2025Chun-Hao Yang, Bo-Han Feng, Tzu-Yuan Lai +3Autoregressive Language Modeling Sequence Prediction
National Taiwan University

Oct 16, 2026J/K move · Enter open · S save

Next Token or Vector? Learning How Far to Decode

Sai Sreenivas Chintha, Chidera Biringa, Rishab Sharma, Anusha Agrawal+5

Microsoft · University of Massachusetts Amherst

LLM Inference Acceleration Multi-Token Prediction Sequence Prediction Speculative Decoding

Open paper AskPDF arXiv

Standard autoregressive decoding advances one token at a time, even when several upcoming tokens can be predicted together. We propose CADENCE (Context-Adaptive DEcoding), a generation method that learns both what to draft and how far to draft. Gaussian prediction heads represent future spans as continuous latent vectors, and a shared decoder converts the selected vector into token predictions in parallel. A learned router selects the prediction length from the model's own hidden states and predicted Gaussian parameters, adapting generation granularity to the current context with single-token generation as a fallback. The backbone verifies each proposed continuation and retains the prefix that matches its greedy predictions. Experiments with Llama-3.1-8B-Instruct on Alpaca, GSM8K, and HumanEval show that router-selected positions yield higher draft acceptance than applying the corresponding prediction length everywhere. CADENCE achieves up to 1.3x higher generation throughput than standard greedy decoding.