Sequence Modeling

Follow this topic Ask about this topic

Sequence modeling asks how information in an ordered stream—such as words, sensor readings, user actions, or DNA—can be used to predict what comes next or interpret the sequence as a whole. Common approaches include recurrent networks such as LSTMs, Transformer attention, state-space models such as Mamba, and Markov or hidden Markov models; hybrids and diffusion methods are also used for long or structured sequences. Applications range from language and time-series forecasting to recommendation, biological analysis, and action recognition, with evaluation often testing prediction quality, long-range recall, and generalization to new lengths or domains. A central tension is balancing expressive memory and reliable long-context behavior against computation, latency, and memory costs, while ensuring benchmarks measure genuine sequence understanding rather than leakage or repetition.

Latest papers

250 papers22 papers in the last four weeks, up 38% on the four weeks before. 0.1% of all new papers.

PRISM: Parallel Residual Iterative Sequence Model
Feb 11, 2026Jie Jiang, Ke Cheng, Xin Xu +8Sequence Modeling
Tencent AMS., Beijing, China · State Key Laboratory of software development environment, School of Computer Science and Engineering, Beihang University, Beijing, China · State Key Laboratory of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University, Beijing, China.
Quantum Attention by Overlap Interference: Predicting Classical and Many-Body Quantum Sequences
Feb 6, 2026Alessio Pecilli, Matteo RosatiSelf-Attention Transformer Attention
Dipartimento di Ingegneria Civile, Informatica e delle Tecnologie Aeronautiche, Universit`a degli Studi Roma Tre, Via della Vasca Navale 79, 00146 Rome, Italy
From Word Sequences to Behavioral Sequences: Adapting Modeling and Evaluation Paradigms for Longitudinal NLP
Jan 12, 2026Adithya V Ganesan, Vasudha Varadarajan, Oscar NE Kjell +6Sequence Modeling
Stony Brook University · Vanderbilt University · Carnegie Mellon University +3
Lag Operator SSMs: A Geometric Framework for Structured State Space Modeling
Dec 22, 2025Sutashu Tomonaga, Kenji Doya, Noboru MurataSequence Modeling State Space Models
Okinawa Institute of Science and Technology · Waseda University
When do cheap embeddings beat protein language models? A theoretically-grounded hashing sketch for biological sequence classification
Dec 10, 2025Sarwan Ali, Taslim Murad, Imdadullah Khan +1Protein Language Models Randomized Sketching
Columbia University, NY, USA · Institute of Business Administration, Karachi, Pakistan · Lahore University of Management Sciences, Lahore, Pakistan +1
Naga: Vedic Encoding for Deep State Space Models
Nov 17, 2025Melanie Schaller, Nick Janssen, Bodo RosenhahnSequence Modeling State Space Models
Aligning Inductive Bias for Data-Efficient Generalization in State Space Models
Sep 25, 2025Qiyu Chen, Guozhang ChenSequence Modeling Data-Efficient Learning
School of Computer Science, National Key Laboratory for Multimedia Information Processing, Peking University, China · School of Physics, Peking University, China
Leveraging Natural Language Processing to Unravel the Mystery of Life: A Review of NLP Approaches in Genomics, Transcriptomics, and Proteomics
Jun 2, 2025Ella Rannon, David BursteinProtein Language Models Genomics
The Shmunis School of Biomedicine and Cancer Research, George S. Wise Faculty of Life Sciences, Tel-Aviv University, Tel-Aviv, Israel
TEMPURA: Temporal Event Masked Prediction and Understanding for Reasoning in Action
May 2, 2025Jen-Hao Cheng, Yi-Hao Peng, Huapeng Zhou +11Temporal Action Segmentation Temporal Video Grounding
University of Washington · Carnegie Mellon University · National Yang Ming Chiao Tung University +1
CHORDONOMICON: A Dataset of 666,000 Songs and their Chord Progressions
Oct 29, 2024Spyridon Kantarelis, Ioannis Liolitsas, Konstantinos Thomas +3Music Information Retrieval Sequence Modeling
Artificial Intelligence and Learning Systems Laboratory, National Technical University of Athens, Greece

Feb 11, 2026J/K move · Enter open · S save

PRISM: Parallel Residual Iterative Sequence Model

Jie Jiang, Ke Cheng, Xin Xu, Mengyang Pang+7

Tencent AMS., Beijing, China · State Key Laboratory of software development environment, School of Computer Science and Engineering, Beihang University, Beijing, China · State Key Laboratory of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University, Beijing, China.

Sequence Modeling

Open paper AskPDF arXiv

Generative sequence modeling faces a fundamental tension between the expressivity of Transformers and the efficiency of linear sequence models. Existing efficient architectures are theoretically bounded by shallow, single-step linear updates, while powerful iterative methods like Test-Time Training (TTT) break hardware parallelism due to two dimensions of serial dependency: token-level state reliance and step-level iteration loops. We propose PRISM (Parallel Residual Iterative Sequence Model) to resolve this tension. PRISM explicitly approximates the expressive gate-residual-direction iteration pattern of TTT in a parallelizable form. We employ a Write-Forget Decoupling strategy that isolates non-linearity within the injection operator. To bypass the serial dependency of explicit solvers, PRISM utilizes a two-stage proxy architecture: a short-convolution anchors the initial residual using local history energy, while a learned predictor estimates the refinement updates directly from the input. This design distills structural patterns associated with iterative correction into a parallelizable feedforward operator. Theoretically, we prove that this formulation achieves Rank-

L

accumulation, structurally expanding the update scheme beyond the single-step Rank-

1

bottleneck. Empirically, it achieves comparable performance to explicit optimization methods while achieving 174x higher throughput. Codes are available in https://github.com/gpr-prism/prism/.