Sequence Modeling
Overview
Sequence modeling asks how information in an ordered stream—such as words, sensor readings, user actions, or DNA—can be used to predict what comes next or interpret the sequence as a whole. Common approaches include recurrent networks such as LSTMs, Transformer attention, state-space models such as Mamba, and Markov or hidden Markov models; hybrids and diffusion methods are also used for long or structured sequences. Applications range from language and time-series forecasting to recommendation, biological analysis, and action recognition, with evaluation often testing prediction quality, long-range recall, and generalization to new lengths or domains. A central tension is balancing expressive memory and reliable long-context behavior against computation, latency, and memory costs, while ensuring benchmarks measure genuine sequence understanding rather than leakage or repetition.
Latest papers
268 papers18 papers in the last four weeks, down 31% on the four weeks before. 0.1% of all new papers.
Mechanistic Interpretability of Structure-Aware Numerical Reasoning in LLaMA 3.1 8B
How Far Should Tokenization Go? Predictive Effectiveness and Relational Losslessness
UniDot: A Unified Network for Sequence Modeling and Feature Interaction in Large-scale Recommendation
SAGA: Structure-Attended Generative Action Embedding Model that encodes Multi-Surface User Action Sequences
RecipeNet: A Hierarchical Transformer for Recipe Data
Learning to Forecast Crop Growth from Earth Observation Data
Sequence prediction under a lying oracle
Cross-View Sequential Visual Localization with Spatio-Temporal Context Modeling for Autonomous Driving
ChronoSSM: Training for Temporally Aware Representations in Autoregressive State Space Models
Recurrent Neural Networks Beyond Time: Learning from Multiple Ordered Projections
iStructTab: Structured Feature Sequencing for Multimodal Learning of Image and Tabular Data
Entity-Aware Sequence Transduction for Player-Centric Ball Action Spotting
Hierarchical Solomonoff Induction: An Unbounded Machine Learning Model
Evolutionary Curriculum Learning Improves Biological Sequence Modeling
PTP: Previous-Token Prediction based LLM Inversion for Near-Exact Prompt Reconstruction
SEDR-Seq2P: A Lightweight Dilated Residual Sequence-to-Point Network for Multi-Task Industrial NILM
Revisiting Predictive Process Monitoring in the Age of Foundation Models: A Comparative Study of Sequence, Tabular, and LLM Approaches
Modeling Decisions in Blockchain Analytics: A Leakage-Aware Evaluation of Tree-Based vs. Sequential Models
GraphIDyOM: A graph-native Python reimplementation of IDyOM for musical expectation modelling
A scaling law of contextual persistence in human language
StageGuard: Physiologically Constrained Sleep Staging
User-Centric Modeling of Transactional Sequences with Explainable State Space Models
Cumsum-Composable Phase Transport for Low-Cost Streaming Keyword Spotting
Sequential Learner Modeling Using Multi-Relational Graph Convolutional Networks
End-to-End Markov State Sequence Learning for Auditory Attention Decoding
Stringological sequence prediction II: Right-to-left automaticity and related complexity measures
dual'' right-to-left (least-significant-digit-first) automaticity, which turns out to be substantially different for our purpose. We also demonstrate a prediction algorithm for a more expressive measure that we call arithmetic repetition complexity''. In particular, the latter can be used for predicting the so-called mix-automatic sequences.