Sequence Modeling

Ask about this topic

Overview

Sequence modeling asks how information in an ordered stream—such as words, sensor readings, user actions, or DNA—can be used to predict what comes next or interpret the sequence as a whole. Common approaches include recurrent networks such as LSTMs, Transformer attention, state-space models such as Mamba, and Markov or hidden Markov models; hybrids and diffusion methods are also used for long or structured sequences. Applications range from language and time-series forecasting to recommendation, biological analysis, and action recognition, with evaluation often testing prediction quality, long-range recall, and generalization to new lengths or domains. A central tension is balancing expressive memory and reliable long-context behavior against computation, latency, and memory costs, while ensuring benchmarks measure genuine sequence understanding rather than leakage or repetition.

Latest papers

CardsList

250 papers22 papers in the last four weeks, up 38% on the four weeks before. 0.1% of all new papers.

  1. DPRM: A Plug-in Doob h transform-induced Token-Ordering Module for Diffusion Language Models

    Apr 27, 2026Dake Bu, Wei Huang, Andi Han +4Doob H-TransformProcess Reward Models

    City University of Hong Kong · CFAR and IHPC, A*STAR · RIKEN AIP +4

  2. Toeplitz MLP Mixers are Low Complexity, Information-Rich Sequence Models

    Apr 24, 2026Benjamin L. Badger, Ethan RolandSequence Modeling

    IBM · AE Studio

  3. HubRouter: A Pluggable Sub-Quadratic Routing Primitive for Hybrid Sequence Models

    Apr 24, 2026Abhinaba BasuTransformer AttentionLanguage Modeling

    National Institute of Electronics and Information Technology (NIELIT)

  4. An explicit operator explains end-to-end computation in the modern neural networks used for sequence and language modeling

    Apr 22, 2026Anif N. Shikder, Ramit Dey, Sayantan Auddy +6Dynamical SystemsNeural Network Interpretability

    Department of Mathematics, Western University, London ON, Canada · Fields Lab for Network Computation, Fields Institute, Toronto ON, Canada · King’s University College at Western University, London ON, Canada +1

  5. Causal-Transformer with Adaptive Mutation-Locking for Early Prediction of Acute Kidney Injury

    Apr 22, 2026Weizhi Nie, Haolin ChenCausal InterpretabilityLongitudinal EHR Modeling

    School of Electrical and Information Engineering, Tianjin University

  6. Markov reads Pushkin, again: A statistical journey into the poetic world of Evgenij Onegin

    Apr 22, 2026Angelo Maria SabatiniComputational Literary StudiesMarkov Models

    The BioRobotics Institute Scuola Superiore Sant’Anna Pisa, Italy

  7. CAST: Modeling Semantic-Level Transitions for Complementary-Aware Sequential Recommendation

    Apr 21, 2026Qian Zhang, Lech Szymanski, Haibo Zhang +1Sequential RecommendationRecommender Systems

    School of Computing, University of Otago · Dunedin, New Zealand · University of New South Wales +1

  8. Streaming Structured Inference with Flash-SemiCRF

    Apr 20, 2026Benjamin K. Johnson, Thomas Goralski, Ayush Semwal +2Efficient InferenceConditional Random Fields

    Department of Epigenetics, Van Andel Institute, 333 Bostwick Ave. NE, Grand Rapids, 49503, MI, USA. · Department of Cell Biology, Van Andel Institute, 333 Bostwick Ave. NE, Grand Rapids, 49503, MI, USA.

  9. Multi-Scale Reversible Chaos Game Representation: A Unified Framework for Sequence Classification

    Apr 20, 2026Sarwan Ali, Taslim MuradSequence Modeling

    Department of Computer Science, Institute of Business Administration Karachi, Pakistan

  10. Boltzmann Machine Learning with a Parallel, Persistent Markov chain Monte Carlo method for Estimating Evolutionary Fields and Couplings from a Protein Multiple Sequence Alignment

    Apr 20, 2026Sanzo MiyazawaSequence ModelingInverse Problems

  11. Generic Triple-Latent Compression with Gated Associative Retrieval

    Apr 17, 2026Liu XiaoMemory-Augmented Language ModelsAutoregressive Language Modeling

  12. Neural architectures for resolving references in program code

    Apr 15, 2026Gergő Szalay, Gergely Zsolt Kovács, Sándor Teleki +2Sequence-to-Sequence ModelsNeural Algorithmic Reasoning

    Faculty of Informatics, ELTE Eötvös Loránd University, Budapest, H-1117

  13. Is Sliding Window All You Need? An Open Framework for Long-Sequence Recommendation

    Apr 14, 2026Sayak Chakrabarty, Souradip PalSequential RecommendationSequence Modeling

    Department of Computer Science, Northwestern University, Evanston, IL 60208, USA · School of Electrical and Computer Engineering, Purdue University, West Lafayette, IN 47906, USA

  14. Beyond Isolated Clients: Integrating Graph-Based Embeddings into Event Sequence Models

    Apr 10, 2026Harry Proshian, Nikita Severin, Sergey Nikolenko +5Self-Supervised LearningContrastive Learning

    Steklov Institute of Mathematics Saint Petersburg, Russia · Independent Researcher Belgrade, Serbia · ISP RAS Steklov Institute of Mathematics Saint Petersburg, Russia +4

  15. EvoLen: Evolution-Guided Tokenization for DNA Language Model

    Apr 9, 2026Nan Huang, Xiaoxiao Zhou, Junxia Cui +4Language ModelingSequence Modeling

    University of California, San Diego, La Jolla, CA 92093, USA · Washington University in St. Louis, St. Louis, MO 63130, USA

  16. PRAGMA: Revolut Foundation Model

    Apr 9, 2026Maxim Ostroukhov, Ruslan Mikhailov, Vladimir Iashin +11Financial ServicesSequence Modeling

    Revolut Research · NVIDIA

  17. PhageBench: Can LLMs Understand Raw Bacteriophage Genomes?

    Apr 7, 2026Yusen Hou, Weicai Long, Haitao Hu +3GenomicsLLM Evaluation

    Hong Kong University of Science and Technology (Guangzhou)

  18. Entropy, Disagreement, and the Limits of Foundation Models in Genomics

    Apr 5, 2026Maxime Rochkoulets, Lovro Vrček, Mile ŠikićGenomic Foundation ModelsGenomics

    Genome Institute of Singapore, A*STAR, Singapore · KU Leuven, Belgium · Faculty of Electrical Engineering and Computing, University of Zagreb, Croatia

  19. CAWN: Continuous Acoustic Wave Networks for Autoregressive Language Modeling

    Apr 5, 2026Dejan Čugalj, Aleksandar JevremovicLong-Context Language ModelingSequence Modeling

    Singidunum University

  20. BTS-rPPG: Orthogonal Butterfly Temporal Shifting for Remote Photoplethysmography

    Apr 2, 2026Ba-Thinh Nguyen, Thi-Duyen Ngo, Thanh-Trung Huynh +2Remote PhotoplethysmographySequence Modeling

    VNU University of Engineering and Technology, Hanoi, Vietnam · College of Engineering and Computer Science, VinUniversity, Hanoi, Vietnam · VinUni-Illinois Smart Health Center, VinUniversity, Hanoi, Vietnam +1

  21. Exploration of Fast-Slow Latent Recurrence for Train-Short, Test-Long Generalization

    Apr 2, 2026Shota Takashiro, Masanori Koyama, Takeru Miyato +3OOD GeneralizationRecurrent Neural Networks

    The University of Tokyo, Tokyo, Japan · University of Tübingen, Tübingen, Germany

  22. An Empirical Markov Chain Car-Following (MC-CF) Model

    Mar 29, 2026Sungyong Chung, Yanlin Zhang, Nachuan Li +2Intelligent Transportation SystemsMarkov Chains

    Department of Civil and Environmental Engineering, University of Illinois Urbana-Champaign · Northwestern University Transportation Center, Northwestern University

  23. ImmSET: Sequence-Based Predictor of TCR-pMHC Specificity at Scale

    Mar 27, 2026Marco Garcia Noceda, Matthew T Noakes, Andrew FigPope +3Protein-Protein Interaction PredictionSequence Modeling

    Adaptive Biotechnologies, Seattle, WA, USA

  24. When Perplexity Lies: Generation-Focused Distillation of Hybrid Sequence Models

    Mar 27, 2026Juan Gabriel Kostelec, Qinghai GuoLLM Inference EfficiencyLLM Evaluation

    Huawei Switzerland · ACS Lab, Huawei Technologies

  25. Entanglement as Memory: Mechanistic Interpretability of Quantum Language Models

    Mar 27, 2026Nathan RollLong-Range Dependency ModelingCircuits in Language Models

    Department of Linguistics, Stanford University

  26. Stringological sequence prediction I: efficient algorithms for predicting highly repetitive sequences

    Mar 27, 2026Vanessa KosoySequence PredictionSequence Modeling

    Faculty of Mathematics, Technion – Israel Institute of Technology, Haifa, Israel 3200003. · Computational Rational Agents Laboratory (coral-research.org), 2810 N Church St PMB 49028, Wilmington, DE 19802.

  27. Physics-Informed Neural Networks and Sequence Encoder: Application to heating and early cooling of thermo-stamping process

    Mar 27, 2026Mouad Elaarabi, Domenico Borzacchiello, Philippe Le Bot +2Physics-Informed MLNeural Surrogate Modeling

    Nantes Université, Ecole Centrale Nantes, CNRS, GeM, UMR 6183, Nantes, 44321, France · Nantes Université, IRT Jules Verne, Bouguenais, 44340, France

  28. Deep Attention-based Sequential Ensemble Learning for BLE-Based Indoor Localization in Care Facilities

    Mar 22, 2026Minh Triet Pham, Quynh Chi Dang, Le Nhat TanWireless LocalizationEnsemble Learning

    School of Computing and Information Systems, The University of Melbourne, Melbourne, Australia · Faculty of Applied Science, Ho Chi Minh City University of Technology, Vietnam National University - Ho Chi Minh City, Ho Chi Minh City, Vietnam

  29. A two-step sequential approach for hyperparameter selection in finite context models

    Mar 20, 2026José Contente, Ana Martins, Armando J. Pinho +1Hyperparameter OptimizationSequence Modeling

    Institute of Electronics and Informatics Engineering of Aveiro (IEETA), Department of Electronics, Telecommunications and Informatics (DETI), University of Aveiro (UA), Aveiro, Portugal · Intelligent Systems Associate Laboratory (LASI), Portugal

  30. Learning to Predict, Discover, and Reason in High-Cardinality Event Sequences

    Mar 17, 2026Hugo MathFault DiagnosisPredictive Maintenance

    Faculty of Applied Computer Science, University of Augsburg