Context Length

Follow this topic Ask about this topic

Momentum

4 papers in the last four weeks, up 33% on the four weeks before. 0.0% of all new papers.

Jul 6Week of Sep 21

Latest papers 28

All topics

Staying on Task: Testing the Foundations of Long-Horizon Agent Reliability
Sep 30, 2026Jeffrey Willette, Krishna C. Puvvada, Boris GinsburgLong-Horizon Agents Long-Horizon Task Planning
NVIDIA
CoEM: Empowering Long-Context Reasoning with Commit-on-Evidence Memory
Sep 29, 2026Jingguang Li, Yebo Wu, Zuyi Guo +7Long-Context Reasoning LLM Reasoning Strategies
Korea Advanced Institute of Science & Technology · University of Macau · Massachusetts Institute of Technology
KV-streams for Efficient Compaction in Agentic Reinforcement Learning
Sep 28, 2026Emiliano Penaloza, Dane Malenfant, Dheeraj Vattikonda +15Compaction Agentic Reinforcement Learning
Mila · Microsoft · Université de Montréal +9
Where Activation Sparsity and KV-Cache Sparsity Cross in LLM Decoding
Sep 27, 2026Jungseob Lee, Seungyoon Lee, Seongtae Hong +2Dynamic Sparse Attention Large Language Model Decoding
Korea University · Yonsei University Mirae Campus
Uncheatable Eval: Dynamic Compression-Based Evaluation of Language Models
Sep 23, 2026Kaifeng Tan, Yudong Li, Linlin ShenContext Length Data Compression Methods
Shenzhen University · Tsinghua University
Distance generalization in transformers: why bother with positional encoding?
Sep 11, 2026Daniel Henrik Nevermann, Claudius GrosPositional Encoding Transformer Architectures
Institute for Theoretical Physics, Goethe University Frankfurt, Germany
HeadWiseKV: Budgeted Per-Head Cache Residency for Hybrid Long-Context Language Models
Sep 2, 2026Renjie Xie, Juncheng Yang, Aoting Hu +4Efficient Long-Context Inference Key-Value Cache
Nanjing University of Posts and Telecommunications · Tylogi AI Lab / TAIL · Anhui University of Technology +2
REIGN: Refurbished Embeddings with Integrated Guidance Networks for Efficient Context-Length Scaling
Aug 30, 2026Devrim Çavuşoğlu, Emre AkbaşEfficient Long-Context Inference Retrieval Layer
Middle East Technical University · OBSS AI
VLZip: Unified Visual and Textual Compression for Interleaved Long-Context Modeling
Aug 9, 2026Yuqi Zhang, Cheng Chen, Yuyu Guo +6Recent Vision-Language Models Efficient Long-Context Inference
Shanghai Key Lab of Intell. Info. Processing, School of CS, Fudan University, China · Shanghai Innovation Institute, China · Ant Group, China
Distractor-Aware Truncation: Disentangling Context-Length Effects from Signal Loss in Long-Context LLM Benchmarks
Aug 4, 2026Mohsen ArjmandiContext Length Large Language Model Memory
evolutionID GmbH
Sequential Correlations Change In-Context Learning: Effective Context Length and Architectural Mismatch
Jul 4, 2026Mary Letey, Yue M. Lu, Cengiz Pehlevan +1In-Context Learning Sequence Modeling
John A. Paulson School of Engineering and Applied Sciences, Harvard University · Kempner Institute for the Study of Natural and Artificial Intelligence, Harvard University · Society of Fellows and Center for Brain Science, Harvard University
BamiBERT: A New BERT-based Language Model for Vietnamese
Jul 2, 2026Dat Quoc Nguyen, Thinh Pham, Chi Tran +1Bert-Based Models Language Modeling
Qualcomm AI Research · Virginia Tech
KVEraser: Learning to Steer KV Cache for Efficient Localized Context Erasing
Jun 15, 2026Mufei Li, Shikun Liu, Dongqi Fu +5Kv-Cache Management Key-Value Cache
Georgia Institute of Technology · Meta
Training and Evaluating Diffusion Policies with Long Context Lengths
Jun 15, 2026Abhinav Agarwal, Adam Wei, Taylan Kargin +6Imitation Learning Diffusion Policies
Massachusetts Institute of Technology
A Large-Scale Multi-Dimensional Empirical Study of LLMs for Conversation Summarization
Jun 14, 2026Weixiao Zhou, Gengyao Li, Xianfu Cheng +3Summarization Fact-Checking
CCSE, Beihang University · Fanyu AI Laboratory · MAIS, CASIA
End-to-End Context Compression at Scale
Jun 8, 2026Ang Li, Sean McLeish, Haozhe Chen +12Efficient Long-Context Inference Context Length
New York University · University of Maryland · Princeton University +3
Separating Semantic Competition from Context Length in RAG Reading
May 26, 2026Vyzantinos Repantis, Ameya Gawde, Harshvardhan Singh +4Question-Answering Benchmarks Context Length
Meta Platforms
Compress the Context, Keep the Commitments: A Formal Framework for Verifiable LLM Context Compression
May 17, 2026Natalia Trukhina, Vadim VashkelisLarge Language Model Compression Context Length
Embedded Intelligence Lab (EMILAB)
LiteFrame: Efficient Vision Encoders Unlock Frame Scaling in Video LLMs
May 17, 2026Jihwan Kim, Nikhil Parthasarathy, Danfeng Qin +5Vision Encoders Context Length
1Google DeepMind · 2Seoul National University
EndPrompt: Efficient Long-Context Extension via Terminal Anchoring
May 14, 2026Han Tian, Luxuan Chen, Xinran Chen +10Efficient Long-Context Inference Context Length
1Nankai University · 2Baidu Inc. · 3Shanghai Jiao Tong University +1
Classifier Context Rot: Monitor Performance Degrades with Context Length
May 12, 2026Sam Martin, Fabien RogerContext Length Agentic Benchmarks
Anthropic Fellows Program · Anthropic
On Problems of Implicit Context Compression for Software Engineering Agents
May 11, 2026Kirill Gelvan, Igor Slinko, Felix Steinbauer +3Context Length Large Language Model Backbones
JetBrains Research · Technical University of Munich, Germany
RDKV: Rate-Distortion Bit Allocation for Joint Eviction and Quantization of the KV Cache
May 8, 2026Junkai Zhang, Hang Guo, Luca Benini +1Key-Value Cache Compression Key-Value Cache
ETH Zurich · Tsinghua University
Evaluating LLMs on Large-Scale Graph Property Estimation via Random Walks
May 2, 2026Sunil Kumar Maurya, Xin LiuLLM Reasoning Strategies Large Language Model Evaluation
University of Tokyo Tokyo, Japan · AIST Tokyo, Japan
BERAG: Bayesian Ensemble Retrieval-Augmented Generation for Knowledge-based Visual Question Answering
Apr 24, 2026Jinghong Chen, Jingbiao Mei, Guangyu Yang +1Knowledge-Based Visual Question Answering Context Length
Department of Engineering University of Cambridge
GenericAgent: A Token-Efficient Self-Evolving LLM Agent via Contextual Information Density Maximization (V1.0)
Apr 18, 2026Jiaqing Liang, Jinyi Han, Weijia Li +15Large Language Model Agents Self-Evolving Agents
Advantage AI Agent Lab (A3 Lab)
Beyond Sequential Distance: Inter-Modal Distance Invariant Position Encoding
Mar 11, 2026Lin Chen, Bolin Ni, Qi Yang +5Cross-Modal Attention Rotary Position Embedding
1MAIS, Institute of Automation, Chinese Academy of Sciences, China · University of Chinese Academy of Sciences, China · 3Tencent +1
CompLLM: Compression for Long Context Q&A
Sep 23, 2025Gabriele Berton, Jayakrishnan Unnikrishnan, Son Tran +1Efficient Long-Context Inference Large Language Model Compression
Amazon · Center For Research in Computer Vision, University of Central Florida

Sep 30, 2026, cs.AIJ/K move · Enter open · S save

Staying on Task: Testing the Foundations of Long-Horizon Agent Reliability

Jeffrey Willette, Krishna C. Puvvada, Boris Ginsburg

NVIDIA

Long-Horizon Agents Long-Horizon Task Planning Context Length Ledger

Open paper AskPDF arXiv

Long-horizon agentic workflows require models to sustain repeated state-dependent actions all while the context grows, sub-task complexity changes, and new data arrives. Each situation represents an independent axis along which an agent may fail. An agent reconciling a long ledger, for example, must repeatedly read its state, update the correct record, and preserve alignment across thousands of outputs. A model may accept the entire ledger yet lose its place or stop applying the operation consistently as generation proceeds. We introduce Long-Transduction, a controlled diagnostic that tests a model's ability to stay on task during long generation while continuously reading, mutating, and outputting input-context dependent operations such as arithmetic, sorting, variable lookups, and table transformations. Long-Transduction evaluation independently varies local task complexity, input data formatting, and context length isolate failures along each axis. We evaluate seven open-weight models, finding a 62.8% decrease when scaling context length from 4-128K, a 36.5% decrease when varying input format, and a 39.9% decrease by increasing local task complexity. Together, these failures represent critical liabilities in long-horizon agentic workflows.

Momentum

Related topics

Latest papers 28