Long-Context Modeling

Follow this topic Ask about this topic

Momentum

11 papers in the last four weeks, up 83% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 105

All topics

Diagnosing Evidence Utilization in Long-Context and Retrieval-Augmented Language Models under Matched Evidence Conditions
Jun 4, 2026Haizhou XiaLLM Evaluation LLM Grounding
University of Western Ontario, Canada
Learning Long Range Spatio-Temporal Representations over Continuous Time Dynamic Graphs with State Space Models
Jun 3, 2026Ayushman Raghuvanshi, Thummaluru Siddartha Reddy, Sundeep Prabhakar Chepuri +1Long-Context Modeling Temporal Link Prediction
Department of Electrical Communication Engineering, Indian Institute of Science, Bangalore
InfoMem: Training Long-Context Memory Agents with Answer-Conditioned Information Gain
Jun 2, 2026Tiancheng Han, Yong Li, Wuzhou Yu +2Long-Context Modeling RL for LLM Agents
1Tongji University · 2Shanghai Innovation Institute · 3Shanghai AI Laboratory
Why Do Time Series Models Need Long Context Windows?
Jun 1, 2026Luca Butera, Giovanni De Felice, Andrea Cini +1Time Series Forecasting Long-Context Modeling
1Università della Svizzera Italiana · 2EPFL · 3Politecnico di Milano
Connecting the Dots: Benchmarking Reflective Memory in Long-Horizon Dialogue
May 31, 2026Jingjie Lin, Bingbing Wang, Zihan Wang +4Memory-Augmented Language Models Long-Context Modeling
Harbin Institute of Technology, Shenzhen · The Hong Kong Polytechnic University · Fudan University +1
AnyEdit++: Adaptive Long-Form Knowledge Editing via Bayesian Surprise
May 31, 2026Bowen Tian, Caixue He, Jiemin Wu +4Knowledge Editing Long-Context Modeling
The Hong Kong University of Science and Technology (Guangzhou), Guangzhou 511400, China · Southeast University, Nanjing, China · Knowin AI, Shenzhen, China +2
SHARP: Sleep-based Hierarchical Accelerated Replay for Long Range Non-Stationary Temporal Pattern Recognition
May 30, 2026Jayanta Dey, Shikhar Srivastava, Itamar Lerner +2Memory-Augmented Neural Networks Continual Learning
University of Texas at San Antonio, USA · University of Rochester, USA
Multi-Mixer Models: Flexible Sequence Modeling with Shared Representations
May 27, 2026Kevin Y. Li, Asher Trockman, Ananda Theertha Suresh +1Transformer Attention Long-Context Modeling
Carnegie Mellon University · Google Research
ATLAS: All-round Testing of Long-context Abilities across Scales
May 27, 2026Deli Huang, Cunguang Wang, Hongyin Tang +15LLM Evaluation Benchmark Construction
Meituan · Fudan University
VoiceGiraffe: A Benchmark for Extreme Long-Context Audio-Language Understanding
May 27, 2026Jashin Ye, Dongxiao Wang, Yixuan Ye +10Audio-Language Model Evaluation Audio Understanding
Future Living Lab, Alibaba
Tensor Memory: Fixed-Size Recurrent State for Long-Horizon Transformers
May 26, 2026Kabir Swain, Sijie Han, Daniel Karl I. Weidele +2Memory-Augmented Neural Networks Memory-Augmented Language Models
Massachusetts Institute of Technology, Cambridge, MA, USA · University of Toronto, Toronto, Canada · IBM Research, Cambridge, MA, USA
ACC: Compiling Agent Trajectories for Long-Context Training
May 21, 2026Qisheng Su, Zhen Fang, Shiting Huang +8Supervised Fine-Tuning Long-Context Modeling
MoE Key Lab of BIPC, University of Science and Technology of China · Shanghai Innovation Institute · Shanghai AI Laboratory
GoLongRL: Capability-Oriented Long Context Reinforcement Learning with Multitask Alignment
May 19, 2026Minxuan Lv, Tiehua Mei, Tanlong Du +9Reinforcement Learning Long-Context Modeling
Kuaishou Technology · University of Chinese Academy of Sciences
DashAttention: Differentiable and Adaptive Sparse Hierarchical Attention
May 18, 2026Yuxiang Huang, Nuno M. T. Gonçalves, Federico Alvetreti +5Self-Attention Long-Context Modeling
1Tsinghua University · 2Instituto Superior Técnico, Universidade de Lisboa. · 3Instituto de Telecomunicações. +6
Towards Generalization of Block Attention via Automatic Segmentation and Block Distillation
May 15, 2026Shuaiyi Li, Zhisong Zhang, Yan Wang +5Long-Context Modeling LLM Inference Acceleration
The Chinese University of Hong Kong · City University of Hong Kong · Tencent +2
Training Long-Context Vision-Language Models Effectively with Generalization Beyond 128K Context
May 13, 2026Zhaowei Wang, Lishu Luo, Haodong Duan +9Visual Question Answering Vision-Language Models
1CSE Department, HKUST · 2ByteDance Seed · *Work done at ByteDance Seed
LongBEL: Long-Context and Document-Consistent Biomedical Entity Linking
May 13, 2026Adam Remaki, Xavier Tannier, Christel GérardinLong-Context Modeling Entity Linking
Sorbonne Université, Inserm, Université Sorbonne Paris Nord, Limics, 75006 Paris, France · Service de médecine interne, Hôpital Tenon, Assistance Publique - Hôpitaux de Paris, Paris, France
A Unified Framework for Critical Scaling of Inverse Temperature in Self-Attention
May 12, 2026Tomohiro Hayase, Ryo KarakidaSoftmax Attention Self-Attention
AIST
Classifier Context Rot: Monitor Performance Degrades with Context Length
May 12, 2026Sam Martin, Fabien RogerLanguage Model Safety Evaluation Long-Context Language Model Evaluation
Anthropic Fellows Program · Anthropic
EHR-RAGp: Prototype-Guided Retrieval of Longitudinal Electronic Health Records for Clinical Prediction Models
May 12, 2026Saeed Shurrab, Mariam Al-Omari, Dana El Samad +1Long-Context Modeling Clinical Prediction
New York University · New York University Abu Dhabi
CLEF: EEG Foundation Model for Learning Clinical Semantics
May 11, 2026Peng Cao, Ali Mirzazadeh, Jong Woo Lee +2Electroencephalography Long-Context Modeling
MIT CSAIL · Brigham and Women’s Hospital, Harvard Medical School · Massachusetts General Hospital, Harvard Medical School
Route Before Retrieve: Activating Latent Routing Abilities of LLMs for RAG vs. Long-Context Selection
May 11, 2026Yiwen Chen, Kuan Li, Fuzhen Zhuang +6LLM Prompting Long-Context Modeling
Beihang University · HKUST · Alibaba Group +1
FRACTAL: SSM with Fractional Recurrent Architecture for Computational Temporal Analysis of Long Sequences
May 9, 2026Mengqi Li, Wensheng Lin, Jinshuai Yang +1Long-Context Modeling Multiscale Time-Series Modeling
School of Electronics and Information, Northwestern Polytechnical University, Xi’an, Shaanxi, China. · DecoreX Intelligent Technologies Co., Ltd., Xi’an 710075 China.
Scaling Limits of Long-Context Transformers
May 8, 2026Giuseppe Bruno, Shi Chen, Zhengjiang Lin +2Softmax Attention Self-Attention
Adaptive Memory Decay for Log-Linear Attention
May 7, 2026Yaxita Amin, Helen Zichen Li, Mengfan Zhang +1Long-Context Modeling Linear Attention
University of Maryland
Cubit: Token Mixer with Kernel Ridge Regression
May 7, 2026Chuanyang Zheng, Jiankai Sun, Yihang Gao +6Kernel Ridge Regression Transformer Attention
MiA-Signature: Approximating Global Activation for Long-Context Understanding
May 7, 2026Yuqing Li, Jiangnan Li, Mo Yu +3Long-Context Modeling Context Compression
Institute of Information Engineering, Chinese Academy of Sciences · School of Cyber Security, University of Chinese Academy of Sciences · Pattern Recognition Center, WeChat AI, Tencent +1
The Impossibility Triangle of Long-Context Modeling
May 6, 2026Yan ZhouLong-Context Modeling Associative Memory
School of Mathematics and Statistics Changsha University of Science and Technology Changsha, Hunan 410114, China
Retrieval Mechanisms Surpass Long-Context Scaling in Time Series Forecasting
May 6, 2026Rishi Ahuja, Kumar Prateek, Simranjit Singh +1Time Series Forecasting Long-Context Modeling
Department of Information Technology, Dr. B.R. Ambedkar National Institute of Technology Jalandhar, Punjab, 144008, India.
Mapping Text to Multiplex Graph: Prompt Compression as Lévy Walk-Guided Graph Pruning
May 4, 2026Yaxin Gao, Yao Lu, Jinhong Deng +7LLM Compression Long-Context Modeling

Jun 4, 2026, cs.CLJ/K move · Enter open · S save

Diagnosing Evidence Utilization in Long-Context and Retrieval-Augmented Language Models under Matched Evidence Conditions

Haizhou Xia

University of Western Ontario, Canada

LLM Evaluation LLM Grounding Long-Context Modeling RAG Evaluation

Open paper AskPDF arXiv

Final-answer accuracy, retrieval recall, and citation overlap do not reveal how much answer advantage a long-context or retrieval-augmented language model actually recovers from supplied evidence. A model may answer from parametric priors, fail to use evidence that is present, or cite relevant text without converting it into the final answer. This paper introduces a four-condition diagnostic protocol for evidence-utilization evaluation under matched examples, models, prompts, and scoring rules. The protocol compares no-evidence, full-context, retrieved-evidence, and oracle-evidence reference conditions, and uses Oracle-Reference Normalized Context Utilization (ONCU) as a denominator-valid estimate of recovered oracle-reference evidence advantage. The empirical study evaluates five local open-weight models from the Qwen, Gemma, Llama, and Mistral families over Controlled-ONCU-safe16K, HotpotQA-ONCU, and 2WikiMultiHopQA-ONCU, comprising 18,000 ONCU-compatible predictions. Results show a task-dependent diagnostic pattern: controlled synthetic settings expose reduced recovery when the same evidence is embedded in long input rather than supplied compactly, while realistic multi-hop reconstructions show that full-context inputs outperform the tested retrieved inputs in denominator-free answer and evidence metrics, with ONCU supporting the same direction on oracle-improving groups. Sensitivity audits with stronger retrieval settings narrow some gaps but do not overturn the scoped interpretation. The main contribution is therefore not a single utilization ratio, but a matched diagnostic protocol that separates no-evidence answerability, oracle-evidence recoverability, full-context recovery, retrieval-conditioned recovery, denominator validity, and companion answer/evidence diagnostics.

Momentum

Related topics

Latest papers 105