Long-Context Inference

Follow this topic Ask about this topic

Momentum

2 papers in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 40

All topics

UNREAL: Unifying Retrieval and Long-Context with a Single Model
Oct 6, 2026Edan Kinderman, Elad Hoffer, Yochai Blau +4Retrieval-Augmented Generation Knowledge Augmentation for Language Models
NVIDIA · Technion
LongHarness Bench: Stress-Testing Language Model Harnesses for Long-Context Reasoning
Sep 29, 2026Quang Hieu Pham, Thuy Duong Nguyen, Jocelyn Qiaochu Chen +1Long-Context Language Model Evaluation Long-Context Language Model Inference
University of Alberta · Alberta Machine Intelligence Institute (Amii)
Realize What Matters: Principled Context Representation for Large-Scale Reasoning
Sep 23, 2026Michael Theologitis, Dean Light, Shuyue Stella Li +3Long-Context Inference LLM Context Management
University of Washington
Trace as State: Reasoning Traces as Conditional States for Long-Context Transformers
Sep 2, 2026Xu Zou, Jie TangLong-Context Inference CoT Reasoning
Tsinghua University
Sliding-window beats linear attention
Aug 28, 2026Alexia Jolicoeur-Martineau, Rhea Sanjay Sukthanker, Pashmina Cameron +1KV Caching LLM Inference Acceleration
Microsoft Applied Sciences Group (ASG) · Independent
Relevant but Incomplete: Referential Dangling as a Paradigm-Level Failure Mode in Hard Prompt Compression
Aug 5, 2026Zhengpei Hu, Kai Li, Dapeng Fu +5LLM Compression Long-Context Inference
School of Computer Technology and Application, Qinghai University · Tsinghua University · Ant Group Security and Intelligence Laboratory (SIL)
ResKV: Reconstructing Omitted Attention Contributions for Fixed-Budget KV Cache Compression
Jul 31, 2026Yuhang Zhan, Lisi Chen, Shuo ShangSoftmax Attention KV Caching
University of Electronic Science and Technology of China
C $^2$ KV: Compressed and Composable KV Cache Reuse for Efficient LLM Inference
Jul 20, 2026Chuheng Du, Junyi Chen, Hanlin Tang +7KV Caching LLM Inference Acceleration
School of Computer Science Shanghai Jiao Tong University Shanghai, China · Alibaba Group Hangzhou, China
RECON: Benchmarking Agent Memory for Compositional Reasoning over Long Contexts
Jul 18, 2026Mihir Shriniwas AryaCompositional Reasoning LLM Agent Memory
Department of Computer Science and Engineering RV College of Engineering
Self-Guided Test-Time Training for Long-Context LLMs
Jul 10, 2026Xinyu Zhu, Zhe Xu, Xiaohan Wei +10Long-Context Language Modeling Test-Time Training
Meta AI · University of Virginia
WILDTRACE: Benchmarking Natural Evidence Trails in Long-Context Reasoning
Jul 10, 2026Zixin Chen, Peng Liu, Haobo Li +7Evidence-Grounded Reasoning Long-Context QA
Hong Kong University of Science and Technology · Qwen Team, Alibaba Group · *Work done during internship at Qwen Team
Understanding Axes of Difficulty For Long Context Tasks Via PredicateLongBench
Jul 9, 2026Siddhartha Jain, Ameya VelingkerLLM Evaluation Long-Context Language Model Evaluation
NVIDIA
The Key to Going Linear: Analysis-Driven Transformer Linearization
Jul 8, 2026Anna Kuzina, Paul N. Whatmough, Babak Ehteshami BejnordiSoftmax Attention LLM Inference Acceleration
Qualcomm AI Research∗ · Qualcomm AI Research
ReContext: Recursive Evidence Replay as LLM Harness for Long-Context Reasoning
Jul 2, 2026Yanjun Zhao, Ruizhong Qiu, Tianxin Wei +6Long-Context Inference
University of Illinois Urbana-Champaign
Evidence-State Rewards for Long-Context Reasoning
Jul 2, 2026Ya Gao, Pekka MarttinenProcess Reward Models RL for Language Model Reasoning
1Aalto University
Information-Aware KV Cache Compression for Long Reasoning
Jun 25, 2026Jushi Kai, Zhuiri Xiao, Alexandra Birch +1KV Caching Long-Context Language Model Inference
LUMIA Lab, School of Artificial Intelligence, Shanghai Jiao Tong University · Shanghai Jiao Tong University · School of Informatics, University of Edinburgh
Randomized YaRN Improves Length Generalization for Long-Context Reasoning
Jun 22, 2026Manas Mehta, Fangcong Yin, Greg DurrettLong-Context Language Modeling LLM Fine-Tuning
New York University
Beyond Reward Engineering: A Data Recipe for Long-Context Reinforcement Learning
Jun 17, 2026Xiaoyue Xu, Sikui Zhang, Xiaorong Wang +2Training Data Curation RL for Language Model Reasoning
OpenBMB · Tsinghua University
Recursive Agent Harnesses
Jun 11, 2026Elias Lumer, Sahil Sen, Kevin Paul +1Tool-Augmented Language Model Agents LLM Agent Orchestration
PricewaterhouseCoopers, U.S.
HMARS: A Hierarchical Multi-Agent Memory System for Long-Context Reasoning
Jun 3, 2026Zeju Li, Ziyang Zheng, Yizhou Zhou +1Agent Memory Multi-Agent LLM Systems
The Chinese University of Hong Kong · Independent Researcher
LongAttnComp: Cross-Family Context Compression for Long-Context Reasoning
May 31, 2026Mengmeng Ji, Ravi Shanker Raju, Jonathan Lingjie Li +1LLM Compression LLM Fine-Tuning
SambaNova Systems, Inc. San Jose, CA, USA
LongTraceRL: Learning Long-Context Reasoning from Search Agent Trajectories with Rubric Rewards
May 29, 2026Nianyi Lin, Jiajie Zhang, Lei Hou +1LLM Grounding RL for Language Model Reasoning
Tsinghua University
Thinking as Compression: Your Reasoning Model is Secretly a Context Compressor
May 27, 2026Guoxin Ma, Yibing Liu, Chengzhengxu Li +7LLM Compression Long-Context Inference
Baidu Inc., Beijing, China · Xi’an Jiaotong University · City University of Hong Kong +1
Adaptive Mass-Segmented KV Compression for Long-Context Reasoning
May 22, 2026Junzhe Yang, Xiaoyu ShenKV Caching Long-Context Language Model Inference
Shanghai Jiao Tong University · Institute of Digital Twin, Eastern Institute of Technology, Ningbo
ACC: Compiling Agent Trajectories for Long-Context Training
May 21, 2026Qisheng Su, Zhen Fang, Shiting Huang +8Supervised Fine-Tuning Long-Context Modeling
MoE Key Lab of BIPC, University of Science and Technology of China · Shanghai Innovation Institute · Shanghai AI Laboratory
PEEK: Context Map as an Orientation Cache for Long-Context LLM Agents
May 19, 2026Zhuohan Gu, Qizheng Zhang, Omar Khattab +1LLM Agent Memory Long-Context Inference
MIT CSAIL · Stanford University
KV-Fold: One-Step KV-Cache Recurrence for Long-Context Inference
May 12, 2026Alireza Nadali, Patrick Cooper, Ashutosh Trivedi +1Transformer Inference KV Caching
Department of Computer Science University of Colorado Boulder Boulder, Colorado, 80301
Training-Inference Consistent Segmented Execution for Long-Context LLMs
May 12, 2026Xianpeng Shang, Jiang Li, Zehua Duo +2Self-Attention Long-Context Language Modeling
College of Computer Science, Inner Mongolia University, Hohhot 010021, China · National & Local Joint Engineering Research Center of Intelligent Information Processing Technology for Mongolian, Hohhot 010021, China · Inner Mongolia Key Laboratory of Multilingual Artificial Intelligence Technology, Hohhot 010021, China +1
The First Drop of Ink: Nonlinear Impact of Misleading Information in Long-Context Reasoning
May 11, 2026Muhan Gao, Zih-Ching Chen, Kuan-Hao HuangLong-Context Inference Language Model Robustness
Department of Computer Science and Engineering, Texas A&M University, College Station, TX, USA · NVIDIA AI Technology Center, NVIDIA Corporation, Santa Clara, CA, USA
MemReread: Enhancing Agentic Long-Context Reasoning via Memory-Guided Rereading
May 11, 2026Baibei Ji, Xiaoyang Weng, Juntao Li +3Long-Context Inference Memory-Augmented Language Model Agents
Soochow University · Peking University

Oct 6, 2026, cs.CLJ/K move · Enter open · S save

UNREAL: Unifying Retrieval and Long-Context with a Single Model

Edan Kinderman, Elad Hoffer, Yochai Blau, Brian Chmiel+3

NVIDIA · Technion

Retrieval-Augmented Generation Knowledge Augmentation for Language Models LLM Inference Acceleration Information Retrieval Long-Context Inference

Open paper AskPDF arXiv

Long-context inference and Retrieval-Augmented Generation (RAG) handle evidence selection at vastly different scales, from a single long prompt to an entire corpus. We ask whether a single model-internal mechanism can select evidence across this range. We introduce UNifying REtrieval And Long-Context with a Single Model (UNREAL), a model-native evidence selection framework to span corpus retrieval and long-context inference. UNREAL encodes chunks and derives retrieval queries directly from the frozen LLM's internal representations. It adds fewer than 500K trainable parameters and leaves the backbone unchanged. On a 3B-token, 21M-chunk Wikipedia index, all four dense and hybrid UNREAL backbones outperform state-of-the-art retriever-reranker systems. The best model raises recall from 49.1% to 73.2% on HotpotQA, from 31.7% to 60.1% on 2WikiMultiHopQA, and from 8.8% to 14.4% on MuSiQue. Applied to long-context tasks, the same selection mechanism removes distractors before generation, raising NoLiMa accuracy from 1.0% to 24.83% at its maximum context length of 128K tokens, and LV-Eval's F1 score from 49.97% to 54.66% at 256K. UNREAL also reduces FLOPs and time-to-first-token relative to full-context inference from roughly 32K tokens onward, with larger gains as context grows. Together, these results establish model-internal evidence selection as a common foundation for corpus retrieval and evidence-sparse long-context inference.

Momentum

Related topics

Latest papers 40