Long-Context VLMs

VLM: Vision-Language Model

Follow this topic Ask about this topic

Momentum

6 papers in the last four weeks, against 2 the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 19

All topics

ReMem: Streaming Video Understanding With Long Context Retention
Oct 5, 2026Li Yiheng, He Xu, Wang Shaobo +2Memory-Augmented VLMs Streaming Video Understanding
Nanyang Technological University · Shanghai Jiaotong University · University of the Chinese Academy of Sciences
VidHarness: Evolving Agent Harnesses for Cost-Efficient Long Video Understanding
Sep 29, 2026Susan Liang, Jianmin Wu, Daxiang DongLong-Video Understanding LLM Agent Harnesses
Baidu, Inc.
FocusVTC: Efficient and High-Performance Visual Text Compression with Adaptive Resolution
Sep 29, 2026FangZhi Zhong, Xuerui Qiu, Yuqi Pan +4Efficient VLM Inference VLM Reasoning
Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · Zhongguancun Academy +1
An Empirical Study of VLM Pipelines for Long-Document QA
Sep 24, 2026Kenan E. Ak, Jay Mohta, Gwang Gook Lee +2VLM Evaluation Tool Use in VLMs
PREM: Prefix-Steered Recurrent Memory for Long-Video Understanding
Sep 20, 2026Siru Zhong, Qiongyan Wang, Xiaohui Lv +5Memory-Augmented VLMs Recurrent Neural Networks
The Hong Kong University of Science and Technology (Guangzhou) · Beta Infinity · Tsinghua University
Video-HolmesV2: Can MLLMs Reason with Spatio-Temporal Audio-Visual Evidence in Long Videos?
Sep 15, 2026Zhaoyang Wei, Zipeng Wang, Yushe Cao +10Audio-Visual Understanding Video-Language Model Evaluation
University of Chinese Academy of Sciences, China · Tsinghua University, China · Tencent, China
VideoScout: Learning Agentic Active Exploration with Adaptive Reasoning Pacing for Long Video Understanding
Sep 14, 2026Weixin Xu, Zhenyu Yang, Bing Wang +2Video Understanding Long-Video Understanding
VideoHarness-RSI: Recursive Harness Self-Improvement for Long-Video Understanding with Frozen Vision-Language Models
Aug 25, 2026Guoyang Xu, Hao ChenProgram Synthesis Long-Video Understanding
Tencent
LongEarth-R1: Benchmarking and Aligning Vision-Language Models for Long-Horizon Earth Observation Reasoning
Aug 13, 2026Yupan Ding, Jing Xiao, Zhenyuan Zhang +4Remote Sensing Image Understanding Remote Sensing VQA
School of Artificial Intelligence, Wuhan University, Wuhan, China · School of Computer Science, Wuhan University, Wuhan, China · Xi’an University of Electronic Science and Technology, Xi’an, China +1
VLZip: Unified Visual and Textual Compression for Interleaved Long-Context Modeling
Aug 9, 2026Yuqi Zhang, Cheng Chen, Yuyu Guo +6Vision-Language Models Efficient VLM Inference
Shanghai Key Lab of Intell. Info. Processing, School of CS, Fudan University, China · Shanghai Innovation Institute, China · Ant Group, China
Linear Multi-Timescale Retention as a Memory-Efficient Vision-Language Bridge
Aug 3, 2026Ashfak Yeafi, Mehedi Hasan, Md Khairul IslamMemory-Augmented VLMs Vision-Language Models
Department of Electrical and Electronic Engineering Khulna University of Engineering & Technology Khulna-9203, Bangladesh · Department of Computer Science and Engineering Brac University, Dhaka, Bangladesh · Department of Mathematics and Computer Science Hobart and William Smith Colleges Geneva, NY, USA
ReToken: Improving Long-Context VLMs with Visual Retrieval Token
Jul 30, 2026Yao Xiao, Reuben Tan, Zhen Zhu +3Vision-Language Models Vision-Language Retrieval
University of Illinois at Urbana-Champaign · Microsoft Research · Google DeepMind
HyFL-CLIP: Hyperbolic Fine-Tuning of CLIP for Robust Long-Context Understanding
Jul 1, 2026Ji Ha Jang, Hayeon Kim, Chulwon Lee +2VLM Robustness Vision-Language Models
1Dept. of Electrical and Computer Engineering, 2IPAI, 3INMC & AIIS, Seoul National University, Republic of Korea
Can Retrieval Heads See Images? Multimodal Retrieval Heads in Long-Context Vision-Language Models
May 26, 2026Aaron Branson Cigres Li, Zhaowei Wang, Yu Zhao +9Vision-Language Models VLM Interpretability
1HKUST · University of Edinburgh · 4CUHK +2
MemLens: Benchmarking Multimodal Long-Term Memory in Large Vision-Language Models
May 14, 2026Xiyu Ren, Zhaowei Wang, Yiming Du +11VLM Evaluation Memory-Augmented VLMs
CSE Deparment, HKUST · CUHK · OmniMemory (Shenzhen) Intelligent Technology Co., Ltd. +1
Training Long-Context Vision-Language Models Effectively with Generalization Beyond 128K Context
May 13, 2026Zhaowei Wang, Lishu Luo, Haodong Duan +9Visual Question Answering Vision-Language Models
1CSE Department, HKUST · 2ByteDance Seed · *Work done at ByteDance Seed
Internalized Reasoning for Long-Context Visual Document Understanding
Mar 31, 2026Austin VeselkaLong-Context Modeling VLM Reasoning
LightOn
How to Train Your Long-Context Visual Document Model
Feb 16, 2026Austin VeselkaVisual Question Answering Vision-Language Models
LightOn
Ambient @ EgoLongQA 2026: Distilling Long-Video perception into a Sub-2B Model
Date pendingLogesh Kumar UmapathiEfficient VLM Inference Egocentric Video QA
Team Ambient

Oct 5, 2026, cs.CVJ/K move · Enter open · S save

ReMem: Streaming Video Understanding With Long Context Retention

Li Yiheng, He Xu, Wang Shaobo, Shao Ling+1

Nanyang Technological University · Shanghai Jiaotong University · University of the Chinese Academy of Sciences

Memory-Augmented VLMs Streaming Video Understanding VLM Adaptation Memory-Augmented Video Understanding Long-Context VLMs

Open paper AskPDF arXiv

Despite their impressive performance on a wide range of video understanding tasks, current Vision Language Models (VLMs) are predominantly designed for offline scenarios and struggle to handle online streaming videos that demand low latency response. Several studies have explored memory and token compression strategies in an attempt to adapt offline VLMs for streaming video understanding tasks. However, through our probing experiment, we identify that most existing works tend to progressively lose long context information as length of input stream increases. To address this, we propose ReMem, a novel training-free adaptation technique that enables VLMs to process streaming videos of arbitrary lengths while improving their long context information retention capability. ReMem exploits memory from two perspectives, implemented as two core components. The Streaming Context Memory (SCM) continuously compresses historical context with query-independent attention. The Retrieved Vision Memory (RVM) then retrieves the most salient, query-relevant context from memory to augment the VLM's input. Comprehensive experiments demonstrate that the proposed ReMem achieves state-of-the-art (SOTA) performance across a variety of widely used benchmarks, spanning both streaming video and general long video understanding tasks.

Momentum

Related topics

Latest papers 19