Vision-Language Models

Also known as VLM

Follow this topic Ask about this topic

Momentum

55 papers in the last four weeks, up 120% on the four weeks before. 0.5% of all new papers.

Jul 13Week of Sep 28

Latest papers 491

All topics

ID-Align: RoPE-Conscious Position Remapping for Dynamic High-Resolution Adaptation in Vision-Language Models
May 27, 2025Bozhou Li, Wentao ZhangRotary Positional Embeddings Vision-Language Models
Peking University Beijing, China
LaViDa: A Large Diffusion Language Model for Multimodal Understanding
May 22, 2025Shufan Li, Konstantinos Kallidromitis, Hritik Bansal +7Vision-Language Models Efficient VLM Inference
1UCLA · 2Panasonic AI Research · 4Salesforce Research +1
Mitigating Hallucinations via Inter-Layer Consistency Aggregation in Large Vision-Language Models
May 18, 2025Kai Tang, Jinhao You, Yichen Guo +8Vision-Language Models Hallucination in Language Models
Peking University · University of Pennsylvania · Nanyang Technological University +4
Concept-as-Tree: A Controllable Synthetic Data Framework Makes Stronger Personalized VLMs
Mar 17, 2025Ruichuan An, Kai Zeng, Ming Lu +5Synthetic Data Augmentation Vision-Language Models
1Peking University · 2CUHK MMLab · 3Zhongguancun Academy +1
REVEAL: Robust Evolution of Vision-Language Models for Explainable AI-Video Detection
Feb 20, 2025Yun-Yun Tsai, Qingyuan Liu, Ruijian Zha +4Tool Use in VLMs Vision-Language Models
Columbia University · Rutgers University
Enhanced Vision-Language Models for Diverse Sensor Understanding: Cost-Efficient Optimization and Benchmarking
Dec 30, 2024Sangyun Chung, Youngjoon Yu, Se Yeon Kim +2VLM Evaluation Vision-Language Models
Integrated Vision Language Lab., School of Electrical Engineering, Korea Advanced Institute of Science and Technology (KAIST), 291 Daehak-ro, Yuseong-gu, Daejeon, 34141, Republic of Korea
VLMGuard: Bootstrapping Malicious Prompt Detectors from Unlabeled Vision-Language Prompts in the Wild
Oct 1, 2024Junlin Fang, Wenyu Chen, Reshmi Ghosh +7VLM Robustness Vision-Language Models
College of Computing and Data Science Nanyang Technological University · School of Physical and Mathematical Sciences Nanyang Technological University · Microsoft Corp. +1
Prompting with Sign Parameters for Low-resource Sign Language Instruction Generation
Date pendingMd Tariquzzaman, Md Farhan Ishmam, Saiyma Sittul Muna +2VLM Evaluation Vision-Language Models
Department of Computer Science and Engineering Islamic University of Technology, Bangladesh
CLIP-RD: Relational Distillation for Efficient CLIP Knowledge Distillation
Date pendingJeannie Chung, Hanna Jang, Ingyeong Yang +2Cross-Modal Learning Cross-Modal Knowledge Distillation
Ewha Womans University, Seoul 03760, South Korea
TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
Date pendingPeng Cai, Zhaofan Zou, Shifa Liu +7Vision-Language Models Document Parsing
China Telecom Artificial Intelligence Technology (Beijing) Co., Ltd.
StreamTTT: Reconciling Real-Time Perception and Long-Term Memory in Streaming VLMs
Date pendingJoya Chen, Zeyun Zhong, Mike Zheng ShouMemory-Augmented VLMs Streaming Video Understanding
National University of Singapore · Karlsruhe Institute of Technology

May 27, 2025, cs.CVJ/K move · Enter open · S save

ID-Align: RoPE-Conscious Position Remapping for Dynamic High-Resolution Adaptation in Vision-Language Models

Bozhou Li, Wentao Zhang

Peking University Beijing, China

Rotary Positional Embeddings Vision-Language Models VLM Adaptation Positional Encoding

Open paper AskPDF arXiv

Currently, a prevalent approach for enhancing Vision-Language Models (VLMs) performance is to encode both the high-resolution version and the thumbnail of an image simultaneously. While effective, this method generates a large number of image tokens. When combined with the widely used Rotary Position Embedding (RoPE), its long-term decay property hinders the interaction between high-resolution tokens and thumbnail tokens, as well as between text and image. To address these issues, we propose ID-Align, which alleviates these problems by reordering position IDs. In this method, high-resolution tokens inherit IDs from their corresponding thumbnail token while constraining the overexpansion of positional indices. Our experiments conducted within the LLaVA-Next framework demonstrate that ID-Align achieves significant improvements, including a 6.09% enhancement on MMBench's relation reasoning tasks and notable gains across multiple benchmarks. Our code is available at the following link: https://github.com/zooblastlbz/ID-Align.

Momentum

Related topics

Latest papers 491