Multimodal

Follow this field Ask about this field

Recent momentum

emerging

0 papers in the last 28 days · 0.0% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this field, kept on the site without email delivery.

Period ending 2026-09-21

65 new papers

A weekly snapshot of new work published in Multimodal.

Period ending 2026-09-14

18 new papers

A weekly snapshot of new work published in Multimodal.

Period ending 2026-09-07

26 new papers

A weekly snapshot of new work published in Multimodal.

Inside this field

Focused directions

Focused topic

Multimodal Learning

Within Multimodal

456 papersExplore

Focused topic

Cross-Modal Learning

Within Multimodal

432 papersExplore

Focused topic

Multi-Modal Large Language Models

Within Multimodal

376 papersExplore

Focused topic

Multimodal Reasoning

Within Multimodal

224 papersExplore

Focused topic

Multimodal Benchmarks

Within Multimodal

171 papersExplore

Focused topic

Unified Multimodal Models

Within Multimodal

100 papersExplore

Focused topic

Cross-Modal Attention

Within Multimodal

75 papersExplore

Focused topic

Input Modality

Within Multimodal

73 papersExplore

Focused topic

Modality Gap

Within Multimodal

73 papersExplore

Focused topic

Multimodal Generation

Within Multimodal

73 papersExplore

Focused topic

Cross-Modal Alignment

Within Multimodal

66 papersExplore

Focused topic

Video Multimodal Large Language Models

Within Multimodal

58 papersExplore

1,965 papers

Latest in Multimodal

Browse all topics →

Discrete Tokenization for Multimodal LLMs: A Comprehensive Survey
Jul 21, 2025Jindong Li, Yali Fu, Jiahong Liu +5Calibration-Free Low-Bit Vector Quantization Large Language Model Quantization
ADMC: Attention-based Diffusion Model for Missing Modalities Feature Completion
Jul 8, 2025Yuhan Li, Wei Zhang, Juan Chen +3Missing Modalities Cross-Modal Learning
Region-Aware Multimodal Large Language Model via SlowFast Tokenization and Pseudo-Mask Guidance for 3D CT Report Generation
Jun 29, 2025Sunggu Kyung, Jinyoung Seo, Hyunseok Lim +7Radiology Report Generation Computed Tomography Dataset
Anatomy-Grounded Weakly Supervised Prompt Tuning for Chest X-ray Latent Diffusion Models
Jun 12, 2025Konstantinos Vilouras, Ilias Stogiannidis, Junyu Yan +2Chest Radiography Chest X-Ray
Evidence-Grounded Trustworthy Multimodal Reasoning and Evaluation Benchmark in Complex Urban Scenes
Jun 11, 2025Zhaoyang Wei, Bowen Jiang, Xumeng Han +6Multimodal Reasoning Visual Reasoning
HunyuanVideo-HOMA: Generic Human-Object Interaction in Multimodal Driven Human Animation
Jun 10, 2025Ziyao Huang, Zixiang Zhou, Juan Cao +9Articulated Objects Controllable Video Generation
Multimodal Representation Alignment for Cross-modal Information Retrieval
Jun 10, 2025Fan Xu, Luis A. LeivaMultimodal Alignment Multimodal Retrieval
Lingshu: A Generalist Foundation Model for Unified Multimodal Medical Understanding and Reasoning
Jun 8, 2025Weiwen Xu, Hou Pong Chan, Long Li +15Multimodal Clinical Data Multi-Modal Large Language Models
Donate or Create? Comparing Data Collection Strategies for Emotion-labeled Multimodal Social Media Posts
May 30, 2025Christopher Bagdon, Aidan Combs, Carina Silberer +1Emotion Recognition Social Media
Multiplicity is an Inevitable and Inherent Challenge in Multimodal Learning
May 26, 2025Sanghyuk Chun, Olga RussakovskyRobust Multimodal Learning Cross-Modal Learning
Dimple: Discrete Diffusion Multimodal Large Language Model with Parallel Decoding
May 22, 2025Runpeng Yu, Xinyin Ma, Xinchao WangDiffusion Language Models Multi-Modal Large Language Models
Explainable embeddings with Distance Explainer
May 21, 2025Christiaan Meijer, E. G. Patrick BosExplainable Artificial Intelligence Xai
The P $^3$ Dataset: Pixels, Points and Polygons for Multimodal Building Vectorization
May 21, 2025Raphael Sulzer, Liuyun Duan, Nicolas Girard +1Light Detection And Ranging Point Clouds Real3D-Ad Datasets
A quantitative analysis of semantic information in deep representations of text and images
May 21, 2025Santiago Acevedo, Andrea Mascaretti, Riccardo Rende +3Cross-Modal Learning Deepseek
HumaniBench: A Human-Centric Framework for Large Multimodal Models Evaluation
May 16, 2025Shaina Raza, Aravind Narayanan, Vahid Reza Khazaie +6Multimodal Benchmarks Human-Annotated Benchmark
Decoding the Multimodal Mind: Generalizable Brain-to-Text Translation via Multimodal Alignment and Adaptive Routing
May 15, 2025Chunyu Ye, Yunhao Zhang, Jingyuan Sun +3Brain-Computer Interface Multimodal Alignment
Purify then Guide: Rethinking Domain Generalization for Multimodal Face Anti-Spoofing
May 14, 2025Yingjie Ma, Xun Lin, Zitong Yu +7Face Forgery Detection Multimodal Domain Generalization
Multimodal Language Models as Text-to-Image Model Evaluators
May 1, 2025Jiahui Chen, Candace Ross, Reyhane Askari-Hemmat +5Text-To-Image Evaluation Benchmarks
Q-SiT: Teaching LMMs for Image Quality Scoring and Interpreting
Mar 12, 2025Zicheng Zhang, Haoning Wu, Ziheng Jia +2Image Quality Assessment Multimodal Evaluation
Token Communications: A Large Model-Driven Framework for Cross-modal Context-aware Semantic Communications
Feb 17, 2025Li Qiao, Mahdi Boloursaz Mashhadi, Zhen Gao +3Semantic Communications Token Communications
Histopathology Multi-modal Embedding for Pathology Composed Retrieval
Feb 11, 2025Qifeng Zhou, Wenliang Zhong, Thao M. Dang +4Pathology Foundation Models Pathology
A Systematic Survey of Semantic Role Labeling in the Era of Pretrained Language Models
Feb 9, 2025Huiyao Chen, Meishan Zhang, Jing Li +4Sentence-Level Classification Multi-Modal Large Language Models
Divide and Conquer: A Hybrid Strategy Defeats Multimodal Large Language Models
Dec 21, 2024Yanxu Mao, Peipei Liu, Tiehan Cui +3Large Language Model Jailbreaks Adversarial Jailbreaks
Deepfake Media Generation and Detection in the Generative AI Era: A Survey and Outlook
Nov 29, 2024Florinel-Alin Croitoru, Andrei-Iulian Hiji, Vlad Hondru +7Deepfake Detection Multimodal Benchmarks
MLLM-LLaVA-FL: Multimodal Large Language Model Assisted Federated Learning
Sep 9, 2024Jianyi Zhang, Hao Frank Yang, Ang Li +5Multi-Modal Large Language Models Cross-Modal Learning
VLMEvalKit: An Open-Source Toolkit for Evaluating Large Multi-Modality Models
Jul 16, 2024Haodong Duan, Xinyu Fang, Junming Yang +41Multimodal Benchmarks Evaluation Benchmarks
V-Retrver: Evidence-Driven Agentic Reasoning for Universal Multimodal Retrieval
Date pendingDongyang Chen, Chaoyang Wang, Dezhao Su +6Multimodal Retrieval Multimodal Reasoning
Early Intervention for VFM-based Multimodal Medical Image Classification
Date pendingQijie Wei, Hailan Lin, Xirong LiMedical Image Classification Medical Imaging
Uni-HOI:A Unified framework for Learning the Joint distribution of Text and Human-Object Interaction
Date pendingMengfei Zhang, Jinlu Zhang, Zhigang TuArticulated Objects Unified Multimodal Models
A2DINOv3: Rethinking Multi-Modal Object Detection via Socialized Collaboration
Date pendingJiekang Feng, Zhihe Fan, Yunqi Zhu +6Cross-Modal Fusion Cross-Modal Learning
Modular Sensory Stream for Integrating Physical Feedback in Vision-Language-Action Models
Date pendingJimin Lee, Huiwon Jang, Myungkyu Koo +2Vision-Language-Action Models Sensory Feedback
Unified Text-Image Generation with Weakness-Targeted Post-Training
Date pendingJiahui Chen, Philippe Hansen-Estruch, Xiaochuang Han +7Text-To-Image Multimodal Generation
Lumina-OmniLV: A Unified Multimodal Framework for General Low-Level Vision
Date pendingYuandong Pu, Le Zhuo, Kaiwen Zhu +7Large Vision Language Models Multimodal Learning
When Minute-Resolution Monitoring Meets Session-Level Injury Labels: Landmark-Based Discrimination in Elite Women's Football
Date pendingEvangelos Chatzidimitriou, Konstantinos TserpesSports Science Physical Activities
Reason Through the Latent! Making Latent Visual Reasoning Necessary
Date pendingSuhyeong Park, Junha Jung, Jaewoo KangLatent Visual Reasoning Visual Reasoning
MedGEN-Bench: A Contextually Entangled Benchmark for Open-ended Multimodal Medical Generation
Date pendingJunjie Yang, Yuhao Yan, Gang Wu +12Medical Visual Question Answering Medical Vision-Language Models
Solve the Missing First Step: Can VLMs Standardize Raw Heterogeneous Medical Data?
Date pendingXin Chen, Dongliang Xu, Cunhao Zhu +5Diagnostic Benchmark Multimodal Clinical Data
MMGR: Multi-Modal Generative Reasoning Benchmark and Evaluation
Date pendingZefan Cai, Haoyi Qiu, Tianyi Ma +13Multimodal Reasoning Benchmarks Fluency
SparSTAR: Sparse Attention for SpaceTime AutoRegressive Video Synthesis
Date pendingJongbeom Lee, Hyunwoo Yu, Jincheol Yang +2Dynamic Sparse Attention Spatial Attention
Do Multimodal LLMs See Before They Read? Diagnosing Contextual Sycophancy
Date pendingYi-Cheng Lai, Hen-Hsen HuangMultimodal Learning Visual Evidence
Slot2Text: Object-Centric Visual Tokenization for Efficient and Spatially Traceable Surgical MLLMs
Date pendingGuiqiu Liao, Matjaz Jogan, Daniel A. HashimotoMulti-Modal Large Language Models Video Latents
CLIP-RD: Relational Distillation for Efficient CLIP Knowledge Distillation
Date pendingJeannie Chung, Hanna Jang, Ingyeong Yang +2Modality Gap
MAD: Modality-Adaptive Decoding for Mitigating Cross-Modal Hallucinations in Multimodal Large Language Models
Date pendingSangyun Chung, Se Yeon Kim, Youngchae Chee +1Multi-Modal Large Language Models Cross-Modal Learning
Mr.LHDR: A Benchmark for Multimodal Real-World Long-Horizon Deep Research Agents
Date pendingMinghao Guo, Meng Cao, Sui Zhao +10Deep Research Multimodal Reasoning Benchmarks
FastE: Readout-Triggered Token Compression for LLM Embedding Inference
Date pendingJinsong Shu, Jinyong Wen, Baokun Wang +4Large Language Model Compression LLM Inference Efficiency

Jul 21, 2025 · cs.CLJ/K move · Enter open · S save

Discrete Tokenization for Multimodal LLMs: A Comprehensive Survey

Jindong Li, Yali Fu, Jiahong Liu, Linxiao Cao+4

The rapid advancement of large language models (LLMs) has intensified the need for effective mechanisms to transform continuous multimodal data into discrete representations suitable for language-based processing. Discrete tokenization, with vector quantization (VQ) as a central approach, offers both computational efficiency and compatibility with LLM architectures. Despite its growing importance, there is a lack of a comprehensive survey that systematically examines VQ techniques in the context of LLM-based systems. This work fills this gap by presenting the first structured taxonomy and analysis of discrete tokenization methods designed for LLMs. We categorize 8 representative VQ variants that span classical and modern paradigms and analyze their algorithmic principles, training dynamics, and integration challenges with LLM pipelines. Beyond algorithm-level investigation, we discuss existing research in terms of classical applications without LLMs, LLM-based single-modality systems, and LLM-based multimodal systems, highlighting how quantization strategies influence alignment, reasoning, and generation performance. In addition, we identify key challenges including codebook collapse, unstable gradient estimation, and modality-specific encoding constraints. Finally, we discuss emerging research directions such as dynamic and task-adaptive quantization, unified tokenization frameworks, and biologically inspired codebook learning. This survey bridges the gap between traditional vector quantization and modern LLM applications, serving as a foundational reference for the development of efficient and generalizable multimodal systems. A continuously updated version is available at: https://github.com/jindongli-Ai/LLM-Discrete-Tokenization-Survey.

Calibration-Free Low-Bit Vector Quantization Large Language Model Quantization Multi-Modal Large Language Models

Open paper Ask about it PDF