Multimodal Token Compression

Multimodal token compression reduces the number or size of image, video, audio, and other input representations so models can process longer inputs with less computation and memory. Common approaches prune or merge redundant tokens, pool them into learned bottlenecks, allocate budgets using queries or modality-specific signals, and compress key-value caches. These methods are evaluated on tasks such as visual question answering, video understanding, audio comprehension, and multimodal retrieval, often by comparing accuracy with latency, FLOPs, or memory use at different compression levels. The central challenge is deciding what information can safely be removed: useful evidence varies by task, modality, and model layer, and aggressive compression can discard fine details or weaken joint understanding and generation.