cs.CVNov 2, 2026

R2^2-Omni: Reducing Structural and Precision Redundancy in Omni-LLMs for Long-Form Generation

Authors: Bolong Feng, Pengfei Zhou, Zhenglin Wan, Wei Wang, Wangbo Zhao

Organizations: InfRec, Cardinal AI Lab · Shanghai Jiao Tong University · National University of Singapore · The Hong Kong University of Science and Technology

Abstract

Omni-modal large language models (Omni-LLMs) typically process large numbers of multimodal tokens from audio-visual inputs, incurring substantial computational costs and a large memory footprint for the key-value (KV) cache. Aggressive token pruning can alleviate this problem while maintaining competitive performance on multiple-choice tasks, which require only a single option as output. However, we observe substantial quality degradation when generating long-form, open-ended responses. We attribute this gap to query-guided selection, which can retain sufficient evidence for answer selection while discarding details needed for comprehensive generation. This motivates reducing overlapping information across tokens, termed structural redundancy, while preserving diverse content. However, the retained representations may still contain precision redundancy due to unnecessarily high numerical precision in their cached values. To address these two levels of redundancy in token representations, we introduce a training-free framework named R2^2-Omni. First, Structure-Aware Token Consolidation (STC) consolidates redundant representations using spatiotemporal similarity in video and local continuity in audio. Then, Semantic-Sensitivity Guided Precision Allocation (SSPA) allocates value precision to the retained representations according to the estimated reduction in attention-output error under quantized keys, with query attention and audio-visual alignment providing a semantic prior. Extensive experiments on long-form generation benchmarks demonstrate an average 5.0×\times reduction in the memory footprint while preserving 97.98% of baseline generation quality and achieving a 1.48×\times end-to-end speedup, highlighting the benefits of jointly reducing structural and precision redundancy.

Figures & tables

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. OmniPack: Unified Token Compression for Efficient Omni-modal Large Language Models

    Aug 4, 2026Wanshun Su, Yang Shi, Feihu Liu +10Efficient Multimodal InferenceAudio Token Compression

  2. OmniScope: Modality-Decoupled Token Compression for Omnimodal Large Language Models

    Jul 25, 2026Jinsen Su, Yongdong Luo, Yuexiao Ma +3Efficient Multimodal InferenceAudio Token Compression