Overview

Token pruning studies how to shorten sequences processed by transformers while retaining information needed for the task, from visual and audio inputs to long-context and diffusion language models. Methods rank tokens using attention, query relevance, entropy, representation changes across layers, or temporal and spatial structure; they then drop, merge, cache, or selectively recompute tokens, often with adaptive budgets. Work evaluates these approaches on image and video understanding, speech, generation, reasoning, and embodied control, using measures such as accuracy, fidelity, latency, throughput, and memory. The central tension is that aggressive pruning can discard evidence needed later, while scoring and selection overhead may reduce real speedups; reliably predicting which tokens remain important across tasks and layers is still an open challenge.

Latest papers

CardsList

57 papers6 papers in the last four weeks, down 14% on the four weeks before. 0.0% of all new papers.

  1. Temporal Aware Pruning for Efficient Diffusion-based Video Generation

    May 18, 2026Sheng Li, Yang Sui, Junhao Ran +3Video Diffusion ModelsToken Pruning

  2. SToRe3D: Sparse Token Relevance in ViTs for Efficient Multi-View 3D Object Detection

    May 13, 2026Sandro Papais, Lezhou Feng, Charles Cossette +1Multi-View 3D Object DetectionToken Pruning

  3. EchoPrune: Interpreting Redundancy as Temporal Echoes for Efficient VideoLLMs

    May 11, 2026Jiameng Li, Minye Wu, Jiezhang Cao +2Efficient VLM InferenceToken Pruning

  4. Uncertainty-Aware Token Importance Estimation in Spiking Transformers

    May 10, 2026Wenxuan Liu, Zecheng Hao, Tong Bu +2Spiking TransformersUncertainty Quantification

  5. HeadRouter: Dynamic Head-Weight Routing for Task-Adaptive Audio Token Pruning in Large Audio Language Models

    Apr 26, 2026Peize He, Yaodi Luo, Xiaoqian Liu +7Audio Token CompressionSelf-Attention

  6. ST-Prune: Training-Free Spatio-Temporal Token Pruning for Vision-Language Models in Autonomous Driving

    Apr 21, 2026Lin Sha, Haiyun Guo, Tao Wang +4VLMs for Autonomous DrivingEfficient VLM Inference

  7. CATP: Confidence-Aware Token Pruning for Camouflaged Object Detection

    Apr 18, 2026Yuhan Gao, Shuhao Kang, Xin He +3Token PruningEfficient ViTs

  8. Efficient Multi-View 3D Object Detection by Dynamic Token Selection and Fine-Tuning

    Apr 15, 2026Danish Nazir, Antoine Hanna-Asaad, Lucas Görnhardt +3Multi-View 3D Object Detection3D ViTs

  9. HieraVid: Hierarchical Token Pruning for Fast Video Large Language Models

    Apr 2, 2026Yansong Guo, Chaoyang Zhu, Jiayi Ji +2Token PruningVideo-Language Models

  10. Rényi Entropy: A New Token Pruning Metric for Vision Transformers

    Mar 29, 2026Wei-Yuan Su, Ruijie Zhang, Zheng ZhangToken PruningVisual Token Pruning

  11. ReDiPrune: Relevance-Diversity Pre-Projection Token Pruning for Efficient Multimodal LLMs

    Mar 25, 2026An Yu, Ting Yu Tsai, Zhenfei Zhang +3Efficient VLM InferenceToken Pruning

  12. ES-dLLM: Efficient Inference for Diffusion Large Language Models by Early-Skipping

    Mar 10, 2026Zijian Zhu, Fei Ren, Zhanhong Tan +1Diffusion Language Model InferenceDiffusion Language Models

  13. Speed3R: Sparse Feed-forward 3D Reconstruction Models

    Mar 9, 2026Weining Ren, Xiao Tan, Kai Han3D ReconstructionFeed-Forward 3D Reconstruction

  14. FastSTAR: Spatiotemporal Token Pruning for Efficient Autoregressive Video Synthesis

    Mar 7, 2026Sungwoong Yune, Suheon Jeong, Joo-Young KimVideo GenerationAutoregressive Video Generation

  15. History-Conditioned Spatio-Temporal Visual Token Pruning for Efficient Vision-Language Navigation

    Mar 6, 2026Qitong Wang, Yijun Liang, Ming Li +2Efficient VLM InferenceToken Pruning

  16. TP-Spikformer: Token Pruned Spiking Transformer

    Feb 28, 2026Wenjie Wei, Xiaolong Zhou, Malu Zhang +8Spiking TransformersToken Pruning

  17. EntropyPrune: Matrix Entropy Guided Visual Token Pruning for Multimodal Large Language Models

    Feb 19, 2026Yahong Wang, Juncheng Wu, Zhangkai Ni +6Token PruningVisual Token Pruning

  18. Shiva-DiT: Residual-Based Differentiable Top-kk Selection for Efficient Diffusion Transformers

    Feb 5, 2026Jiaji Zhang, Hailiang Zhao, Jiaju Wu +3Diffusion Model AccelerationDiffusion Transformer

  19. Window-Diffusion: Accelerating Diffusion Language Model Inference with Windowed Token Pruning and Caching

    Jan 28, 2026Fengrui Zuo, Zhiwei Ke, Yiming Liu +3Diffusion Language Model InferenceToken Pruning

  20. Streaming-dLLM: Accelerating Diffusion LLMs via Suffix Pruning and Dynamic Decoding

    Jan 25, 2026Zhongyu Xiao, Zhiwei Hao, Jianyuan Guo +4Adaptive InferenceBlockwise Diffusion Language Models

  21. Probe and Skip: Self-Predictive Token Skipping for Efficient Long-Context LLM Inference

    Jan 19, 2026Zimeng Wu, Donghao Wang, Chaozhe Jin +2LLM Inference AccelerationLong-Context LLM Inference