Token Merging

Token merging studies how to combine similar or redundant tokens in transformer sequences so models use less computation and memory. Methods include training-free similarity matching, salience- and structure-aware aggregation, temporal or spatial grouping, and learned merging modules; some systems combine merging with pruning or early exit. Applications span image and video models, diffusion, language, speech and audio, and 3D data, with evaluations measuring both task quality and costs such as FLOPs, latency, and storage. The central challenge is deciding which information can be merged: aggressive reduction saves more resources, but may erase boundaries, objects, or temporal details, and the best strategy can depend on the task.