Deformable Attention

Follow this topic Ask about this topic

Momentum

0 papers in the last four weeks, with none the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 10

All topics

MegaSlide-DiT: Memory-Centric Adaptation and Deformable Local Attention for Efficient Video Diffusion
Jul 17, 2026Jiacheng Liu, Jason LiuVideo Diffusion Models Diffusion Transformer
Trendinsight Lab / UC San Diego
GADA: Geometry-Aware Deformable Aggregation for Image-Based Gaussian Splatting
Jul 1, 2026Siwoo Lim, Sunjae Yoon, Gwanhyeong Koo +1Multi-View Consistency 3D Gaussian Splatting
Korea Advanced Institute of Science and Technology (KAIST), Republic of Korea · Chung-Ang University, Republic of Korea
Progressive Pixel-Neighborhood Deformable Cross-Attention for Multispectral Object Detection
Jun 23, 2026Tian Qiu, Jifeng Shen, Xin ZuoMultispectral Object Detection Cross-Modal Alignment
School of Electrical and Information Engineering, Jiangsu University, Zhenjiang, 212013, China · School of Computer Science and Engineering, Jiangsu University of Science and Technology, Zhenjiang, 212003, China
Multi-Modal Hyper-Graph Fusion for Low-Light Crowd Counting
Jun 17, 2026Hao-Yuan Ma, Li Zhang, Yushi Qiu +3Crowd Counting Deformable Attention
School of Computer Science and Technology, Soochow University, Suzhou, China
Parameter-Efficient Fine-Tuning of Large Pretrained Models for Instance Segmentation Tasks
Jun 1, 2026Nermeen Abou Baker, David Rohrschneider, Uwe HandmannFine-Tuning Instance Segmentation
Ruhr West University of applied sciences · Lützowstraße 5, 46236 Bottrop, Germany: Ruhr West University of Applied Sciences
VigilFormer: Deformable Attention for Video Anomaly Detection with Causal Risk Inference
May 31, 2026Xinze ZhangTransformer Attention Deformable Attention
University of Southern California, Los Angeles, CA 90007, USA
VGGT-Occ: Geometry-Grounded and Density-Aware Gated Fusion for 3D Occupancy Prediction
May 16, 2026Xun Chen, Tianchen Deng, Rui Wang +53D Semantic Segmentation 3D Occupancy Prediction
Nanyang Technological University · Shanghai Jiao Tong University · ETH Zurich
Towards Open World Sound Event Detection
May 5, 2026P. H. Hai, L. T. Minh, L. H. SonContinual Learning Detection Transformer
VNU University of Engineering and Technology, Hanoi, 100000, Vietnam · Artificial Intelligence Research Center, VNU Information Technology Institute, Hanoi, 100000, Vietnam
CAM3DNet: Comprehensively mining the multi-scale features for 3D Object Detection with Multi-View Cameras
Apr 18, 2026Mingxi Pang, Dingheng Wang, Zekun Li +4Multi-Scale Feature Fusion Multi-View 3D Object Detection
College of Intelligent Science and Technology, National University of Defense Technology, Changsha, 410073, China · Northwest Institute of Mechanical & Electrical Engineering, Xianyang, 712099, China. · School of Aerospace Engineering, Xi’an Jiaotong University, Xi’an, 710049, China
DiDA: Video Object Segmentation with Distillation Learning of Deformable Attention
Jan 25, 2024Quang-Trung Truong, Duc Thanh Nguyen, Binh-Son Hua +1Video Object Segmentation Transformer Attention
Hong Kong University of Science and Technology · Deakin University · Trinity College Dublin

Jul 17, 2026, cs.CVJ/K move · Enter open · S save

MegaSlide-DiT: Memory-Centric Adaptation and Deformable Local Attention for Efficient Video Diffusion

Jiacheng Liu, Jason Liu

Trendinsight Lab / UC San Diego

Video Diffusion Models Diffusion Transformer Diffusion Model Fine-Tuning Deformable Attention Memory-Efficient Training

Open paper AskPDF arXiv

High-resolution video diffusion models built on Diffusion Transformers (DiTs) deliver strong fidelity but quickly exhaust the memory budget of a single workstation. A 100 billion-plus parameter DiT easily requires over a terabyte of persistent state, while naive spatiotemporal self-attention grows quadratically in sequence length. These two walls -- parameter memory and activation memory -- prevent researchers from adapting massive generative models without large GPU clusters. We revisit this problem from a systems perspective and introduce MegaSlide-DiT, a prototype that demonstrates how a pre-trained 105B DiT can be adapted on a single H200 GPU with 1.5 TB of host RAM. Our key insight is that the GPU need not own the model state: all persistent weights, master weights and optimizer moments remain in host memory, while only transient shards are streamed to the GPU on demand. Simultaneously, we replace quadratic global attention with 3D Deformable Slide Attention (3D-DSA), a motion-adaptive local attention operator that reduces both memory and computational complexity to linear in the sequence length. We report detailed memory accounting, execution traces and evaluation results to substantiate our design. MegaSlide-DiT does not claim to train a 105B model from scratch on a single GPU, nor does it magically solve bandwidth limits; rather, it offers a pragmatic path for full-parameter adaptation of massive video diffusion models on high-end workstations.

Momentum

Related topics

Latest papers 10