Transformer Based
Transformer-based models are revolutionizing various fields by leveraging self-attention mechanisms to capture long-range dependencies in sequential data, achieving state-of-the-art results in tasks ranging from natural language processing and image recognition to time series forecasting and robotic control. Current research focuses on improving efficiency (e.g., through quantization and optimized architectures), enhancing generalization capabilities, and addressing challenges like handling long sequences and endogeneity. These advancements are significantly impacting diverse scientific communities and practical applications, leading to more accurate, efficient, and robust models across numerous domains.
Papers
Efficient Transformer Encoders for Mask2Former-style models
Manyi Yao, Abhishek Aich, Yumin Suh, Amit Roy-Chowdhury, Christian Shelton, Manmohan Chandraker
Integrating Mamba and Transformer for Long-Short Range Time Series Forecasting
Xiongxiao Xu, Yueqing Liang, Baixiang Huang, Zhiling Lan, Kai Shu
SC-HVPPNet: Spatial and Channel Hybrid-Attention Video Post-Processing Network with CNN and Transformer
Tong Zhang, Wenxue Cui, Shaohui Liu, Feng Jiang
Progressive Token Length Scaling in Transformer Encoders for Efficient Universal Segmentation
Abhishek Aich, Yumin Suh, Samuel Schulter, Manmohan Chandraker
PointDifformer: Robust Point Cloud Registration With Neural Diffusion and Transformer
Rui She, Qiyu Kang, Sijie Wang, Wee Peng Tay, Kai Zhao, Yang Song, Tianyu Geng, Yi Xu, Diego Navarro Navarro, Andreas Hartmannsgruber
Texture, Shape and Order Matter: A New Transformer Design for Sequential DeepFake Detection
Yunfei Li, Yuezun Li, Xin Wang, Baoyuan Wu, Jiaran Zhou, Junyu Dong