Augmented Transformer

Augmented Transformers represent a significant advancement in deep learning, aiming to improve the performance and efficiency of standard Transformer architectures by incorporating additional mechanisms to address limitations in handling long sequences, multimodal data, and cross-domain generalization. Current research focuses on integrating state-space models, which excel at capturing long-range dependencies, and employing techniques like adaptive n-gram embeddings and contrastive learning to enhance feature extraction and multimodal fusion. These improvements are driving progress in diverse applications, including speech recognition, scene text recognition, 3D hand trajectory forecasting, and multi-modal object detection, leading to more robust and efficient models for complex tasks.

Papers

August 27, 2024

DualKanbaFormer: Kolmogorov-Arnold Networks and State Space Model Transformer for Multimodal Aspect-based Sentiment Analysis
Adamu Lawan, Juhua Pu, Haruna Yunusa, Muhammad Lawan, Aliyu Umar, Adamu Sani Yahya
Sentiment Analysis Cross Modal Multimodal Phenomenon State Space Model Kolmogorov Arnold Network Aspect Based Sentiment Analysis Textual Representation Augmented Transformer

July 17, 2023

Uncertainty-aware State Space Transformer for Egocentric 3D Hand Trajectory Forecasting
Wentao Bao, Lele Chen, Libing Zeng, Zhong Li, Yi Xu, Junsong Yuan, Yu Kong
Trajectory Forecasting Finger Motion Egocentric Action Anticipation Augmented Transformer Egocentric 3D Hand Pose Estimation

February 28, 2023

Augmented Transformers with Adaptive n-grams Embedding for Multilingual Scene Text Recognition
Xueming Yan, Zhihang Fang, Yaochu Jin
Scene Text N Gram Multilingual Scenario Augmented Transformer

February 27, 2023

Diagonal State Space Augmented Transformers for Speech Recognition
George Saon, Ankit Gupta, Xiaodong Cui
Speech Recognition Temporal Convolution Neural Transducer Linear RNN Transition Matrix Augmented Transformer Diagonal State Space

December 29, 2022

Efficient Movie Scene Detection using State-Space Transformers
Md Mohaiminul Islam, Mahmudul Hasan, Kishan Shamsundar Athrey, Tony Braskich, Gedas Bertasius
Video Recognition Scene Detection Augmented Transformer

December 15, 2022

Efficient Long Sequence Modeling via State Space Augmented Transformer
Simiao Zuo, Xiaodong Liu, Jian Jiao, Denis Charles, Eren Manavoglu, Tuo Zhao, Jianfeng Gao
Language Understanding Language Generation Encoder Decoder Long Sequence Language Modeling Task Augmented Transformer

October 18, 2022

Cross-Domain Aspect Extraction using Transformers Augmented with Knowledge Graphs
Phillip Howard, Arden Ma, Vasudev Lal, Ana Paula Simoes, Daniel Korat, Oren Pereg, Moshe Wasserblat, Gadi Singer
Knowledge Graph Aspect Term Extraction Query Expansion Fine Grained Sentiment Specific Knowledge Graph Augmented Transformer

April 1, 2022

CAT-Det: Contrastively Augmented Transformer for Multi-modal 3D Object Detection
Yanan Zhang, Jiaxin Chen, Di Huang
3D Object Detection Cross Modal Transformer Multimodal Data Augmentation Augmented Transformer

Augmented Transformer

Papers

DualKanbaFormer: Kolmogorov-Arnold Networks and State Space Model Transformer for Multimodal Aspect-based Sentiment Analysis

Uncertainty-aware State Space Transformer for Egocentric 3D Hand Trajectory Forecasting

Augmented Transformers with Adaptive n-grams Embedding for Multilingual Scene Text Recognition

Diagonal State Space Augmented Transformers for Speech Recognition

Efficient Movie Scene Detection using State-Space Transformers

Efficient Long Sequence Modeling via State Space Augmented Transformer

Cross-Domain Aspect Extraction using Transformers Augmented with Knowledge Graphs

CAT-Det: Contrastively Augmented Transformer for Multi-modal 3D Object Detection