Attention Based Network

Attention-based networks are a class of neural networks designed to selectively focus on the most relevant information within input data, improving efficiency and accuracy in various tasks. Current research emphasizes the development of novel attention mechanisms, often integrated into transformer, convolutional, or hybrid architectures, to enhance performance in areas like object detection, visual navigation, and pose estimation. These networks are proving highly effective across diverse applications, from medical image analysis and video enhancement to natural language processing and robotics, driving advancements in fields requiring sophisticated information processing.

Papers

October 31, 2024

Approximate attention with MLP: a pruning strategy for attention-based model in multivariate time series forecasting
Suhan Guo, Jiahong Deng, Yi Wei, Hui Dou, Furao Shen, Jian Zhao
Full Model Multivariate Time Series Temporal Network Single Scene Specific MLP Self Attention Network Pruning Strategy Attention Based Network Approximate Attention

August 24, 2024

Discovering Long-Term Effects on Parameter Efficient Fine-tuning
Gaole Dai, Yiming Tang, Chunkai Fan, Qizhe Zhang, Zhi Zhang, Yulu Gan, Chengqing Zeng, Shanghang Zhang, Tiejun Huang
Mixed Effect Parameter Efficient Fine Tuning Biological Neural Network Attention Based Network Pre Trained Neural Network

April 12, 2024

TDANet: Target-Directed Attention Network For Object-Goal Visual Navigation With Zero-Shot Ability
Shiwei Lian, Feitian Zhang
Deep Reinforcement Learning Zero Shot Object Goal Navigation Attention Based Network

March 25, 2024

Multi-attention Associate Prediction Network for Visual Tracking
Xinglong Sun, Haijiang Sun, Shan Jiang, Jiacheng Wang, Xilai Wei, Zhonghe Hu
Visual Tracking Siamese Tracker Deep Tracker Attention Based Network Spatial Alignment

March 15, 2024

A Hybrid SNN-ANN Network for Event-based Object Detection with Spatial and Temporal Attention
Soikat Hasan Ahmed, Jan Finkbeiner, Emre Neftci
Neuromorphic Hardware Temporal Attention Spatial Domain Attention Based Deep Hybrid Neural Network Attention Based Network SNN Based Event Based Object Detection

February 28, 2024

Attention-Propagation Network for Egocentric Heatmap to 3D Pose Lifting
Taeho Kang, Youngki Lee
Camera Pose Pose Estimate Egocentric Perception Attention Based Network Pose Lifting Egocentric 3D

December 3, 2023

The mechanistic basis of data dependence and abrupt learning in an in-context classification task
Gautam Reddy
Context Learning Transformer Model Different Context Dependent Data Attention Based Network Mechanistic Study

August 29, 2023

Analysis of Learned Features and Framework for Potato Disease Detection
Shikha Gupta, Soma Chakraborty, Renu Rameshan
Convolutional Neural Network Training Data General Analysis New Framework Plant Disease Attention Based Network Learned Feature Potato Disease

July 14, 2023

AIC-AB NET: A Neural Network for Image Captioning with Spatial Attention and Text Attributes
Guoyun Tu, Ying Liu, Vladimir Vlassov
Neural Network Image Captioning Textual Feature Caption Generation Spatial Attention Attention Framework Attention Based Network

March 21, 2023

Automatic evaluation of herding behavior in towed fishing gear using end-to-end training of CNN and attention-based networks
Orri Steinn Guðfinnsson, Týr Vilhjálmsson, Martin Eineborg, Torfi Thorhallsson
Action Recognition CNN Network Automatic Evaluation Herding Algorithm Attention Based Network End to End Training Two Stream Multi Class Classification Task ANGLEr Behavior

March 20, 2023

Relate auditory speech to EEG by shallow-deep attention-based network
Fan Cui, Liyong Guo, Lang He, Jiyao Liu, ErCheng Pei, Yujun Wang, Dongmei Jiang
EEG Signal Attention Based Deep EEG Signal Analysis Attention Based Network Relation Information Attention Based Correlation Module Deep Similarity

March 7, 2023

How Do Transformers Learn Topic Structure: Towards a Mechanistic Understanding
Yuchen Li, Yuanzhi Li, Andrej Risteski
Transformer Megatron Decepticons Self Attention Layer Latent Dirichlet Allocation Semantic Structure Attention Based Network Mechanistic Understanding Pairwise Attention Topic Structure

March 3, 2023

Pyramid Pixel Context Adaption Network for Medical Image Classification with Supervised Contrastive Learning
Xiaoqing Zhang, Zunjie Xiao, Xiao Wu, Yanlin Chen, Jilu Zhao, Yan Hu, Jiang Liu
Medical Image Classification Supervised Contrastive Learning Attention Based Network Pixel Attention Pyramid Context Network

November 27, 2022

Leveraging Image Matching Toward End-to-End Relative Camera Pose Regression
Fadi Khatib, Yuval Margalit, Meirav Galun, Ronen Basri
Pose Regression Attention Based Network Relative Pose Regression

October 20, 2022

Learning Attention Propagation for Compositional Zero-Shot Learning
Muhammad Gul Zain Ali Khan, Muhammad Ferjad Naeem, Luc Van Gool, Alain Pagani, Didier Stricker, Muhammad Zeshan Afzal
Zero Shot Learning Compositional Zero Shot Learning Attention Based Network Unseen Composition

September 11, 2022

Vec2Face-v2: Unveil Human Faces from their Blackbox Features via Attention-based Network in Face Recognition
Thanh-Dat Truong, Chi Nhan Duong, Ngan Le, Marios Savvides, Khoa Luu
Generative Adversarial Network Face Recognition Black Box Facial Prior Face Reconstruction Attention Based Network Unmasked Face Black Box Face Recognition

June 16, 2022

PeQuENet: Perceptual Quality Enhancement of Compressed Video with Adaptation- and Attention-based Network
Saiping Zhang, Luis Herranz, Marta Mrak, Marc Gorriz Blanch, Shuai Wan, Fuzheng Yang
Generative Adversarial Network Adaptation Concern Perceptual Quality Compressed Video Attention Based Network Video Quality Enhancement

June 2, 2022

XBound-Former: Toward Cross-scale Boundary Modeling in Transformers
Jiacheng Wang, Fei Chen, Yuxi Ma, Liansheng Wang, Zhaodong Fei, Jianwei Shuai, Xiangdong Tang, Qichao Zhou, Jing Qin
Transformer Megatron Decepticons Polyp Segmentation Skin Lesion Skin Lesion Segmentation Multiscale Modeling Attention Based Network Cross Scale

June 1, 2022

Dual-stream spatiotemporal networks with feature sharing for monitoring animals in the home cage
Ezechukwu I. Nwokedi, Rasneer S. Bains, Luc Bidaut, Xujiong Ye, Sara Wells, James M. Brown
Dual Stream Animal Specie Attention Based Network Behavior Classification Mouse Data Spatiotemporal Network Spatiotemporal Deep Learning Net CAGE

April 26, 2022

Function-words Enhanced Attention Networks for Few-Shot Inverse Relation Classification
Chunliu Dou, Shaojuan Wu, Xiaowang Zhang, Zhiyong Feng, Kewen Wang
Shot Learning Relation Classification Attention Framework Attention Based Network

Attention Based Network

Papers

Approximate attention with MLP: a pruning strategy for attention-based model in multivariate time series forecasting

Discovering Long-Term Effects on Parameter Efficient Fine-tuning

TDANet: Target-Directed Attention Network For Object-Goal Visual Navigation With Zero-Shot Ability

Multi-attention Associate Prediction Network for Visual Tracking

A Hybrid SNN-ANN Network for Event-based Object Detection with Spatial and Temporal Attention

Attention-Propagation Network for Egocentric Heatmap to 3D Pose Lifting

The mechanistic basis of data dependence and abrupt learning in an in-context classification task

Analysis of Learned Features and Framework for Potato Disease Detection

AIC-AB NET: A Neural Network for Image Captioning with Spatial Attention and Text Attributes

Automatic evaluation of herding behavior in towed fishing gear using end-to-end training of CNN and attention-based networks

Relate auditory speech to EEG by shallow-deep attention-based network

How Do Transformers Learn Topic Structure: Towards a Mechanistic Understanding

Pyramid Pixel Context Adaption Network for Medical Image Classification with Supervised Contrastive Learning

Leveraging Image Matching Toward End-to-End Relative Camera Pose Regression

Learning Attention Propagation for Compositional Zero-Shot Learning

Vec2Face-v2: Unveil Human Faces from their Blackbox Features via Attention-based Network in Face Recognition

PeQuENet: Perceptual Quality Enhancement of Compressed Video with Adaptation- and Attention-based Network

XBound-Former: Toward Cross-scale Boundary Modeling in Transformers

Dual-stream spatiotemporal networks with feature sharing for monitoring animals in the home cage

Function-words Enhanced Attention Networks for Few-Shot Inverse Relation Classification