Cross-Modal Attention
Overview
Cross-modal attention studies how information from one modality—such as text, images, audio, video, or sensor data—can guide the representation of another. It matters for tasks including visual question answering, retrieval, medical diagnosis, speech separation, and autonomous systems, where useful evidence may be distributed across inputs. Common approaches include bidirectional or asymmetric cross-attention, query- and register-based fusion, and sparse, local, or temporally guided attention, sometimes combined with confidence gating or feature disentanglement. Performance has improved across many benchmarks, but reliable fusion remains difficult when modalities are misaligned, unevenly informative, or prone to dominating one another; attention patterns also do not always guarantee faithful grounding or reasoning.
Latest papers
93 papers6 papers in the last four weeks, down 33% on the four weeks before. 0.0% of all new papers.
Learning to Unify Deformable Shape and Texture Representations for Cardiac Video Classification
AT-Attn: Temporal-Aware Cross-Attention for Longitudinal Multimodal Alzheimer's Disease Diagnosis
ELSA3D: Elastic Semantic Anchoring for Unified 3D Understanding and Generation
Cross-Modal Fusion of OCT and OCT angiography enface for Improved Diagnostics of Diabetic Retinopathy
Q-TriM: Question-Guided Tri-Modal Attention for Audio-Visual Question Answering
ADAPT: Attention Dynamics Alignment with Preference Tuning for Faithful MLLMs
Modeling Local, Global, and Cross-Modal Context in Multimodal 3D MRI
Multi-modality Image Fusion under Adverse Weather: Mask-Guided Feature Restoration and Interaction
Progressive Pixel-Neighborhood Deformable Cross-Attention for Multispectral Object Detection
Attention-Spectrum Regularization for Replay-Free Continual Multimodal LLMs
Text Dictates, Music Decorates: Energy-based Attention for Editable Dance Motion Generation
DREAM: Extending Vision-Language Models with Dual-Objective Encoding for Cross-Modal Retrieval
Multi-View Speech Representation Learning for Parkinson's Disease Detection Using Context-guided Cross-modal Attention
DyCo-RL: Dynamic Cross-Modal Coordination for Visual Reasoning
UniCA: Bi-directional Cross-Attention with Positive Similarity Loss for Robust Multi-Modal Retrieval
Query-based Cross-Modal Projector Bolstering Mamba Multimodal LLM
MOSS-Video-Preview: Toward Real-Time Video Understanding via Cross-Attention
CL-DMDF:Dynamic Multimodal Data Fusion Model Based on Contrastive Learning
MultiAct: Text-to-Motion Generation from Composite Text via Tailored Attention Guidance
Generic Interpretation Approach for Transformer Models Incorporating Heterogenous Attention Structures
Vision-Language Binding in In-Context Image Generation
Correcting Visual Blur Induced by Attention Distraction to Reduce Hallucinations: Algorithm and Theory
MLLMs Know When Before Speaking: Revealing and Recovering Temporal Grounding via Attention Cues
RAVE: Re-Allocating Visual Attention in Large Multimodal Models
CAM-VFD: Cross-Attention Multimodal Video Forgery Detection
MHSA: A Lightweight Framework for Mitigating Hallucinations via Steered Attention in LVLMs
FS-I2P:A Hierarchical Focus-Sweep Registration Network with Dynamically Allocated Depth
PRIMED: Adaptive Modality Suppression for Referring Audio-Visual Segmentation via Biased Competition
PC-MNet: Dual-Level Congruity Modeling for Multimodal Sarcasm Detection via Polarity-Modulated Attention
MUStARD benchmark and its spurious-correlation-mitigated balanced datasets demonstrate that our approach achieves new state-of-the-art performance, surpassing the strongest multimodal baseline by a substantial 3.14% improvement in Macro-F1. By architecturally isolating atomic, composition, and contextual conflicts. This work provides a robust, decoupled paradigm for modeling subtle pragmatic incongruities in human communication.