Cross-Attention

Ask about this topic

Overview

Cross-attention studies how queries from one representation can select and combine relevant information from another, supporting tasks such as image-text understanding, sensor fusion, and sequence forecasting. Common approaches include bidirectional or gated fusion, multi-scale attention, and sparse or geometry-constrained routing, with applications ranging from vision-language models and diffusion generation to medical imaging and robotics. Evaluation spans benchmarks for image recognition, segmentation, multimodal question answering, forecasting, and cross-view matching, with growing emphasis on efficiency and robustness to missing or unreliable inputs. A central tension is that selective attention can improve fusion and control, but attention maps are not always faithful explanations, and simpler concatenation can work better when features are already aligned.

Latest papers

CardsList

78 papers9 papers in the last four weeks, up 13% on the four weeks before. 0.1% of all new papers.

  1. When Slides Met Scans: Cross-Attention for Renal Cancer Prognosis

    Oct 26, 2026Anusha Krishnamoorthy, Basudha PalCancer Survival PredictionMultimodal Survival Analysis

    Wallace H. Coulter Department of Biomedical Engineering, Georgia Institute of Technology & Emory University · Department of Electrical & Computer Engineering, Johns Hopkins University

  2. Acute Stroke Classification Using Asymmetric Data Standardization and Orthogonal-Focal Loss Optimized CNN-ViT Cross-Attention Fusion: A Multicenter External Validation Study

    Oct 25, 2026Tansel UyarMedical Image ClassificationMedical Imaging

    Department of Biomedical Engineering, Faculty of Engineering, Başkent University, Ankara, 06790, Türkiye

  3. DS-BiCA-Net: Pair-Preserving Cross-Temporal Modeling for Remote Sensing Change Captioning

    Oct 23, 2026Licheng Zhang, Naveed Akhtar, Siew-Kei LamRemote Sensing Image Change CaptioningCross-Attention

    the School of Computing and Information Systems, the University of Melbourne, Victoria 3010, Australia · Nanyang Technological University, Singapore

  4. BatchParScale: Sparse Batch-Mixing for Sequence Models

    Oct 18, 2026Hans Peter Lyngsøe Raaschou-jensen, M. Onurcan Kaya, Desmond ElliottParallel Reasoning in Language ModelsLanguage Model Scaling

    University of Copenhagen · Technical University of Denmark

  5. Link Both Views: Cross-Image Routing Attention for Dual-Endpoint V2I Beam-Pair Power Prediction

    Oct 13, 2026Shiliang Lu, Lu Bai, Ziwei Huang +1Cross-View AttentionWireless Communications

    Joint SDU-NTU Centre for Artificial Intelligence Research (C-FAIR), Shandong University, Jinan 250101, China · School of Software, Shandong University · State Key Laboratory of Photonics and Communications, School of Electronics, Peking University, Beijing 100871, China

  6. MP-MNCA: Morphology-Prior Masked Neighbor Cross-Attention for Efficient Spatial Domain Identification

    Oct 12, 2026Sidharthan Sivacoumar Chitra, Sanjeev ManivannanSpatial TranscriptomicsCross-Attention

    IIT Madras

  7. When Attention Does Not Explain the Peak: Temporal Reference vs. Forecast Output in Attention-Based Time-Series Forecasting

    Oct 5, 2026Yuji Akamatsu, Takao YamanakaExplainable Time Series ForecastingCross-Attention

    Sophia University, Tokyo, Japan

  8. Leveraging LLM-Generated Explanations for Detecting Emotionally Rewritten Fake News

    Sep 29, 2026Yupei Guo, Jiajun He, Xiaohan Shi +4Fake News DetectionText Classification

    The University of Osaka Osaka, Japan · Alibaba Inc Hangzhou, China · Nagoya University Nagoya, Japan +1

  9. OCA: ODE-Driven Cross-Attention for Image-to-Point-Cloud Registration

    Sep 29, 2026Pei An, Jiaqi Yang, Yulong Wang +2Cross-AttentionPoint Cloud Registration

    Huazhong University of Science and Technology, China · Northwestern Polytechnical University, China · Huazhong Agricultural University, China +2

  10. Cross-attention encoding models reveal dynamic spatiotemporal routing across human higher visual cortex

    Sep 28, 2026Iishaan Inabathini, Margaret M. HendersonCross-AttentionVisual Neuroscience

    Carnegie Mellon University

  11. Integrating Local Detail and Global Context: A Dual-Input Multi-Task Learning Framework for Bone Tumor Diagnosis

    Sep 23, 2026S. M. Nasif Uddin, Rusab Sarmun, Muhammad E. H. Chowdhury +3Cross-AttentionMulti-Task Learning

    Department of Electrical and Electronic Engineering, Ahsanullah University of Science and Technology, Bangladesh · Department of Electrical and Electronic Engineering, University of Dhaka, Bangladesh · Department of Electrical Engineering, Qatar University, Doha 2713, Doha, Qatar +1

  12. QueryFormer: Winning Solution for KDD Cup 2026 Tencent UniRec Challenge

    Sep 15, 2026Yuanzhe Zhou, Zhaoyang ZengConversion Rate PredictionUser Modeling

    Wuhan University · Sun Yat-sen University

  13. MCRL2: Multi-resource Cross-attention-based Representation Learning-augmented Reinforcement Learning for Cloud Microservice Scheduling

    Sep 11, 2026Tiangang Li, Shi Ying, Xiangbo Tian +2Cross-AttentionCloud Computing

    School of Computer Science, Wuhan University, Wuhan 430072, China · School of Computer Science, Beijing University of Posts and Telecommunications, Beijing 100876, China

  14. A Dual Cross-Attention Framework for Colposcopic CIN Grading and Swede Score Prediction Using a New Multi-Center Dataset

    Sep 11, 2026Dania Khan, Nuzhat Aisha Shaikh, Asfina Hassan Juicy +3Medical ImagingCross-Attention

    mHealth lab, Department of Biomedical Engineering, Bangladesh University of Engineering and Technology (BUET), Dhaka, Bangladesh · Dhaka Medical College, Dhaka, Bangladesh · Center for Bioengineering Innovation and Design (CBID), Johns Hopkins University, Baltimore, Maryland, USA

  15. Time-Frequency Geometric Cross-Attention for Chunked Vision-Language-Action Models

    Sep 9, 2026Shengye Dong, Haochen Niu, Hao Liu +3Cross-AttentionVision-Language-Action Models

  16. Distance-Aware Attention and Wall-Distance Expert Routing for Transformer-Based 3D Flow Prediction

    Sep 7, 2026Sanghyeon Kim, Sunwoong Yang, Namwoo KangCross-AttentionMixture of Experts

    Division of Future Vehicle Korea Advanced Institute of Science and Technology Daejeon, 34051, Republic of Korea · Department of Mechanical Engineering Hanyang University Ansan, 15588, Republic of Korea · Cho Chun Shik Graduate School of Mobility Korea Advanced Institute of Science and Technology Narnia Labs Daejeon, 34051, Republic of Korea

  17. CrossDepth: Geometry-Constrained Attention for Generalizable Multi-View Surround Depth Estimation

    Sep 4, 2026Samer Abualhanud, Max MehltretterCross-View AttentionMulti-View Depth Estimation

    Leibniz University Hannover Hannover, Germany

  18. Disentangling Attention in Deep Operator Learning: A Controlled Study of Data-Driven and Physics-Informed Architectures

    Sep 3, 2026Amar Alem Koric, Qibang Liu, Seid KoricPDE Operator LearningDeep Operator Networks

    Department of Mechanical Engineering, Stanford University, Stanford, CA, 94305, USA · National Center for Supercomputing Applications, University of Illinois at Urbana-Champaign, 1205 W. Clark St., Urbana, IL, 61801, USA · The Grainger College of Engineering, Mechanical Science and Engineering, University of Illinois at Urbana-Champaign, 1206 W. Green St., Urbana, IL, 61801, USA

  19. Generalized Context in Cross Attention for Transfer Learning of Disjoint Tabular Data

    Aug 28, 2026Kazi F. Akhter, Ibna Kowsar, Manar D. SamadCross-Domain Transfer LearningTransfer Learning

    Department of Computer Science Tennessee State University Nashville, TN, USA · Department of Computer Science North Carolina Agricultural and Technical State University Greensboro, NC, USA

  20. Lévy Attention: Single-Pass Predictive Uncertainty for Continuous-Time Attention

    Aug 19, 2026Sotirios P. Chatzis, Loukas PapadoulasTime Series ModelingUncertainty Quantification

    GenML Laboratory, Department of Electrical Engineering and Computer Science and Engineering, Cyprus University of Technology, Limassol, Cyprus · Ethical AI Novelties, Cyprus

  21. Designing a Good Virtual Node: Addressable and Cardinality-Preserving Global Memory for Message Passing Architectures

    Aug 3, 2026Félix MarcocciaCross-AttentionGNN Expressivity

  22. Context-Informed Ship Trajectory Prediction via Conditional Attention

    Jul 29, 2026Yuan Guan, Chandler Squires, Timothy Hu +1Cross-AttentionVessel Trajectory Prediction

    Machine Learning Department, Carnegie Mellon University · Advanced Technology Laboratories Lockheed Martin

  23. Salient Knowledge Pathways: Sparse Cross-Modal Routing for Efficient Knowledge-Intensive Multimodal Question Answering

    Jul 28, 2026Noor Islam S. Mohammad, Uluğ BayazıtMultimodal RAGEfficient Multimodal Inference

    Department of Computer Science, İTÜ, İstanbul, Türkiye · Faculty of Computer Engineering, İTÜ, İstanbul, Türkiye.

  24. UMI3D: Robust 3D Generation on Unconstrained Multi-Image Inputs via Simultaneous Focus Cross-Attention Routing

    Jul 27, 2026Zefan Qu, Zhenwei Wang, Gerhard Petrus Hancke +13D Foundation ModelsCross-Attention

    Department of Computer Science City University of Hong Kong

  25. CrossSpine: Multi-scale Cross-sequence Attention with Anatomical Priors for Automated Pfirrmann Grading

    Jul 22, 2026Hai Son Nguyen, Duong Ngoc Vu, Trong-Nghia Nguyen +5Cross-AttentionMedical Image Analysis

    Business AI Lab, College of Technology, National Economics University, Vietnam · Medical Imaging & Radiological Technology Department, Faculty of Medical Technology, Phenikaa School of Medicine & Pharmacy, Phenikaa University, Vietnam · Radiology & Functional Exploration Center, Phenikaa University Hospital, Vietnam +2

  26. Dual Attention Residuals

    Jul 21, 2026Xingda Yu, Yining Li, Xinzhang Liu +5TransformerCross-Attention

    TeleAI

  27. Thresholded Cross-Attention for Reliable Intensity-Chromaticity Fusion in Low-Light Image Enhancement

    Jul 15, 2026Yanyi Wu, Xu Zhang, Junkai Chen +6Cross-AttentionLow-Light Image Enhancement

    School of Information Engineering, Guangdong University of Technology, Guangzhou 510000, China · School of Computer Science, Wuhan University, Wuhan 430000, China · Department of Computer Vision, Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, United Arab Emirates +2

  28. 2D Rotary Position Embedding for Scene Text Recognition with Transformers

    Jul 15, 2026Zobeir RaisiRotary Positional EmbeddingsCross-Attention

    Electrical Engineering Department, Chabahar Maritime University, Chabahar, Iran.

  29. Token-Based Dual-view Fusion and Adaptation of Large Vision Models for Breast Cancer Classification

    Jul 7, 2026Aysan Ghayouri Pirsoltan, Shima Babakordi, Mohammad Reza MohammadiCross-AttentionCross-View Attention

    School of Computer Engineering, Iran University of Science and Technology, Tehran, Iran · School of Computer Engineering, Iran University of Science and technology, Tehran, Iran

  30. EquiSteer: Cross-Attention Steering Towards a Fairer Text-Guided Image Generation

    Jul 1, 2026Tatiana Gaintseva, Akshit Achara, Gregory Slabaugh +2Cross-AttentionT2I Generation

    Queen Mary University of London, UK · Huawei Noah’s Ark, UK · King’s College London, UK +1