Multimodal Transformers

Ask about this topic

Recent momentum

-67%

2 papers in the last 28 days · 0.1% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this topic, kept on the site without email delivery.

Period ending 2026-09-14

3 new papers

A weekly snapshot of new work published in Multimodal Transformers.

45 papers

Latest in Multimodal Transformers

Open your feed →
CardsList

  1. Vision And Text Transformer For Predicting Answerability On Visual Question Answering

    Sep 15, 2026Tung Le, Huy Tien Nguyen, Le Minh NguyenKnowledge-Based Visual Question AnsweringSelf-Supervised Vision Transformers

  2. Multimodal Emergency Vehicle Classification via Audio-Visual Transformers and Knowledge Distillation

    Sep 15, 2026Vijay John, Amar DabajaMultimodal TransformersCross-Modal Fusion

  3. M3-Former: Multimodal Transformer with Mixture-of-Experts for Long-Term Vessel Trajectory Prediction

    Sep 11, 2026Wenzhe Jin, Haina TangTrajectory PredictionMultimodal Transformers

  4. CEM-TUDASR: Computationally efficient multi-modality transformer based unsupervised domain adaptive super-resolution approach

    Sep 11, 2026Anjali Sarvaiya, Jay Kadel, Kishor Upla +1Video Capsule EndoscopyMultimodal Transformers

  5. Coupled Continuous-Discrete Generation for Scene Text Image Super-Resolution

    Aug 5, 2026Axi Niu, Knag Zhang, Qingsen Yan +3Generative Super-ResolutionMultimodal Transformers

  6. SMM Transformer: Leveraging Spiking Neural Networks for Multimodal Tasks

    Aug 3, 2026Xiubo Liang, Jinxing Han, Yuke Li +3Spiking Neural NetworksMultimodal Transformers

  7. Decoupled Visual Processing: Efficient Multimodal Adaptation via Modality-Specific Transformer Substitution

    Jul 29, 2026Mingkuan Feng, Zhengqi Wen, Jianhua TaoMultimodal TransformersVisual Perception

  8. Multimodal Auto-regressive Transformer Surrogate for Modeling Variable Operations and Quantifying Uncertainty in Geological Carbon Storage

    Jul 26, 2026Yifu Han, Louis J. DurlofskySurrogate ModelsMultimodal Transformers

  9. PRIMS: Physics-guided Representation for Fluid Identification in Multimodal Sensing

    Jul 24, 2026Hai-Long Nguyen, Trung Thanh Nguyen, Lars Holm +2Multimodal TransformersFluid Dynamics

  10. Multi-modal transformer for signal classification in nanopore blockade experiments

    Jul 22, 2026Sandro Kuppel, Julian Hoßbach, Samuel Tovey +1Multimodal Deep LearningBiomarker

  11. Efficient Audio-Visual Event Recognition via Knowledge Distillation and Dynamic INT8 Quantization of a Hybrid Cross-Attention Network

    Jul 18, 2026Parinaz Binandeh Dehaghani, Danilo Pena, A. Pedro AguiarMobilenetv2Self-Supervised Vision Transformers

  12. Phase-Preserving Trimodal Transformer for Tropical Forest Biomass Estimation Using Optical and PolInSAR Data

    Jul 4, 2026Luiz Felipe Parente Santiago, Felipe Ferrari, Daniel Rodrigues dos Santos +1Above-Ground BiomassCanopy

  13. Dual Sparse Aggregation Transformer for Multispectral Object Detection

    Jun 30, 2026Wencong Wu, Xiuwei Zhang, Hanlin Yin +2Feature Pyramid NetworksHyperspectral Imaging

  14. Quality-Aware Modulation for Diffusion Transformers

    Jun 29, 2026Luke Budny, Yuhong Guo, Kevin CheungMultimodal Diffusion TransformersText-To-Image Diffusion Models

  15. An iterative energy-based multimodal transformer for joint retrieval of wheat soil moisture, leaf area index, and plant height from Sentinel-1 and Sentinel-2 time series

    Jun 23, 2026Shubham Kumar Singh, Peilei Fan, Suraj A. Yadav +2Soil Moisture EstimationSentinel-2

  16. Towards Scalable Multi-Task Reinforcement Learning with Large Decision Models

    Jun 23, 2026Thibaut KulakMulti-Task Reinforcement LearningMulti-Agent Reinforcement Learning

  17. Pose Anything Anywhere:Model-free Object Poses from Arbitrary References

    Jun 22, 2026Hongli Xu, Jiaqi Hu, Junwen Huang +5Category-Level Object Pose EstimationRobotic Perception

  18. UniverSat: Resolution- and Modality-Agnostic Transformers for Earth Observation

    Jun 22, 2026Yohann Perron, Guillaume Astruc, Nicolas Gonthier +2Cloud-Aware Earth Observation SequencesEarth Observation

  19. Three-Step Hierarchical Transformer for Multi-Pedestrian Trajectory Prediction

    Jun 22, 2026Raphaël Delécluse, Hazem Wannous, Laurent Grisoni +1Pedestrian Trajectory PredictionSpatio-Temporal Transformers

  20. FiLM-Coordinated Dual-Branch Transformer for Global-Local Dependency Modeling in Language Modeling

    Jun 19, 2026Zhiqiang Zhou, Xu Ling, Junliang DaiTransformer LayersSoftmax Attention

  21. ASTEROID: A Spatiotemporal Information Transformer for Forecasting Multi-Step Time Series of Molecular Dynamics

    Jun 16, 2026Kexin Wu, Luonan Chen, Renxiao WangMolecular DynamicsSpatio-Temporal Forecasting

  22. MMDiff: Extending Diffusion Transformers for Multi-Modal Generation

    Jun 15, 2026Yagmur Akarken, Orest Kupyn, Christian RupprechtMultimodal Diffusion TransformersMultimodal Transformers

  23. DifFRACT: Diffusion Feature Reconstruction and Attribution for Circuit Tracing

    Jun 14, 2026Artyom Mazur, Nina Konovalova, Aibek AlanovMultimodal Diffusion TransformersMultimodal Transformers

  24. The Signs Were Always There: Training-Free Concept Detection and Steering in Raw Transformer Dimensions

    Jun 10, 2026Varun Reddy NalagatlaTransformer ArchitecturesMultimodal Transformers

  25. Multiscale POD of Transformer Attention Fields: Scale-Selective Analysis via Morlet Scalogram

    Jun 4, 2026Athanasios ZerisTransformer AttentionMultimodal Transformers

  26. GRAMformer: Any-Order Modality Interactions via Volumetric Multimodal Cross-Attention

    Jun 4, 2026Giordano Cicchetti, Eleonora Grassucci, Danilo ComminielloCross-Modal AttentionMultimodal Transformers

  27. KITE: A Tri-Modal Transformer Integrating Text, Images, and Knowledge Graphs for Fake News Detection

    Jun 2, 2026Kevin Patel, Shashi Bhushan JhaFake News DetectionMisinformation Detection

  28. MLLM-Microscope: Unlocking Hidden Structure Within Multimodal Large Language Models

    May 30, 2026Ravil Mussabayev, Rustam MussabayevMulti-Modal Large Language ModelsMultimodal Transformers

  29. Assign and Add: A Mechanistic Study of Compositional Arithmetic

    May 29, 2026Brady Exoo, Alberto Bietti, John SousMultilayer PerceptronsMultimodal Transformers

  30. Particle-Lund Multimodality in Jet Taggers

    May 26, 2026Loukas Gouskos, Benedikt MaierHigh Energy PhysicsMultimodal Transformers

  31. Generic Interpretation Approach for Transformer Models Incorporating Heterogenous Attention Structures

    May 25, 2026Yongjin Cui, Xiaohui Fan, Huajun ChenTransformer AttentionMultimodal Transformers

  32. AOEPT: Breaking the Implicit Modality-Reduction Bottleneck in Modality-Missing Prompt Tuning

    May 24, 2026Jian Lang, Rongpei Hong, Ting Zhong +1Missing ModalitiesMultimodal Transformers

  33. Polymorphism Is Rotation: Operational Mechanistic Interpretability from a Two-Layer Transformer to Pythia-70m

    May 23, 2026Jordan F. McCannTransformer ArchitecturesTransformer Layers

  34. Task Vector Geometry Underlies Dual Modes of Task Inference in Transformers

    May 5, 2026Hao Yan, Haolin Yang, Yiqiao ZhongMultimodal TransformersRepresentational Geometry

  35. Neuroscience-Inspired Analyses of Visual Interestingness in Multimodal Transformers

    May 5, 2026Mathis Immertreu, Fitim Abdullahu, Thomas Kinfe +3Multimodal TransformersTransformer Attention

  36. Multi-Perspective Transformers in ARC-AGI-2 Challenge

    May 1, 2026Caleb Talley, Vedant Tibrewal, Seun Adekunle +3PuzzleFinetuning

  37. Semantic Context-aware mOdality fUsion Transformer (SCOUT): A Context-Aware Multimodal Transformer for Concept-Grounded Pathology Report Generation

    May 1, 2026Suryakant Singh, Saarthak Kapse, Joel Saltz +1Whole-Slide ImagesComputational Pathology

  38. Star-Fusion: A Multi-modal Transformer Architecture for Discrete Celestial Orientation via Spherical Topology

    Apr 29, 2026May Hammad, Menatallh HammadSatelliteSpatio-Temporal Transformers

  39. Multimodal Transformer for Sample-Aware Prediction of Metal-Organic Framework Properties

    Apr 21, 2026Seunghee Han, Jaewoong Lee, Jihan KimMolecular Property PredictionCryo-Electron Microscopy

  40. Multi-view Crowd Tracking Transformer with View-Ground Interactions Under Large Real-World Scenes

    Apr 21, 2026Qi Zhang, Jixuan Chen, Kaiyi Zhang +3Multi-Object TrackingMulti-View

  41. What Physics do Data-Driven MoCap-to-Radar Models Learn?

    Apr 19, 2026Kevin Chen, Kenneth W. Parker, Anish AroraDoppler Velocity LogMotion Capture

  42. Feature-level Interaction Explanations in Multimodal Transformers

    Mar 4, 2026Yeji Kim, Housam Khalifa Bashier Babiker, Mi-Young Kim +1Multimodal TransformersExplainability

  43. MetaOthello: A Controlled Study of Multiple World Models in Transformers

    Feb 26, 2026Aviral Chawla, Galen Hall, Juniper LovatoTransformer ArchitecturesMultimodal Transformers

  44. Unveiling the Mechanisms of Multi-Hop Reasoning in Transformers via Identity Bridge

    Sep 29, 2025Pengxiao Lin, Zheng-An Chen, Zhi-Qin John XuTransformer ArchitecturesMultimodal Transformers

  45. Explainability in mulimodal deep transformation models for stroke outcome prediction

    Apr 7, 2025Lisa Herzog, Jonas Brändli, Maurice Schneeberger +7ExplainabilityStroke