Multimodal Classification

Follow this topic Ask about this topic

Momentum

23 papers in the last four weeks, up 360% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 134

All topics

Beyond Modality Fusion: Deep Ensembles for Multimodal Classification
Jul 6, 2026Ilya Burenko, Dmitry VetrovModality Imbalance Multimodal Classification
ScaDS.AI, Technische Universität Dresden · Dresden, Germany · Constructor University +1
AI Wizards at EXIST 2026: Hierarchical Soft-Label Learning for Multimodal Sexism Identification in Memes
Jul 5, 2026Matteo Fasulo, Antonio Gravina, Luca Tedeschini +1Soft-Label Learning Hierarchical Classification
Swiss Data Science Center, ETH Zürich, Andreasturm, Andreasstrasse 5, 8092 Zürich, Switzerland · Everest Systems GmbH, Max-Jarecki-Straße 21, 69115 Heidelberg, Germany · Villanova.ai S.P.A, Località Sa Illetta, SS 195 KM 2.3, 09123 Cagliari, Italy +1
Triple-Phase Multimodal Knowledge Aggregation Framework for Microbial Keratitis Subtype Diagnosis on Slit-Lamp Photography
Jul 4, 2026Yiqing Wang, Maria A. Woodward, Ziyun Yang +11Multimodal Classification Multimodal Fusion
Department of Biomedical Engineering, Duke University, Durham, NC, USA. · Kellogg Eye Center, Department of Ophthalmology and Visual Sciences, University of Michigan, Ann Arbor, MI, USA. · Department of Cornea and Refractive Surgery Services, Aravind Eye Care System, Madurai, Tamil Nadu, India. +4
An Automated Multimodal Glaucoma Detection Framework Using ViT and a Stacking-Based Ensemble
Jul 2, 2026Ishrat Jahan, Muhammad E. H Chowdhury, Murugappan Murugappan +7Multimodal Classification Medical Image Classification
Department of Computer Science and Engineering, Shahjalal University of Science and Technology, Sylhet, Bangladesh · Department of Electrical Engineering, Qatar University, Doha, Qatar · Department of Electronics and Communication Engineering, Kuwait College of Science and Technology, Block 4, Doha, Kuwait +5
Multimodal Fusion for Fine-Grained Classification of Breast Fibroadenoma and Phyllodes Tumors
Jul 2, 2026Chuxi Nan, Di Wu, Hongming Guo +4Multimodal Classification Ultrasound Imaging
School of Integrated Circuits, Wuxi Vocational College of Science and Technology · School of Advanced Technology, Xi’an Jiaotong-Liverpool University · 3Shanghai Gem Science and Technology Co., Ltd. +2
Multi-modal Rail Crossing Safety Analysis
Jul 1, 2026Paimon Goulart, Chansong Lim, Nícolas Roque dos Santos +4Multimodal Classification Multimodal Fusion
University of California, Riverside Riverside, California, USA · Riverside County Transportation Commission Riverside, California, USA
Leveraging Multimodality for Real-Time Classification of Transients and Variables found by the Zwicky Transient Facility
Jun 30, 2026Ved G. Shah, Nabeel Rehemtulla, Adam A. Miller +13Hierarchical Classification Time Series Classification
Department of Physics and Astronomy, Northwestern University, 2145 Sheridan Road, Evanston, IL 60208, USA · Center for Interdisciplinary Exploration and Research in Astrophysics (CIERA), 1800 Sherman Ave, Evanston, IL 60201, USA · NSF-Simons AI Institute for the Sky (SkAI), 172 E. Chestnut St., Chicago, IL 60611, USA +8
Residual-Guided Expert Specialization for Incomplete Multimodal Learning
Jun 29, 2026Seunghun Baek, Jihwan Park, Jaeyoon Sim +3Missing-Modality Learning Multimodal Robustness
Pohang University of Science and Technology (POSTECH), South Korea
Cross-Modal Iteration Distillation for Robust IHD Screening: The IDNet Framework and A New Benchmark
Jun 29, 2026Yongchang Gao, Junjie Pang, Shuaiyu Yang +5Cross-Modal Knowledge Distillation Medical Imaging
University of Chinese Academy of Sciences, Beijing, China · MGI Tech Co., Ltd., Shenzhen, China · Shenzhen University, Shenzhen, China
Traffic-CBM: A Structurally Interpretable Multimodal Framework for Encrypted Traffic Classification
Jun 29, 2026Honglei Jin, Wenshuo Chen, Shaofeng Liang +6Encrypted Network Traffic Analysis Multimodal Classification
HKUST (GZ) Guangzhou, China · Shandong University Qingdao, China
AMR: Adaptive Modality Routing for Multimodal Polyglot Speaker Identification
Jun 28, 2026Chuxiao Zuo, Yao Zhu, Minqiang Xu +3Multimodal Robustness Speech Processing
Honor Device Co., Ltd Nanjing, China · Honor Device Co., Ltd Shanghai, China
Synergistic Dual-Branch Adaptation for Multi-modal Generalized Category Discovery
Jun 19, 2026Yuxun Qu, Minyu Zhou, Yongqiang Tang +2Cross-Modal Alignment Multimodal Classification
State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 101408, China
Geometry-Aware Superpixel Graph Transformer with Metadata for Skin Lesion Classification
Jun 18, 2026Muhammad Azeem, Tanveer Hussain, Amr Ahmed +1Skin Lesion Classification Graph Representation Learning
Edge Hill University, Ormskirk, Lancashire, L39 4QP, United Kingdom
Alzheimer's Disease Diagnosis using a Multimodal Approach with 3D MRI and PET
Jun 18, 2026Loukas Ilias, Anthi-Maria Vozinaki, Christos Ntanos +1Medical Diagnosis 3D Medical Imaging
DSS Lab, School of ECE, NTUA, 15780 Athens, Greece
An approach with Visual and Tabular Mamba to multimodal medical data using Mixed Fusion
Jun 17, 2026Matheus B. Rocha, Gustavo B. Dettogni, Renato A. KrohlingMamba Multimodal Classification
Labcin - Nature Inspired Computing Lab, Federal University of Espírito Santo, Vitória, Brazil · PPGI - Graduate Program in Computer Science, Federal University of Espírito Santo, Vitória, Brazil
Fair Cognitive Impairment Detection Through Unlearning
Jun 17, 2026William Nguyen, Jiali Cheng, Hadi AmiriAdversarial Representation Learning Algorithmic Fairness
University of Massachusetts Lowell, USA
Probing, Fusion, and Trustworthiness: A Systematic Evaluation of Foundation Model Representations for Multimodal Cancer Analysis
Jun 15, 2026Jingyu Hu, Giuseppe Tripodi, Reed Naidoo +2Computational Pathology Multimodal Robustness
The Alan Turing Institute, London, United Kingdom · University of Manchester, Manchester, United Kingdom · The Institute of Cancer Research, London, United Kingdom +1
Multi-Modal Attention for Automated Disaster Damage Assessment Using Remote Sensing Imagery and Deep Learning
Jun 12, 2026Tewodros Syum Gebre, Jagrati Talreja, Leila Hashemi-BeniDisaster Damage Assessment Remote Sensing Change Detection
Built Environment Department, College of Science and Technology, North Carolina A&T State University, Greensboro, NC, USA · United Nations University Institute for Water, Environment and Health, Richmond Hill, ON, Canada
ProMUSE: Progressive Multi-modal Uncertainty-guided Staged Evidential Alzheimer Disease Classification
Jun 11, 2026Long Doan, Branden Chen, Ethan Litton +6Efficient Multimodal Inference Evidential Deep Learning
Department of Computer Science, Kennesaw State University · Department of Data Science and Analytics, Kennesaw State University · Department of Information Technology, Kennesaw State University +2
Multimodal Graph Negative Learning
Jun 11, 2026Zhengyu Wu, Xu Wang, Hongchao Qin +4Multimodal Classification Multimodal Graph Learning
Unifying Acoustic Features and Text with Multimodal LLMs for Neurodegenerative Screening
Jun 10, 2026Qingfeng Zhang, Yuanxiong Guo, Yanmin GongAlzheimer's Disease Multimodal Large Language Models
Dept. of Information Systems and Cybersecurity The University of Texas at San Antonio San Antonio, TX, USA · School of Engineering Medicine Texas A&M University Houston, TX, USA
Missing-Token Prompted Reliability-Aware Fusion for Robust Polyglot Speaker Identification
Jun 10, 2026Peng Jia, Li Dai, Jia Li +3Missing-Modality Learning Multimodal Robustness
Hefei University of Technology Hefei, China · Intelligent Interconnected Systems Laboratory of Anhui Province Hefei University of Technology Hefei, China
YTClickbait21K: Human-Annotated Multimodal Dataset for YouTube Clickbait Detection Across Diverse Channels and Content Categories
Jun 10, 2026Md. Minhazul Islam, Md. Tanbeer Jubaer, Amith Khandakar +5Social Media Analysis Multimodal Classification
Department of Computer Science and Engineering, Rajshahi University of Engineering & Technology, Rajshahi, Bangladesh · Department of Electrical Engineering, Qatar University, Doha, Qatar · Department of Civil and Environmental Engineering, Qatar University, Doha, Qatar +1
Multimodal Brain Tumour Classification Using Feature Fusion
Jun 9, 2026Wajih ul Islam, Muhammad Yaqoob, Javed Ali Khan +1Multimodal Classification Medical Image Classification
School of Physics, Engineering and Computer Science, University of Hertfordshire, UK
Context-Aware Deep Learning for Defect Classification in Atomic-Resolution STEM
Jun 8, 2026Jiadong Dan, Cheng Zhang, Leyi Loh +5Materials Science Multimodal Classification
Department of Biological Sciences, National University of Singapore, Singapore · Centre for BioImaging Sciences (CBIS), National University of Singapore, Singapore · Department of Materials Science and Engineering, City University of Hong Kong, China +5
Multi-View Speech Representation Learning for Parkinson's Disease Detection Using Context-guided Cross-modal Attention
Jun 8, 2026George Theodosiou, Loukas Ilias, Dimitris AskounisAudio Representation Learning Speech Processing
Decision Support Systems Laboratory, School of Electrical and Computer Engineering, National Technical University of Athens, 15780 Athens, Greece
Synesthesia via Direct Latent Augmentation:Bypassing the Decode-Encode Loop for Cross-Modal Distillation
Jun 6, 2026Cristian Sbrolli, Nicolas Michel, Matteo Matteucci +1Cross-Modal Knowledge Distillation Synthetic Data Augmentation
Politecnico di Milano, Milan, Italy · University of Tokyo, Tokyo, Japan
PAMF: Prior-Aware Multimodal Fusion for Incomplete Time Series Data
Jun 4, 2026Ziwen Kan, Wugeng Zheng, Tianlong Chen +1Missing-Modality Learning Flow Matching
Department of Computer Science, University of Central Florida · Department of Computer Science, University of North Carolina at Chapel Hill
Multimodal Sexism Identification and Characterization using Large Language Models and Gradient Boosting
Jun 4, 2026Kyriakos Chaviaras, Maria Lymperaiou, Athanasios VoulodimosHate Speech Detection Multimodal Classification
1Artificial Intelligence and Learning Systems Laboratory School of Electrical and Computer Engineering National Technical University of Athens
Disentangled Fine-Grained Prototype Learning for Incomplete Image-Tabular Classification
Jun 3, 2026Feixiang Zhou, Jianyang Xie, Zhuangzhi Gao +10Missing-Modality Learning Cross-Modal Alignment
School of Eye and Vision Sciences, University of Liverpool, U.K. · School of Computer Science, University of Exeter, U.K. · School of Computer Science and Engineering, South China University of Technology, China +4
FigSIM: A Dataset for Fine-grained Suicide Severity and Figurative Language in Suicide Memes
Jun 1, 2026Liuliu Chen, Elise R. Carrotte, Brian E. Chapman +2Figurative Language Understanding Social Media Analysis
School of Computing and Information Systems, University of Melbourne, Australia · Orygen, The National Centre of Excellence in Youth Mental Health, Australia · Centre for Youth Mental Health, University of Melbourne, Australia +1
Multimodal Approaches for Visually-Rich Document Type Classification: A Comparative Analysis
Jun 1, 2026Catyana Heyne, Jürgen Frikel, Filippo RiccioDocument Understanding Multimodal Classification
OTH Regensburg, Germany
When Tabular Foundation Models Transfer Across Modalities: A Systematic Evaluation Across 95 Datasets, 7 Modalities, and Two Regimes
Jun 1, 2026Julien LafranceCross-Modal Learning Tabular Foundation Models
Télécom Paris, Institut Polytechnique de Paris · Palaiseau, France
SDM-Q: Cost-Aware Staged Decision-Making for Multi-Omics Classification with Deep Q-Learning
May 29, 2026Nan Mu, Yangfan Xiao, Ling Wang +3Cost-Sensitive Learning Deep Q-Learning
College of Computer Science, Sichuan Normal University, Chengdu, Sichuan 610101, China. · Department of Mathematics, College of Science and Mathematics, Kennesaw State University, Kennesaw, GA 30144, USA. · Department of Computer Science, College of Computing and Software Engineering, Kennesaw State University, Marietta, GA 30060, USA.
MinhwaNet: Faithful but Insufficient Object Grounding in Korean Folk Painting
May 26, 2026Joonhyung BaeMultimodal Grounding Multimodal Classification
Korea Advanced Institute of Science and Technology (KAIST), South Korea · Korea Advanced Institute of Science and Technology (KAIST), Daejeon, South Korea
Multi-Modal Building Inspection via Perceiver IO Fusion of Satellite and Street-Level Imagery
May 25, 2026Niels Sombekke, Rob G. J. Wijnhoven, Martin R. OswaldRemote Sensing Image Understanding Multi-Source Image Fusion
University of Amsterdam (UvA), Amsterdam, The Netherlands · Spotr, The Hague, The Netherlands
ConceptM $^3$ oE: Concept-Guided Multimodal Mixture of Experts for Interpretable Computational Pathology
May 23, 2026Xuan Wang, Zhongling Xu, Gopi Kannedhara +13Computational Pathology Concept Bottleneck Models
University of Texas at Austin, Austin, TX, USA · University of North Carolina at Chapel Hill, NC, USA · Dell Children’s Medical Center, Austin, TX, USA +2
Modular Multimodal Classification Without Fine-Tuning: A Simple Compositional Approach
May 20, 2026Herman Bergström, Aditya Mehrotra, Rahul G. KrishnanMultimodal Embedding Multimodal Classification
Chalmers University of Technology and University of Gothenburg · Vector Institute · University of Toronto
QwenSafe: Multimodal Content Rating Description Identification via Preference-Aligned VLMs
May 20, 2026Dishanika Denipitiyage, Aruna Seneviratne, Suranga SeneviratneMultimodal Classification
University of Sydney, Sydney, Australia · University of New South Wales, Australia
CodeBind: Decoupled Representation Learning for Multimodal Alignment with Unified Compositional Codebook
May 18, 2026Zeyu Chen, Jie Li, Kai HanCross-Modal Learning Multimodal Disentangled Representation Learning
Visual AI Lab, The University of Hong Kong
A Conflict-aware Evidential Framework for Reliable Sleep Stage Classification
May 16, 2026Yunzhi Tian, Dekui Wang, Qirong Bu +3Sleep Stage Classification Evidential Deep Learning
College of Computer Science, Northwest University, Xi’an, China.
DeepArrhythmia: Segment-Contextualized ECG Arrhythmia Classification via Selective Evidence Acquisition
May 15, 2026Jiahui Li, Ruili Fang, Zishuai Liu +3Electrocardiography Multimodal Classification
University of Georgia
Machine Learning-Driven Multimodal Spectroscopic Liquid Biopsy for Early Multicancer Detection
May 13, 2026Alejandro Leonardo García Navarro, Javier Cachón Ortiz, Javier González Colsa +2Multimodal Classification Oncology
Signal Processing Group Gregorio Marañón Health Research Institute Madrid, Spain · Amber Health Solutions Madrid, Spain
Resilient Vision-Tabular Multimodal Learning under Modality Missingness
May 12, 2026Camillo Maria Caruso, Valerio Guarrasi, Paolo SodaMissing-Modality Learning Multimodal Robustness
Research Unit of Artificial Intelligence and Computer Systems, Department of Engineering, Università Campus Bio-Medico di Roma, Roma, Italy, Europe · Department of Diagnostics and Intervention, Radiation Physics, Biomedical Engineering, Umeå University, Umeå, Sweden, Europe
Multi-domain Multi-modal Document Classification Benchmark with a Multi-level Taxonomy
May 11, 2026Denghao Ma, Qing Liu, Zulong Chen +5Hierarchical Classification Text Classification
Beijing Information Science and Technology University, China · Alibaba Group, China · Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ), China +2
Simultaneous Long-tailed Recognition and Multi-modal Fusion for Highly Imbalanced Multi-modal Data
May 11, 2026Heegeon Yoon, Heeyoung KimClass-Imbalanced Learning Long-Tailed Classification
Department of Industrial and Systems Engineering, Korea Advanced Institute of Science and Technology (KAIST), 291 Daehak-ro, Yuseong-gu, Daejeon 34141, Republic of Korea
Learning predictive models for combinations of heterogeneous proteomic data sources
May 9, 2026Michal Valko, Richard Pelikan, Miloš HauskrechtMedical Diagnosis Multimodal Classification
Computer Science Department University of Pittsburgh, Pittsburgh, Pennsylvania
Multimodal Stepwise Clinically-Guided Attention Learning for Pathological Complete Response Prediction in Breast Cancer
May 8, 2026Alice Natalina Caragliano, Valerio Guarrasi, Michela Gravina +2Breast Cancer Clinical Outcome Prediction
Unit of Artificial Intelligence and Computer Systems, Department of Engineering, Università Campus Bio-Medico di Roma, Rome, Italy · Department of Electrical Engineering and Information Technology, University of Naples Federico II, Naples, Italy · Department of Diagnostics and Intervention, Radiation Physics, Biomedical Engineering, Umeå University, Umeå, Sweden
Multimodal Deep Generative Model for Semi-Supervised Learning under Class Imbalance
May 7, 2026Heegeon Yoon, Heeyoung KimVariational Autoencoders Class-Imbalanced Learning
Department of Industrial and Systems Engineering, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Republic of Korea
Virtual Scanning for NSCLC Histology: Investigating the Discriminatory Power of Synthetic PET
May 4, 2026Fatih Aksu, Laura Ciuffetti, Francesco Di Feola +6PET/CT Imaging Multimodal Classification
Università Campus Bio-Medico di Roma · Umeå University · Università Vita-Salute San Raffaele
From Static Analysis to Audience Dissemination: A Training-Free Multimodal Controversy Detection Multi-Agent Framework
May 1, 2026Zihan Ding, Ziyuan Yang, Yi ZhangSocial Media Analysis Multimodal Classification
Sichuan University, Chengdu, Sichuan, China · Nanyang Technological University, Singapore
MM-StanceDet: Retrieval-Augmented Multi-modal Multi-agent Stance Detection
Apr 30, 2026Weihai Lu, Zhejun Zhao, Yanshu Li +1Multimodal Reasoning Multimodal Classification
Peking University · Baidu Inc · Brown University +1
JI-ADF: Joint-Individual Learning with Adaptive Decision Fusion for Multimodal Skin Lesion Classification
Apr 30, 2026Phan Nguyen, Dat Cao, Hien Kha +4Skin Lesion Classification Medical Diagnosis
1KAIST · 2Taipei Medical University · 3Yale University
VTBench: A Multimodal Framework for Time-Series Classification with Chart-Based Representations
Apr 29, 2026Madhumitha Venkatesan, Xuyang Chen, Dongyu LiuData Visualization Time Series Classification
University of California, Davis Davis, United States
A Multimodal Pre-trained Network for Integrated EEG-Video Seizure Detection
Apr 29, 2026Tong Lu, Ke Xu, Zimo Zhang +12Electroencephalography EEG Seizure Detection
Beijing Institute for Brain Research, Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing, 102206, China · Chinese Institute for Brain Research, Beijing, Beijing, 102206, China · Department of Neurosurgery, SanBo Brain Hospital, Capital Medical University, Beijing, 100018, China +4
ASR-Agnostic Multimodal Spectrotemporal Modeling for Early Dementia Detection
Apr 29, 2026Chukwuemeka Ugwu, Oluwafemi Richard OyelekeSpeech-Based Dementia Detection Speech Processing
DARC-CLIP: Dynamic Adaptive Refinement with Cross-Attention for Meme Understanding
Apr 25, 2026Qiyuan JinVLM Adaptation Hate Speech Detection
The Hong Kong University of Science and Technology
STEP-PD: Stage-Aware and Explainable Parkinson's Disease Severity Classification Using Multimodal Clinical Assessments
Apr 19, 2026Md Mezbahul Islam, John Michael Templeton, Christian Poellabauer +1Multimodal Classification Clinical Prediction
School of Computing & Information Sciences, Florida International University, Miami, FL, USA · College of AI, Cybersecurity and Computing, University of South Florida, Tampa, FL, USA
Multimodal Fusion of Histopathology Images and Electronic Health Records for Early Breast Cancer Diagnosis
Apr 18, 2026Aditya Shribhagwan Khandelwal, Mohammad Samar Ansari, Asra AslamBreast Cancer Computational Pathology
University of Sheffield, UK · University of Chester, UK
IMA-MoE: An Interpretable Modality-Aware Mixture-of-Experts Framework for Characterizing the Neurobiological Signatures of Binge Eating Disorder
Apr 18, 2026Lin Zhao, Qiaohui Gao, Elizabeth Martin +5Multimodal Classification Mixture-of-Experts Models
Department of Biomedical Engineering, New Jersey Institute of Technology, Newark, 07102, NJ, USA · College of Engineering, Northeastern University, Boston, 02115, MA, USA · Department of Psychiatry, Icahn School of Medicine at Mount Sinai, New York, 30602, NY, 10029 +1

Jul 6, 2026, cs.LGJ/K move · Enter open · S save

Beyond Modality Fusion: Deep Ensembles for Multimodal Classification

Ilya Burenko, Dmitry Vetrov

ScaDS.AI, Technische Universität Dresden · Dresden, Germany · Constructor University+1

Modality Imbalance Multimodal Classification Deep Ensembles Multimodal Fusion

Open paper AskPDF arXiv

In multimodal classification, late-fusion approaches classify concatenated modality-specific features extracted by unimodal neural networks. When modality imbalance is pronounced, various regularization techniques have been proposed to balance the learning process and overcome the inferior performance of late-fusion networks. In contrast, this work demonstrates that multimodal data can be effectively classified without any explicit modality fusion, using deep ensembles of unimodal networks. We systematically compare deep ensembles to late-fusion networks at equal parameter count and show that ensembles consistently outperform state-of-the-art late-fusion methods designed to address modality imbalance. This advantage also holds over intermediate-fusion techniques we evaluated and over hybrid methods that combine unimodal and multimodal predictions. We propose and empirically validate a method for selecting the number of models per modality in an ensemble, avoiding computationally expensive exhaustive search. Under extreme modality imbalance and small ensemble sizes, the heuristic indicates that ensembles of unimodal models trained solely on the stronger modality are preferable; as the ensemble scales up, incorporating models from the weaker modality becomes beneficial. Both predictions align with our empirical findings. To systematically explore the challenges of optimizing multimodal models, we propose a synthetic multimodal framework that allows control over both the number of modalities and their predictive strength; our findings are consistent across synthetic and real-world datasets. Finally, by fitting scaling laws to bimodal datasets, we estimate the asymptotic performance of ensembles.

Momentum

Related topics

Latest papers 134