Multimodal Classification

Ask about this topic

Overview

Multimodal classification studies how to combine different kinds of evidence—for example, medical images with clinical records, or meme images with text—to predict categories. Common approaches include feature or decision fusion, cross-attention and contrastive alignment, mixture-of-experts models, and large vision-language models; benchmarks range from medical diagnosis and sentiment analysis to document, time-series, and content moderation tasks. Current work also addresses missing or unreliable inputs through modality dropout, imputation, adaptive routing, and selective acquisition, while evaluating fairness, calibration, and interpretability. A central tension is that adding a modality can improve predictions when it contributes complementary information, but can also hurt when it is noisy, redundant, or unavailable at deployment.

Latest papers

CardsList

200 papers24 papers in the last four weeks, down 4% on the four weeks before. 0.1% of all new papers.

  1. Explainable Multimodal AI Approach to Reduce Unnecessary Breast Biopsies

    Oct 26, 2026Sadam Hussain, Mansoor Ali, Daly Betzabeth Avendaño-Avalos +1Breast CancerRadiology VLMs

    School of Engineering and Sciences, Tecnologico de Monterrey, Monterrey, Nuevo Leon, Mexico · School of Medicine and Health Sciences, Tecnologico de Monterrey, Monterrey, Nuevo Leon, Mexico

  2. An Error Budget for Shuffled Multimodal Windows in the 3rd WEAR Challenge: Stacking, Count Constraints and Seam Matching

    Oct 23, 2026Soheil KooklanHuman Activity RecognitionMultimodal Classification

    Independent Researcher

  3. CRANE: Post-Hoc Counterfactual Retrieval for Multimodal Classifiers

    Oct 22, 2026Franco Rugolon, Ioanna Miliou, Panagiotis PapapetrouCounterfactual ExplanationsMultimodal Model Interpretability

    Dept. of Computer and Systems Sciences, Stockholm University, Stockholm, Sweden

  4. GAMF-Net: Grouped Availability-Aware Multimodal Fusion for Smartphone Skin Lesion Classification

    Oct 20, 2026Yue Zhuo, Jingjia Yan, Fangyu Liu +4Skin Lesion ClassificationMultimodal Classification

    Huizhou University, Huizhou 516007, Guangdong, China

  5. HypTI: Hyperbolic Geometry-Aware Multimodal Feature Fusion Framework for Dementia Detection Based on Tabular-Image Data

    Oct 20, 2026Jianfei Xu, Huizhi Liang, Judith Harrison +3Multimodal FusionMultimodal Classification

    Newcastle University, Newcastle upon Tyne NE1 7RU, UK

  6. SZED at ArGuard Shared Tasks: Multi-Task Label-Attention and Cross-Modal Fusion for Arabic Hateful Meme Detection

    Oct 18, 2026Selma Khelili, Zohra AbdeliHarmful Meme DetectionMulti-Label Classification

    ENSIA, Algeria

  7. Joint, Decoupled, or Agent: When Each Multimodal Alignment Paradigm Wins

    Oct 16, 2026Yi Fang, Gaurav Srivastava, Sirui Ding +1Multimodal FusionMultimodal Model Evaluation

    Department of Computer Science, Virginia Tech, Blacksburg, VA, USA · Department of Biomedical Data Science, Stanford University, Stanford, CA, USA

  8. Task-Centric Multimodal Learning for Cognitive Impairment Detection

    Oct 14, 2026Sankalp Vaish, Rishank Singh, Sumukh Veeramalla +3Multimodal LearningSpeech-Based Dementia Detection

    Department of Computer Science, University of Massachusetts Boston, MA, USA · School of Medicine, University of North Carolina at Chapel hill, NC, USA

  9. Single-Pass Decision Scores Enable Rapid Classification with Adjustable Thresholds in Medical Multimodal Large Language Models

    Oct 14, 2026Patrick Wienholt, Frederik Hauke, Christiane Kuhl +3Medical LLMsMultimodal Classification

    Lab for Artificial Intelligence in Medicine, Department of Diagnostic and Interventional Radiology, University Hospital RWTH Aachen, Aachen, Germany. · Department of Diagnostic and Interventional Radiology, University Hospital RWTH Aachen, Aachen, Germany. · Department of Medical Oncology, National Center for Tumor Diseases (NCT), Heidelberg University Hospital, Heidelberg, Germany. +2

  10. Towards Modality-Agnostic Seizure Classification: A Unified Deep Learning Framework for Heterogeneous EEG and Intracranial EEG Signals

    Oct 12, 2026Ramya Sundaram, Muhammad Aslam Jarwar, M Uma +2EEG Seizure DetectionMultimodal Classification

    School of Computing and Digital Technologies, Sheffield Hallam University, Sheffield and University of Strathclyde, Glasgow, United Kingdom. · School of Computing and Digital Technologies, Sheffield Hallam University, Sheffield, United Kingdom · Department of Cmputational Intelligence, SRM Institute of Science and Technology,Kattankulathur-603203, India. +2

  11. Siam-ALS: An Explainable Siamese Deep Learning Framework for Early ALS Detection Using EEG and Eye-Tracking Signals

    Oct 11, 2026Abdur Rahman, A. A. M. Muzahid, Istiak Ahmed +4Clinical PredictionMultimodal Classification

    School of Electrical and Electronic Engineering, Shanghai University of Engineering Science, Shanghai, 201620, China. · School of Information Technology, Murdoch University, Murdoch, 6150, WA, Australia. · Department of Information and Communication Engineering, Daffodil International University, Dhaka, 1216, Bangladesh.

  12. Can Multimodal Decision Models Classify Breast Ultrasound Images? Benchmarking CLEF and CLEF-Flash Across Prompts and Datasets

    Oct 9, 2026Tran Cao Minh, Hai Tran, Ha Minh TanMedical Image ClassificationMultimodal Classification

    Research and Development Center, VNSO Technology JSC, Ho Chi Minh City, Vietnam · Department of Information Technology, Ho Chi Minh University of Education, Ho Chi Minh City, Vietnam · Faculty of Information Science and Engineering, University of Information Technology, Ho Chi Minh City, Vietnam +1

  13. Learning What to Trust in Multimodal Learning under Noisy Supervision

    Oct 8, 2026Jiashuo Zou, Xiaobo XiaLearning with Noisy LabelsMultimodal Classification

    University of Science and Technology of China

  14. RSJEV: Discriminative Remote Sensing Scene Classification with Multimodal Large Language Models

    Oct 6, 2026Dongchen Si, Di Wang, Mingzhen Xu +3Remote Sensing Image UnderstandingEfficient Multimodal Inference

    School of Computer Science, Wuhan University, Wuhan 430072, China · State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan 430079, China

  15. Multimodal Knowledge Distillation for Gastric Adenocarcinoma Classification from Whole-Slide Images

    Oct 6, 2026Shrihari Dumbre, Bikash SantraCross-Modal Knowledge DistillationMultimodal Classification

    Indian Institute of Technology, Jodhpur

  16. Efficient Multimodal Inference through Adaptive Acquisition and Sequential Fusion

    Oct 5, 2026Payal Mohapatra, Haodong Yang, Yueyuan Sui +3Efficient Multimodal InferenceMultimodal Classification

    Arm Inc., USA · Northwestern University, USA

  17. Anatomy-aware Fine-grained Multimodal Fusion for Laryngopharyngeal Cancer T-Staging Prediction Using CT and Radiology Report

    Oct 5, 2026Xingyue Zhao, Yanzhou Su, Fang Zhang +12Radiology VLMsMultimodal Classification

    DAMO Academy, Alibaba Group · Department of Neurosurgery, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China · Hupan Laboratory, Hangzhou, China +6

  18. ASD-FEAT: A Multi-Modal Infant Video-Derived Dataset for Early ASD Risk Prediction

    Oct 2, 2026Sidrah Liaqat, Halil Helvaci, Sen-Ching Cheung +4Autism Spectrum DisorderMultimodal Classification

    University of Kentucky · Department of Electrical and Computer Engineering University of Kentucky Lexington, KY 40506, USA · University of California Davis +3

  19. Masked Privileged-Information Distillation for Multimodal Skin Lesion Classification Under Missing Clinical Metadata

    Oct 2, 2026Anirban Barua, Md Mahir Abrar Khan, Ayman Iktidar +1Skin Lesion ClassificationMultimodal Classification

    Department of Computer Science and Engineering Chittagong Independent University (CIU), Chattogram, Bangladesh

  20. Mutual Equilibrium: Multimodal Representation Learning through Reciprocal Feedback

    Sep 30, 2026Ho-min Park, Byungkon KangMultimodal GroundingMultimodal Embedding

    Data Science Center, Texas Children’s Hospital Baylor College of Medicine Houston, Texas 77030, USA · Department of Computer Science SUNY Korea Incheon, Republic of Korea

  21. OmniMed-Jev: Calibrating LVLM Confidence for Trustworthy Medical Multimodal Decisions via System One

    Sep 30, 2026Luyao Tang, Cheng ChenProbability CalibrationMultimodal Classification

    Department of Electrical and Computer Engineering The University of Hong Kong, Hong Kong SAR, China

  22. Detail in Context: A Dual-Scale Machine Learning Framework for Mycosis Fungoides Detection

    Sep 29, 2026Mohamed Hazem, Tarek Waleed, Omar Khaled +8Computational PathologyMedical Diagnosis

    Faculty of Engineering, Cairo University, Giza, Egypt · Dermatology Department, Faculty of Medicine, Cairo University, Cairo, Egypt

  23. Paired Multimodal Scaling Laws

    Sep 28, 2026Marcus Ma, Shrikanth NarayananMultimodal ClassificationScaling Laws

    University of Southern California

  24. MERID: Multimodal Exploration via Recursive Self-Improvement Agents for Major Depression Analysis

    Sep 28, 2026Lei Liu, Zhaokang Liang, Qingcheng Zeng +4Depression DetectionMultimodal Classification

    Yale University · Zhejiang University · Northwestern University +3

  25. A Multimodal Autonomic Sensing Framework for Objective Assessment of Patient Responses to Dental Pulp Stimulation

    Sep 28, 2026Youngsun Kong, Yubin Choi, Dongjin Song +3Multimodal ClassificationMultimodal Sensor Fusion

    Department of Biomedical Engineering, University of Connecticut, Storrs, CT 06269 USA · Department of Endodontics, University of Connecticut Health, Farmington, CT 06030 USA · School of Computing, University of Connecticut, Storrs, CT 06269 USA

  26. MAD-Guard: Controlled Study of Autoregressive Generation versus Direct Decision Interfaces for Closed Multimodal Forensic Tasks

    Sep 27, 2026Hao ChenEfficient Multimodal InferenceImage Forgery Detection

    School of Cyber Security, Guangdong Polytechnic Normal University, Guangzhou, China

  27. GraphSelect for Budgeted Representation Selection in Multimodal Graph Inference

    Sep 27, 2026Xu Wang, Xunkai Li, Yinlin Zhu +1Efficient Multimodal InferenceMultimodal Classification

    School of Airspace Science and Engineering, Shandong University, Weihai, China · Department of Computer Science, Beijing Institute of Technology, Beijing, China · School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou, China

  28. Confidence Falls Short: Asymmetric Certainty Gains from Optimization Hinder Multimodal Classification

    Sep 23, 2026Longfei Huang, Xiangyu Wu, Yang YangModality ImbalanceMultimodal Optimization

    Nanjing University of Science and Technology · Alibaba Group

  29. Small Cues, Big Consequences: Learning Pivotal Cues for Multimodal Meme Classification

    Sep 22, 2026Akshit Sharma, Prashant W. PatilMultimodal RobustnessCross-Modal Alignment

    CVPR Lab, MFSDSAI Indian Institute of Technology Guwahati

  30. GeoBalance: Geometry-Aware Monitoring and Reconstruction with Asymmetric Optimization for Balanced Multimodal Learning

    Sep 20, 2026Zechang Xiong, Da Li, Rong Yin +7Modality ImbalanceGeometric Representation Learning

    Beihang University, Beijing, China · Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China · Amap, Alibaba Group, Beijing, China +3