Multimodal Learning

Follow this topic Ask about this topic

Recent momentum

-18%

49 papers in the last 28 days · 0.8% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this topic, kept on the site without email delivery.

Period ending 2026-09-21

18 new papers

A weekly snapshot of new work published in Multimodal Learning.

456 papers

Latest in Multimodal Learning

Browse all topics →

Multimodal Cultural Heritage Knowledge Graph Extension with Language and Vision Models
May 17, 2026Yang Zhang, Nada Mimouni, Jean-Claude Moissinac +1Knowledge Graph Embeddings Cultural Heritage
Multilingual and Multimodal LLMs in the Wild: Building for Low-Resource Languages
May 16, 2026Firoj Alam, Shammur Absar Chowdhury, Enamul Hoque PrinceLow-Resource Languages Multi-Modal Large Language Models
Navigating the Emotion Tree: Hierarchical Hyperbolic RAG for Multimodal Emotion Recognition
May 16, 2026Zeheng Wang, Bo Zhao, Yijie Zhu +6Emotion Recognition Multimodal Reasoning
Linked Multi-Modal Data on Russian Domestic and Foreign Policy Speeches
May 15, 2026Daria Blinova, Gayathri Emuru, Rakesh Emuru +4Russian Political Discourse
ForMaT: Dataset for Visually-Grounded Multilingual PDF Translation
May 15, 2026Michał Ciesiółka, Dawid Wiśniewski, Adrian Charkiewicz +1Machine Translation Models Multimodal Learning
Reversing the Flow: Generation-to-Understanding Synergy in Large Multimodal Models
May 15, 2026Yujun Tong, Dongliang Chang, Zijin Yin +3Multimodal Artificial Intelligence Multimodal Learning
Attribute-Grounded Selective Reasoning for Artwork Emotion Understanding with Multimodal Large Language Models
May 15, 2026Cheng Zhang, Yuer Liu, Zhiyu Zhou +2Emotion Recognition Emotion
GOMA: Toward Structure-Driven Multimodal Alignment from a Graph Signal Smoothing Perspective
May 15, 2026Xu Wang, Xunkai Li, Yinlin Zhu +2Multimodal Alignment Modality-Specific Encoders
Think When Needed: Adaptive Reasoning-Driven Multimodal Embeddings with a Dual-LoRA Architecture
May 14, 2026Longxiang Zhang, Weilong Dai, Guanghao Zhang +2Multimodal Embeddings Multimodal Learning
GeoViSTA: Geospatial Vision-Tabular Transformer for Multimodal Environment Representation
May 14, 2026Yuhao Liu, Sadeer Al-Kindi, Ashok Veeraraghavan +1Geospatial Foundation Models Spatial Representation
Multimodal Graph-based Classification of Esophageal Motility Disorders
May 13, 2026Alexander Geiger, Lars Wagner, Daniel Rueckert +3Electrocardiogram Classification Multimodal Learning
MMSkills: Towards Multimodal Skills for General Visual Agents
May 13, 2026Kangning Zhang, Shuai Shao, Qingyao Li +8Multimodal Agents Reusable Agent Skills
FIND: Toward Multimodal Financial Reasoning and Question Answering for Indic Languages
May 13, 2026Sarmistha Das, Vaibhav Vishal, Syed Ibrahim Ahmad +2Multilingual Financial Question Answering Multimodal Reasoning Benchmarks
Supervised Deep Multimodal Matrix Factorization for Interpretable Brain Network Analysis
May 13, 2026Amjad Seyedi, Lifang He, Songlin Zhao +2Human Brain Networks Nonnegative Matrix Factorization
Utility-Oriented Visual Evidence Selection for Multimodal Retrieval-Augmented Generation
May 13, 2026Weiqing Luo, Zongye Hu, Xiao Wang +3Multi-Source Retrieval-Augmented Generation Visual Evidence
Machine Learning-Driven Multimodal Spectroscopic Liquid Biopsy for Early Multicancer Detection
May 13, 2026Alejandro Leonardo García Navarro, Javier Cachón Ortiz, Javier González Colsa +2Non-Small Cell Lung Cancer Multi-Omic Framework
Leveraging Multimodal Self-Consistency Reasoning in Coding Motivational Interviewing for Alcohol Use Reduction
May 13, 2026Guangzeng Han, James G. Murphy, Benjamin O. Ladd +2Interviews Multimodal Reasoning
Towards Visually-Guided Movie Subtitle Translation for Indic Languages
May 12, 2026Tarun Chintada, Kshetrimayum Boynao Singh, Asif EkbalAudio-Visual Consistency Low-Resource Languages
Multimodal Abstractive Summarization of Instructional Videos with Vision-Language Models
May 12, 2026Maham Nazir, Muhammad Aqeel, Richong Zhang +1Frozen Contrastive Language-Image Pre-Training Visual Encoder Summarization
Learn to Think: Improving Multimodal Reasoning through Vision-Aware Self-Improvement Training
May 12, 2026Qihuang Zhong, Liang Ding, Wenjie Xuan +3Multimodal Reasoning Multimodal Learning
STAGE: Tackling Semantic Drift in Multimodal Federated Graph Learning
May 12, 2026Zekai Chen, Xun Wu, Xunkai Li +3Federated Graph Learning Multimodal Learning
Very Efficient Listwise Multimodal Reranking for Long Documents
May 12, 2026Yiqun Sun, Pengfei Wei, Lawrence B. HsiehReranking Multimodal Learning
Leveraging Multimodal Large Language Models for All-in-One Image Restoration via a Mixture of Frequency Experts
May 12, 2026Eunho Lee, Rei Kawakami, Youngbae HwangImage Restoration Multimodal Pretraining
Behavioral Mode Discovery for Fine-tuning Multimodal Generative Policies
May 12, 2026Alberta Longhini, David Emukpere, Jean-Michel Renders +1Reinforcement Fine-Tuning Action Space
PresentAgent-2: Towards Generalist Multimodal Presentation Agents
May 12, 2026Wei Wu, Ziyang Xu, Zeyu Zhang +2Multimodal Generation Multimodal Agents
Towards On-Policy Data Evolution for Visual-Native Multimodal Deep Search Agents
May 11, 2026Shijue Huang, Hangyu Guo, Guanting Dong +8Multimodal Search Agents Multimodal Learning
Dynamic Cross-Modal Prompt Generation for Multimodal Continual Instruction Tuning
May 11, 2026Tao Hu, Da-Wei ZhouMultimodal Instruction Tuning Instruction Tuning
Geospatial-Temporal Sensemaking of Remote Sensing Activity Detections with Multimodal Large Language Model
May 11, 2026David F. Ramirez, Tim Overman, Kristen Jaskie +1Sentinel-2 Multimodal Dataset
AllocMV: Optimal Resource Allocation for Music Video Generation via Structured Persistent State
May 11, 2026Huimin Wang, Leilei Ouyang, Chang Xia +3Knapsack Constraint Persistent State
Empowering VLMs for Few-Shot Multimodal Time Series Classification via Tailored Agentic Reasoning
May 10, 2026Lin Li, Jiawei Huang, Qihao Quan +7Time-Series Classification Few-Shot Learning
FLAME: Adaptive Mixture-of-Experts for Continual Multimodal Multi-Task Learning
May 10, 2026Xing Han, Shravan Chaudhari, Tanvi Ranade +2Mixture-of-Experts Models Unseen-Task Adaptation
Measuring What Matters: Benchmarking Generative, Multimodal, and Agentic AI in Healthcare
May 8, 2026Prasanna Desikan, Harshit Rajgarhia, Shivali Dalmia +1Clinical Decision Support Real-World Tasks
Built Environment Reasoning from Remote Sensing Imagery Using Large Vision--Language Models
May 8, 2026Dongdong Wang, Deepak Balakrishnan, Ravi Srinivasan +1Remote Sensing Urban Environments
InterLV-Search: Benchmarking Interleaved Multimodal Agentic Search
May 8, 2026Bohan Hou, Jiuning Gu, Jiayan Guo +5Multimodal Search Agents Multimodal Learning
UniD-Shift: Towards Unified Semantic Segmentation via Interpretable Share-Private Multimodal Decomposition
May 8, 2026Shuai Zhang, Zhecheng Shi, Zhuxiao Li +43D Semantic Segmentation Dense Semantic Segmentation
Structured Role-Aware Policy Optimization for Multimodal Reasoning
May 8, 2026Bingqing Jiang, Difan ZouMultimodal Reasoning Latent Visual Reasoning
MIST: Multimodal Interactive Speech-based Tool-calling Conversational Assistants for Smart Homes
May 7, 2026Maximillian Chen, Xuanming Zhang, Michael Peng +3Voice Agents Smart Homes
MINER: Mining Multimodal Internal Representation for Efficient Retrieval
May 7, 2026Weien Li, Rui Song, Zeyu Li +8Retrieval Layer Multimodal Retrieval
From Review to Design: Ethical Multimodal Driver Monitoring Systems for Risk Mitigation, Incident Response, and Accountability in Automated Vehicles
May 7, 2026Bilal Khana, Waseem Shariff, Rory Coyne +2Autonomous Driving Ethics
Debiased Multimodal Personality Understanding through Dual Causal Intervention
May 7, 2026Yangfu Zhu, Zitong Han, Nianwen Ning +4Personality Causal Intervention
TableVista: Benchmarking Multimodal Table Reasoning under Visual and Structural Complexity
May 7, 2026Zheyuan Yang, Liqiang Shang, Junjie Chen +6Multimodal Learning Vista
ICU-Bench:Benchmarking Continual Unlearning in Multimodal Large Language Models
May 7, 2026Yuhang Wang, Wenjie Mei, Junkai Zhang +3Large Language Model Unlearning Robust Multimodal Learning
Null Space Constrained Contrastive Visual Forgetting for MLLM Unlearning
May 7, 2026Yuhang Wang, Zhenxing Niu, Haoxuan Ji +3Catastrophic Forgetting Multimodal Learning
The Cost of Context: Mitigating Textual Bias in Multimodal Retrieval-Augmented Generation
May 7, 2026Hoin Jung, Xiaoqian WangMulti-Source Retrieval-Augmented Generation Multi-Modal Large Language Models
Causal Probing for Internal Visual Representations in Multimodal Large Language Models
May 7, 2026Zehao Deng, Tianjie Ju, Zheng Wu +5Visual Representations Multimodal Learning
RoboAlign-R1: Distilled Multimodal Reward Alignment for Robot Video World Models
May 5, 2026Hao Wu, Yuqi Li, Yuan Gao +10Video World Models LLM Post-Training Methods
Multimodal Learning on Low-Quality Data with Conformal Predictive Self-Calibration
May 5, 2026Xun Jiang, Yufan Gu, Disen Hu +5Robust Multimodal Learning Conformal Prediction
ReasonAudio: A Benchmark for Evaluating Reasoning Beyond Matching in Text-Audio Retrieval
May 5, 2026Honglei Zhang, Yuting Chen, Chenpeng Hu +2Audio Reasoning Benchmark
A-CODE: Fully Atomic Protein Co-Design with Unified Multimodal Diffusion
May 5, 2026Chaoran Cheng, Jiaqi Guan, Milong Ren +5Protein Design Molecular Generation
Can Multimodal Large Language Models Understand Pathologic Movements? A Pilot Study on Seizure Semiology
May 5, 2026Lina Zhang, Tonmoy Monsoor, Mehmet Efe Lorasdagi +8Interictal Epileptiform Discharges Multi-Modal Large Language Models
Stable Multimodal Graph Unlearning via Feature-Dimension Aware Quantile Selection
May 5, 2026Jingjing Zhou, Yongshuai Yang, Qing Qing +5Federated Graph Learning Graph Foundation Model
VEBench:Benchmarking Large Multimodal Models for Real-World Video Editing
May 5, 2026Andong Deng, Dawei Du, Zhenfang Chen +7Video Editing Multimodal Learning
Sentinel2Cap: A Human-Annotated Benchmark Dataset for Multimodal Remote Sensing Image Captioning
May 4, 2026Lucrezia Tosato, Gianluca Lombardi, Ronny HanschRemote Sensing Image Change Captioning Remote Sensing
Low Resource Multimodal Translation of Nepali Spoken Words into Emotion-Conditioned Sign Language Avatars
May 4, 2026Jatin Bhusal, Salma TamangSign Language Recognition Sign Language
Divide and Conquer: Decoupled Representation Alignment for Multimodal World Models
May 3, 2026Junyuan Xiao, Dingkang Liang, Xin Zhou +9Multimodal Alignment Multimodal Learning
DataArc-SynData-Toolkit: A Unified Closed-Loop Framework for Multi-Path, Multimodal, and Multilingual Data Synthesis
May 2, 2026Zhichao Shi, Cehao Yang, Hao Zhou +6Synthetic Data Multimodal Learning
EO-Gym: A Multimodal, Interactive Environment for Earth Observation Agents
May 2, 2026Sai Ma, Zhuang Li, Sichao Li +4Earth Observation Real-World Environments
Multimodal Confidence Modeling in Audio-Visual Quality Assessment
May 2, 2026Mayesha Maliha R. Mithila, Mylene C. Q. FariasMultimodal Evaluation Cross-Modal Attention
EmoMM: Benchmarking and Steering MLLM for Multimodal Emotion Recognition under Conflict and Missingness
May 1, 2026Yueru Sun, Yimeng Zhang, Haoyu Gu +5Emotion Recognition Modality Gap
WildTableBench: Benchmarking Multimodal Foundation Models on Table Understanding In the Wild
May 1, 2026Junzhe Huang, Xiaoxiao Sun, Yan Yang +6Tabular Data Multimodal Benchmarks

May 17, 2026 · cs.AIJ/K move · Enter open · S save

Multimodal Cultural Heritage Knowledge Graph Extension with Language and Vision Models

Yang Zhang, Nada Mimouni, Jean-Claude Moissinac, Fayçal Hamdi

Télécom Paris, Institut Polytechnique de Paris, France · Center for Studies and Research in Computer Science and Communication, CNAM · Center for Studies and Research in Computer Science and Communication, CNAM, France · LTCI, Télécom Paris, Institut polytechnique de Paris, France

The preservation and interpretation of cultural heritage increasingly rely on digital technologies, among which Knowledge Graphs (KGs) stand out for their ability to structure vast amounts of data. However, the construction and expansion of these KGs often face challenges due to the diverse and complex nature of cultural heritage information. In this paper, we propose a novel approach for extending KG resources in the domain of cultural heritage, which we applied to French data. First, we introduce a new knowledge graph in the domain of French cultural heritage, WJoconde, which is distinguished by its multimodality as it integrates both textual and image information of the entities. We further introduce three variants of WJoconde to facilitate downstream research, such as Knowledge Graph Completion (KGC). We also built a comprehensive benchmark for KGC methods on our dataset. Second, we propose a new framework for extending cultural heritage KGs using multi-modal approaches leveraging Large Language Models (LLMs) and Vision-Language Models (VLMs), which includes automated data extraction from unstructured resources combined with a special validation pipeline for grounding the output of both models, to further extend WJoconde. Our results show that by integrating the rich text and image information in cultural heritage data, we can efficiently enhance KGs with high reliability. We open-source all code and benchmark datasets with text and images, as well as the original data with an interactive access point

Knowledge Graph Embeddings Cultural Heritage Multimodal Learning Luxembourgish Vision Foundation Models

Open paper Ask about it PDF

Recent momentum

Neighbouring directions

Recent digests

18 new papers

Latest in Multimodal Learning