Multimodal

Follow this field Ask about this field

Recent momentum

emerging

0 papers in the last 28 days · 0.0% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this field, kept on the site without email delivery.

Period ending 2026-09-21

65 new papers

A weekly snapshot of new work published in Multimodal.

Period ending 2026-09-14

18 new papers

A weekly snapshot of new work published in Multimodal.

Period ending 2026-09-07

26 new papers

A weekly snapshot of new work published in Multimodal.

Inside this field

Focused directions

Focused topic

Multimodal Learning

Within Multimodal

456 papersExplore

Focused topic

Cross-Modal Learning

Within Multimodal

432 papersExplore

Focused topic

Multi-Modal Large Language Models

Within Multimodal

376 papersExplore

Focused topic

Multimodal Reasoning

Within Multimodal

224 papersExplore

Focused topic

Multimodal Benchmarks

Within Multimodal

171 papersExplore

Focused topic

Unified Multimodal Models

Within Multimodal

100 papersExplore

Focused topic

Cross-Modal Attention

Within Multimodal

75 papersExplore

Focused topic

Input Modality

Within Multimodal

73 papersExplore

Focused topic

Modality Gap

Within Multimodal

73 papersExplore

Focused topic

Multimodal Generation

Within Multimodal

73 papersExplore

Focused topic

Cross-Modal Alignment

Within Multimodal

66 papersExplore

Focused topic

Video Multimodal Large Language Models

Within Multimodal

58 papersExplore

1,965 papers

Latest in Multimodal

Browse all topics →

Bolek: A Multimodal Language Model for Molecular Reasoning
May 4, 2026Frederic Grabowski, Jacek Szczerbiński, Maciej Jaśkowski +4Molecular Structure Elucidation Multi-Modal Large Language Models
Triple Spectral Fusion for Sensor-based Human Activity Recognition
May 4, 2026Ye Zhang, Longguang Wang, Qing Gao +3Human Activity Recognition Multi-Sensor Fusion
Low Resource Multimodal Translation of Nepali Spoken Words into Emotion-Conditioned Sign Language Avatars
May 4, 2026Jatin Bhusal, Salma TamangSign Language Recognition Sign Language
Mamoda2.5: Enhancing Unified Multimodal Model with DiT-MoE
May 4, 2026Yangming Shi, Shixiang Zhu, Tao Shen +14Pre-Trained Video Diffusion Models Unified Multimodal Models
ViewSAM: Learning View-aware Cross-modal Semantics for Weakly Supervised Cross-view Referring Multi-Object Tracking
May 4, 2026Jiawei Ge, Xintian Zhang, Jiuxin Cao +9Multi-Object Tracking Photometric Supervision
Enhancing Multimodal In-Context Learning via Inductive-Deductive Reasoning
May 4, 2026Haoyu Wang, Haonan Wang, Yuyan Chen +5Multimodal Reasoning In-Context Learning
SpectraDINO: Bridging the Spectral Gap in Vision Foundation Models via Lightweight Adapters
May 4, 2026Yagiz Nalcakan, Hyeongjin Ju, Incheol Park +3Vision Foundation Models Hyperspectral Imaging
VIDA: A dataset for Visually Dependent Ambiguity in Multimodal Machine Translation
May 3, 2026Jingheng Pan, Xintong Wang, Longyue Wang +3Multimodal Benchmarks Out-Of-Distribution Detection
MER-DG: Modality-Entropy Regularization for Multimodal Domain Generalization
May 3, 2026Yavuz Yarici, Ghassan AlRegibMultimodal Domain Generalization Cross-Modal Learning
Structured Diffusion Bridges: Inductive Bias for Denoising Diffusion Bridges
May 3, 2026Eitan Kosman, Gabriele Serussi, Chaim BaskinMultimodal Diffusion Transformers Cross-Modal Learning
Divide and Conquer: Decoupled Representation Alignment for Multimodal World Models
May 3, 2026Junyuan Xiao, Dingkang Liang, Xin Zhou +9Multimodal Alignment Multimodal Learning
Chart-FR1: Visual Focus-Driven Fine-Grained Reasoning on Dense Charts
May 3, 2026Hongkun Pan, Yuwei Wu, Wanyi Hong +8Chart Multimodal Reasoning
TMD-Bench: A Multi-Level Evaluation Paradigm for Music-Dance Co-Generation
May 3, 2026Xiaoda Yang, Majun Zhang, Changhao Pan +10Music Generation Duet
VT-Bench: A Unified Benchmark for Visual-Tabular Multi-Modal Learning
May 3, 2026Zi-Yi Jia, Zi-Jian Cheng, Xin-Yue Zhang +4Multimodal Benchmarks Tabular Learning
Mitigating Multimodal LLMs Hallucinations via Relevance Propagation at Inference Time
May 3, 2026Itai Allouche, Joseph KeshetLarge Language Model Hallucination Multi-Modal Large Language Models
Omni-Fake: Benchmarking Unified Multimodal Social Media Deepfake Detection
May 2, 2026Tianxiao Li, Zhenglin Huang, Haiquan Wen +10Explainable Deepfake Detection Multimodal Benchmarks
SplAttN: Bridging 2D and 3D with Gaussian Soft Splatting and Attention for Point Cloud Completion
May 2, 2026Zhaoyang Li, Zhichao You, Tianrui LiGaussian Splatting Cross-Modal Learning
Segment-Aligned Policy Optimization for Multi-Modal Reasoning
May 2, 2026Lei Gao, Zhuoming Li, Mengxi Jia +4Multimodal Reasoning
Beyond Perceptual Shortcuts: Causal-Inspired Debiasing Optimization for Generalizable Video Reasoning in Lightweight MLLMs
May 2, 2026Jingze Wu, Quan Zhang, Hongfei Suo +2Video Understanding Visual Reasoning
DataArc-SynData-Toolkit: A Unified Closed-Loop Framework for Multi-Path, Multimodal, and Multilingual Data Synthesis
May 2, 2026Zhichao Shi, Cehao Yang, Hao Zhou +6Synthetic Data Multimodal Learning
EO-Gym: A Multimodal, Interactive Environment for Earth Observation Agents
May 2, 2026Sai Ma, Zhuang Li, Sichao Li +4Earth Observation Real-World Environments
Multimodal Confidence Modeling in Audio-Visual Quality Assessment
May 2, 2026Mayesha Maliha R. Mithila, Mylene C. Q. FariasMultimodal Evaluation Cross-Modal Attention
Multimodal Data Curation Through Ranked Retrieval
May 1, 2026Pratyush Muthukumar, Harshil Kotamreddy, Sarah Amiraslani +4Multimodal Embeddings Cross-Modal Learning
EmoMM: Benchmarking and Steering MLLM for Multimodal Emotion Recognition under Conflict and Missingness
May 1, 2026Yueru Sun, Yimeng Zhang, Haoyu Gu +5Emotion Recognition Modality Gap
WildTableBench: Benchmarking Multimodal Foundation Models on Table Understanding In the Wild
May 1, 2026Junzhe Huang, Xiaoxiao Sun, Yan Yang +6Tabular Data Multimodal Benchmarks
Let ViT Speak: Generative Language-Image Pre-training
May 1, 2026Yan Fang, Mengcheng Lan, Zilong Huang +7Large Vision Language Models Vision Encoders
Physiology-Aware Masked Cross-Modal Reconstruction for Biosignal Representation Learning
May 1, 2026Hao Zhou, Simon A. Lee, Cyrus Tanade +12Multimodal Physiological Signals Pediatric-Adult Electrocardiogram Alignment
MedMosaic: A Challenging Large Scale Benchmark of Diverse Medical Audio
May 1, 2026Harshit Rajgarhia, Shuubham Ojha, Asif Shaik +4Multimodal Clinical Data Diagnostic Benchmark
EASE: Federated Multimodal Unlearning via Entanglement-Aware Anchor Closure
May 1, 2026Zihao Ding, Beining Wu, Jun HuangLarge Language Model Unlearning Cross-Modal Learning
Ablation Study of Multimodal Perception, Language Grounding, and Control for Human-Robot Interaction in an Object Detection and Grasping Task
May 1, 2026Zi Tian, Guanting ShenHuman-Robot Interaction Perception-Action Loop
UniVidX: A Unified Multimodal Framework for Versatile Video Generation via Diffusion Priors
May 1, 2026Houyuan Chen, Hong Li, Xianghao Kong +8Pre-Trained Video Diffusion Models Long Video Generation
Label-Conditioned Cross-Modal Fusion for Adult-to-Pediatric ECG Transfer via Curriculum-Gated Contrastive Alignment
May 1, 2026Xinran Liu, Yuwen Li, Hongxiang Gao +4Pediatric-Adult Electrocardiogram Alignment Intensive Care Unit
Learning Multimodal Energy-Based Model with Multimodal Variational Auto-Encoder via MCMC Revision
May 1, 2026Jiali Cui, Zhiqiang Lao, Heather YuVariational Autoencoder Markov Chain Monte Carlo
CMTA: Leveraging Cross-Modal Temporal Artifacts for Generalizable AI-Generated Video Detection
May 1, 2026Hang Wang, Chao Shen, Chenhao Lin +3High-Fidelity Video Generation Cross-Modal Learning
Scaling Video Understanding via Compact Latent Multi-Agent Collaboration
May 1, 2026Kerui Chen, Jinglu Wang, Jianrong Zhang +3Agentic Video Generation Video Understanding
LIMSSR: LLM-Driven Sequence-to-Score Reasoning under Training-Time Incomplete Multimodal Observations
May 1, 2026Huangbiao Xu, Huanqi Wu, Xiao Ke +1Robust Multimodal Learning Multi-Modal Large Language Models
Multimodal Large Language Models as Synthetic Participants in Video-Based Studies: An Evaluation
May 1, 2026Prabal Shrestha, Bohan Jiang, Haoning Xue +2Video Multimodal Large Language Models User Simulation
Towards Customized Multimodal Role-Play
May 1, 2026Chao Tang, Jianzong Wu, Qingyu Shi +5Multimodal Learning Group
GaMMA: Towards Joint Global-Temporal Music Understanding in Large Multimodal Models
May 1, 2026Zuyao You, Zhesong Yu, Mingyu Liu +3Music Understanding Audio Encoders
Group Cognition Learning: Making Everything Better Through Governed Two-Stage Agents Collaboration
May 1, 2026Chunlei Meng, Pengbin Feng, Rong Fu +7Robust Multimodal Learning Cross-Modal Learning
Are Multimodal LLMs Ready for Clinical Dermatology? A Real-World Evaluation in Dermatology
May 1, 2026Roy Jiang, Hyunjae Kim, Zhenyue Qin +8Multimodal Clinical Data Skin Lesion Classification
REALM: An RGB- and Event-Aligned Latent Manifold for Cross-Modal Perception
Apr 30, 2026Vincenzo Polizzi, David B. Lindell, Jonathan KellyRgbt Cross-Modal Learning
Beyond SFT-to-RL: Pre-alignment via Black-Box On-Policy Distillation for Multimodal RL
Apr 30, 2026Sudong Wang, Weiquan Huang, Xiaomin Yu +9Multimodal Reasoning Supervised Fine-Tuning
MM-StanceDet: Retrieval-Augmented Multi-modal Multi-agent Stance Detection
Apr 30, 2026Weihai Lu, Zhejun Zhao, Yanshu Li +1Tweet-Level Stance Detection Multimodal Agents
One Single Hub Text Breaks CLIP: Identifying Vulnerabilities in Cross-Modal Encoders via Hubness
Apr 30, 2026Hiroyuki Deguchi, Katsuki Chousa, Yusuke SakaiCross-Modal Learning Video Captioning
ClipTBP: Clip-Pair based Temporal Boundary Prediction with Boundary-Aware Learning for Moment Retrieval
Apr 30, 2026Ji-Hyeon Kim, Ho-Joong Kim, Seong-Whan LeeVideo Temporal Grounding Spatio-Temporal Transformers
InteractWeb-Bench: Can Multimodal Agent Escape Blind Execution in Interactive Website Generation?
Apr 30, 2026Qiyao Wang, Haoran Hu, Longze Chen +4Web Mock-Interface Synthesis
COHERENCE: Benchmarking Fine-Grained Image-Text Alignment in Interleaved Multimodal Contexts
Apr 30, 2026Bingli Wang, Huanze Tang, Haijun Lv +5Multi-Modal Large Language Models Video-Text Alignment
Graph-based Semantic Calibration Network for Unaligned UAV RGBT Image Semantic Segmentation and A Large-scale Benchmark
Apr 29, 2026Fangqiang Fan, Zhicheng Zhao, Xiaoliang Ma +2Aerial Imagery Dense Semantic Segmentation
Validating the Clinical Utility of CineECG 3D Reconstructions through Cross-Modal Feature Attribution
Apr 29, 2026Karol Dobiczek, Maciej Mozolewski, Szymon Bobek +3Clinical Cross-Modal Learning
Auto-Relational Reasoning
Apr 29, 2026Ioannis Konstantoulas, Dimosthenis Tsimas, Pavlos Peppas +1Multimodal Reasoning Machine Learning
Attribution-Guided Multimodal Deepfake Detection via Cross-Modal Forensic Fingerprints
Apr 29, 2026Wasim Ahmad, Wei Zhang, Xuerui MaoExplainable Deepfake Detection Deepfake Detection
Benchmarking Complex Multimodal Document Processing Pipelines: A Unified Evaluation Framework for Enterprise AI
Apr 29, 2026Saurabh K. Singh, Sachin RajMultimodal Documents Large Language Model Pipelines
A Multimodal Pre-trained Network for Integrated EEG-Video Seizure Detection
Apr 29, 2026Tong Lu, Ke Xu, Zimo Zhang +12Interictal Epileptiform Discharges Multimodal Learning
Topology-Aware Representation Alignment for Semi-Supervised Vision-Language Learning
Apr 29, 2026Junwon You, Mihyun Jang, Sangwoo Mo +1Multimodal Alignment Large Vision Language Models
TRIP-Evaluate: An Open Multimodal Benchmark for Evaluating Large Models in Transportation
Apr 29, 2026Han Gong, Zhen Zhou, Yunyang Shi +4Multimodal Benchmarks Tour
OmniTrend: Content-Context Modeling for Scalable Social Popularity Prediction
Apr 29, 2026Liliang Ye, Guiyi Zeng, Yunyao Zhang +3Popularity Social Media
Generalized Category Discovery under Domain Shifts: From Vision to Vision-Language Models
Apr 29, 2026Hongjun Wang, Po Hu, Kai HanGeneralized Category Discovery Vision-Language Model Adaptation
ViBE: Visual-to-M/EEG Brain Encoding via Spatio-Temporal VAE and Distribution-Aligned Projection
Apr 29, 2026Ganxi Xu, Zhao-Rong Lai, Yuting Tang +6Vision Encoders Variational Autoencoder
SWAN: World-Aware Adaptive Multimodal Networks for Runtime Variations
Apr 28, 2026Jason Wu, Shir-Kang Scott Jin, Yuyang Yuan +4Multimodal Deep Learning Multimodal Learning

May 4, 2026 · cs.LGJ/K move · Enter open · S save

Bolek: A Multimodal Language Model for Molecular Reasoning

Frederic Grabowski, Jacek Szczerbiński, Maciej Jaśkowski, Kalina Jasińska-Kobus+3

1Ingenix.ai, Warsaw, Poland · Laboratory of Bioinformatics and Computational Genomics, Warsaw University of Technology, Warsaw, Poland

Molecular property models increasingly support high-stakes drug-discovery decisions, but their outputs are often difficult to audit: classical predictors return scores without rationale, while language models can produce fluent explanations weakly grounded in the input molecule. We introduce Bolek, a compact multimodal language model that grounds natural-language reasoning in molecular structure by injecting a Morgan fingerprint embedding into an instruction-tuned text decoder. Bolek is fine-tuned on molecular alignment tasks, including molecule description, RDKit descriptor prediction, and substructure detection, and on downstream reasoning over 15 TDC binary classification tasks using synthetic chains-of-thought anchored in concrete molecular features. Across these tasks, Bolek outperforms its Qwen3-4B-Instruct base on all endpoints in yes/no mode and on 13 of 15 in chain-of-thought mode, raising mean ROC/PR AUC from 0.55 to 0.76. It also outperforms TxGemma-9B-Chat on 13 of 15 binary classification tasks despite being less than half its size. Bolek's explanations are more grounded than those of the baseline LLMs: it cites numerical descriptors 10-100x more often per chain-of-thought, and the cited values agree strongly with RDKit for key descriptors such as TPSA, MolLogP, and MolWt (Spearman rho = 0.87-0.91). Generalisation extends beyond the training panel: on 15 unseen TDC classification endpoints, Bolek matches TxGemma on five, and it produces non-trivial rank correlations on three held-out regression endpoints despite never seeing downstream regression during training. These results suggest that targeted modality injection and reasoning supervision tied to verifiable molecular features can yield compact, auditable molecular reasoning models.

Molecular Structure Elucidation Multi-Modal Large Language Models

Open paper Ask about it PDF