Multimodal Foundation Models

Follow this topic Ask about this topic

Momentum

13 papers in the last four weeks, up 117% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 303

All topics

OmniReasoner: Thinking with Long Audio-Video via Native Tool Use
Jul 21, 2026Yu Chen, Caorui Li, Ziyu Xiong +8Temporal Video Grounding Audio-Visual Understanding
University of Chinese Academy of Sciences · Institute of Automation, CAS · Southeast University +3
Fusion Embedding: A Unified Embedding Space for Text, Image, Video, and Audio
Jul 21, 2026Abdul Basit Tonmoy, Kazi Fardinul Hoque, Md. Shahrier Islam Arham +1VLM Adaptation Multimodal Embedding
1Eximius Labs · 2Wabash College · 3Skop Intelligence Co.
Now We Know? A Systematic Comparison of TerraMind and THOR
Jul 20, 2026Frederick Schindlegger, Kenzo Bounegta, Eva Gmelich Meijling +5Geospatial Foundation Models Remote Sensing
University of Münster, Münster, Germany · 2ESA Φ-lab, ESRIN, Frascati, Italy · 3Rubicon, Paris, France +2
RynnBrain 1.1: Towards More Capable and Generalizable Embodied Foundation Model
Jul 20, 2026Kehan Li, Bohan Hou, Minghao Zhu +28Robot Foundation Models Cross-Embodiment Robot Learning
1DAMO Academy, Alibaba Group
TimeLens2: Generalist Video Temporal Grounding with Multimodal LLMs
Jul 19, 2026Yuhan Zhu, Changlian Ma, Xiangyu Zeng +12Temporal Video Grounding Reinforcement Learning
Nanjing University · Shanghai AI Laboratory · Shanghai Jiao Tong University +3
Geometry-Enhanced Portion Estimation for Multimodal LLMs
Jul 17, 2026Lin Liao, Peng LiMultimodal Large Language Models Multimodal Foundation Models
One Modality to Forget Them All: Enhancing Cross-Modal Unlearning in Vision-Language Models
Jul 17, 2026Sudharshan Balaji, Yili Ren, Guangjing Wang +2VLM Unlearning VLM Robustness
University of South Florida · USA · University of Massachusetts Lowell
Audio-Visual Flamingo: Open Audio-Visual Intelligence for Long and Complex Videos
Jul 17, 2026Sreyan Ghosh, Arushi Goel, Kaousheik Jayakumar +19Audio-Visual Understanding Multimodal Large Language Models
1NVIDIA, USA · University of Maryland, USA
Monkey King Bang: A Unified Scientific Multimodal Foundation Model
Jul 17, 2026Hesen Chen, Xinyu Su, Xiaomeng Yang +11AI for Science Unified Multimodal Models
SAIS Team, Shanghai Academy of AI for Science
Ask Twice, Look Twice: Prompt Echoing Resolves the Question-First Paradox in Vision-Language Models
Jul 17, 2026Rakshanda Hassan Abhinandan, John Galeotti, Deva Ramanan +1Visual Question Answering Vision-Language Models
Carnegie Mellon University, USA
AlphaWiSE: Adaptive Weight Interpolation for Continual Multimodal Representation Learning
Jul 16, 2026Sarthak Jain, Qiran Hu, Zhen Zhu +1Cross-Modal Representation Learning Cross-Modal Alignment
University of Illinois Urbana-Champaign · Google DeepMind
Video = World + Event Stream
Jul 16, 2026Lianghua Huang, Zhi-Fan Wu, Yupeng Shi +24Audio-Visual Understanding Streaming Video Understanding
Wan Team, Alibaba Group
Stop Thinking, Start Looking: Efficient Post-Training for Multimodal Document Question Answering via Reasoning-Free Alignment
Jul 16, 2026Harikrishnan P M, Goutham Vignesh, Ganesh Parab +4Multimodal Grounding Efficient Multimodal Inference
Phi Labs, Quantiphi, India · Phi Labs, Quantiphi, Canada.
A Modern Multimodal Assistant on a 6 GB 2011 GPU: Stage-Validated, All-GPU CUDA Inference for Fermi
Jul 16, 2026A. C. Opus, J. Q. LuEfficient Transformer Inference GPU Kernel Optimization
Department of Physics, University of Puerto Rico, Mayaguez, PR 00680, USA
RxBrain: Embodied Cognition Foundation Model with Joint Language-Visual Reasoning and Imagination
Jul 15, 2026Haotian Liang, Mingkang Chen, Yufei Huang +27Unified Multimodal Models Robot Foundation Models
Tencent Robotics X Team · Futian Laboratory · Tencent Hy Team
FM $^2$ : Unified Federated Foundation Models for Heterogeneous Multimodal Medical Imaging
Jul 15, 2026Shengchao Chen, Ting ShuMultimodal Federated Learning Cross-Modal Alignment
School of Artificial Intelligence, Shenzhen University Shenzhen, China · Australian AI Institute, University of Technology Sydney Sydney, Australia
Boogu-Image-0.1: Boosting Open-Source Unified Multimodal Understanding and Generation
Jul 14, 2026Guoxuan Chen, Chufeng Xiao, Haoran Yang +30Text-Guided Image Editing Unified Multimodal Models
Boogu Team
Hy-Embodied-VLM-1.0: Efficient Physical-World Agents
Jul 14, 2026Ziyi Wang, Xumin Yu, Yongming Rao +19Efficient VLA Models Multimodal Foundation Models
Tencent Robotics X Hy Vision Team Futian Laboratory
Xiaomi-Robotics-U0: Unified Embodied Synthesis with World Foundation Model
Jul 13, 2026Xinghang Li, Jun Guo, Qiwei Li +21Robot Foundation Models Cross-Embodiment Robot Learning
Xiaomi Robotics
MonkeyOCRv2: A Visual-Text Foundation Model for Document AI
Jul 13, 2026Yuliang Liu, Zhang Li, Ziyang Zhang +11Document Parsing Multimodal Foundation Models
1Huazhong University of Science and Technology · 2Kingsoft Office
LaGuadia: Language-Guided Adaptive Distillation from Pathology Foundation Models
Jul 13, 2026Gangsu Kim, Won-Ki JeongComputational Pathology Pathology Foundation Models
Department of Computer Science and Engineering, College of Informatics, Korea University, Republic of Korea
Model Guides You How to Draw: Adaptive Visual Gating for Unified Multimodal Reasoning
Jul 10, 2026Wenxi Gao, Guanxi Lu, Didi Zhu +5Efficient Multimodal Inference Unified Multimodal Models
Imperial College London · Tsinghua University
A Foundation Model for Multimodal Event Sequences in Financial Applications
Jul 10, 2026Nikita Rusakov, Vladislav Meshkov, Konstantin Zorin +4Financial Services Multimodal Pretraining
Sber Moscow, Russian Federation · Sber AI Lab Moscow, Russian Federation
ALICE: Learning a General-Purpose Pathology Foundation Model from Vision, Vision-Language, and Slide-Level Experts
Jul 10, 2026Jiawen Li, Tian Guan, Huijuan Shi +5Computational Pathology Unified Multimodal Models
Institute of Biopharmaceutical and Health Engineering, Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, China · Department of Pathology, The First Affiliated Hospital of Sun Yat-sen University, Guangzhou, China · Department of Engineering Science, University of Oxford, Oxford, UK +1
Test-Time Scaling for Small VLMs on Multilingual Visual MCQ
Jul 10, 2026Spiros Baxevanakis, Peng-Jian YangVisual Question Answering Test-Time Scaling for VLMs
University of Amsterdam, Science Park 904, 1098 XH Amsterdam, The Netherlands
Mixture of Probes: Learning from Privileged Modalities in Multimodal LLMs Through Probing
Jul 9, 2026Dominick Reilly, Qiyu Wu, Hiromi Wakaki +2Missing-Modality Learning Cross-Modal Representation Learning
Sony Group Corporation · University of North Carolina at Charlotte · Sony AI
FabriVLA: A Lightweight Vision-Language-Action Model for Precise Multi-Task Manipulation
Jul 9, 2026Shiyuan Yang, Borong Zhang, Jizheng Zhang +5Efficient VLA Models Robotic Manipulation
University of Macau · 2Mese Technology Limited Co., Ltd. · 3FabriX team at Youibot Robotics Co., Ltd.
Switch-Reasoner: Learn When to Think in Multitask Mixtures via Reinforcement Learning
Jul 9, 2026Yiyang Fang, Pei Fu, Jinjie Li +7Efficient Multimodal Inference RL for Language Model Reasoning
Wuhan University · The Hong Kong University of Science and Technology · Xiaomi Inc +2
DeltaV: Thinking with Visual State Updates in Unified Large Multimodal Models
Jul 9, 2026Pengjie Wang, Linger Deng, Zujia Zhang +6Efficient Multimodal Inference Unified Multimodal Models
Huazhong University of Science and Technology · MiLM Plus, Xiaomi Inc.
Multimodal Unlearning Across Vision, Language, Video, and Audio: Survey of Methods, Datasets, and Benchmarks
Jul 8, 2026Nobin Sarwar, Shubhashis Roy Dipta, Zheyuan Liu +1VLM Unlearning MLLM Unlearning
University of Maryland, Baltimore County · University of Notre Dame · UNC Chapel Hill

Jul 21, 2026, cs.CVJ/K move · Enter open · S save

OmniReasoner: Thinking with Long Audio-Video via Native Tool Use

Yu Chen, Caorui Li, Ziyu Xiong, Yidong Wang+7

University of Chinese Academy of Sciences · Institute of Automation, CAS · Southeast University+3

Temporal Video Grounding Audio-Visual Understanding Efficient Multimodal Inference Tool Use in VLMs Multimodal Large Language Models Multimodal Foundation Models

Open paper AskPDF arXiv

Long audio-video reasoning is difficult for omnimodal LLMs because the decisive evidence is often sparse, cross-modal, and too expensive to preserve with uniformly high-fidelity inputs. We introduce OmniReasoner, a tool-use post-training framework for Thinking with Long Audio-Video: omni-modal LLMs learn, via supervised fine-tuning and reinforcement learning, to decide whether and where to call a zoom-in tool before answering. OmniReasoner first builds a low-cost global preview of the full stream and then, when needed, calls the zoom-in tool with a requested temporal interval for higher-fidelity visual and audio inspection before answering. Because the model observes different sampling granularities before and after this call -- a sparse global preview and a denser local clip -- we introduce TimeAnchor, which keeps the tool's temporal argument valid and round-trip-consistent across these granularities, rather than tied to frame indices from a particular sampling rate. To make this tool-use behavior trainable without expensive manual interval annotation, we build a Temporal Augmented Data Engine that synthesizes tool-use post-training trajectories by video editing and composition. Experiments across omnimodal and video benchmarks show that OmniReasoner improves both answer accuracy and temporal grounding while concentrating high-fidelity computation on informative regions. Code is available at https://github.com/RockyChen0205/OmniReasoner.

Momentum

Related topics

Latest papers 303