Multimodal Foundation Models

Follow this topic Ask about this topic

Momentum

13 papers in the last four weeks, up 117% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 303

All topics

Fashion Florence: Fine-Tuning Florence-2 for Structured Fashion Attribute Extraction
May 11, 2026Anushree BerliaVLM Adaptation Multimodal Foundation Models
DeltaRubric: Generative Multimodal Reward Modeling via Joint Planning and Verification
May 10, 2026Rui Liu, Dian Yu, Zhenwen Liang +6Multimodal Large Language Models Multimodal Reward Modeling
1Tencent Hunyuan · University of Maryland, College Park · University of North Carolina, Chapel Hill
Reinforcing Multimodal Reasoning Against Visual Degradation
May 10, 2026Rui Liu, Dian Yu, Haolin Liu +6VLM Robustness Image Corruption Robustness
Tencent Hunyuan · University of Maryland, College Park · University of Virginia +1
Meow-Omni 1: A Multimodal Large Language Model for Feline Ethology
May 9, 2026Jucheng Hu, Zhangquan Chen, Yulin Chen +9Multimodal Large Language Models Multimodal Foundation Models
University College London · Tsinghua University · Nanjing University +1
MultiMedVision: Multi-Modal Medical Vision Framework
May 9, 2026Frank Li, Bardia Khosravi, Mohammadreza Chavoshi +5Medical Imaging Foundation Models Vision Transformer
Emory University · Yale University
How Many Visual Tokens Do Multimodal Language Models Need? Scaling Visual Token Pruning with F^3A
May 9, 2026YiJie Huang, Yiqun Zhang, Zhuoyue Jia +7Efficient VLM Inference Large Vision-Language Models
School of Computer Science and Engineering, Northeastern University, Shenyang 110819, China · Shanghai Artificial Intelligence Laboratory · School of Computer and Communication Engineering, Northeastern University, Qinhuangdao 066004, China
Earth Science Foundation Models: From Perception to Reasoning and Discovery
May 9, 2026Xiangyu Zhao, Bo Liu, Yuehan Zhang +9AI for Science Geospatial Foundation Models
Shanghai Artificial Intelligence Laboratory, Shanghai, China
ZAYA1-VL-8B Technical Report
May 8, 2026Hassan Shapourian, Kasra Hejazi, Olabode M. Sule +1Vision-Language Models Mixture-of-Experts Language Models
Zyphra Technologies Inc, San Francisco, CA
STARFlow2: Bridging Language Models and Normalizing Flows for Unified Multimodal Generation
May 8, 2026Ying Shen, Tianrong Chen, Yuan Gao +6Unified Multimodal Models Multimodal Generation
UIUC · Apple
Pan-FM: A Pan-Organ Foundation Model with Saliency-Guided Masking for Missing Robustness
May 8, 2026Qiangqiang Wu, Grace McIlvain, Zhou Yu +1Missing-Modality Learning Medical Imaging
Laboratory of AI and Biomedical Science (LABS) · 2Columbia University
Steering Visual Generation in Unified Multimodal Models with Understanding Supervision
May 7, 2026Zeyu Liu, Zanlin Ni, Yang Yue +5Unified Multimodal Models Image Generation
1Tsinghua University · 2Kolors Team, Kuaishou Technology
A foundation model of vision, audition, and language for in-silico neuroscience
May 5, 2026Stéphane d'Ascoli, Jérémy Rapin, Yohann Benchetrit +5Cognitive Modeling fMRI Foundation Models
FAIR at Meta · Laboratoire de Neurosciences Cognitives et Computationnelles, Ecole Normale Supérieure - PSL
Audio-Visual Intelligence in Large Foundation Models
May 5, 2026You Qin, Kai Liu, Shengqiong Wu +12Audio-Visual Understanding Audio-Video Generation
National University of Singapore · University of Oxford · University of Toronto +5
UnAC: Adaptive Visual Prompting with Abstraction and Stepwise Checking for Complex Multimodal Reasoning
May 5, 2026Yifan Wang, Yun FuMultimodal Large Language Models Multimodal Reasoning
Northeastern University
MiniMind-O Technical Report: An Open Small-Scale Speech-Native Omni Model
May 5, 2026Jingyao GongSpeech Foundation Models Unified Multimodal Models
Independent Researcher
JoyAI-Image: Awaking Spatial Intelligence in Unified Multimodal Understanding and Generation
May 5, 2026Lin Song, Wenbo Li, Guoqing Ma +16Unified Multimodal Models Multimodal Foundation Models
Joy Future Academy, JD
MolmoAct2: Action Reasoning Models for Real-world Deployment
May 4, 2026Haoquan Fang, Jiafei Duan, Donovan Clay +26Robotics Efficient VLA Models
1Allen Institute for AI · University of Washington · 3National University of Singapore +6
Visual Latents Know More Than They Say: Unsilencing Latent Reasoning in MLLMs
May 4, 2026Xin Zhang, Qiqi Tao, Jiawei Du +2Inference-Time Optimization Test-Time Optimization
Centre for Frontier AI Research, Agency for Science, Technology and Research, Singapore · Institute of High Performance Computing, Agency for Science, Technology and Research, Singapore · Singapore University of Technology and Design +1
Mamoda2.5: Enhancing Unified Multimodal Model with DiT-MoE
May 4, 2026Yangming Shi, Shixiang Zhu, Tao Shen +14Diffusion Transformer Unified Multimodal Models
Mamoda Team, ByteDance
Metric Unreliability in Multimodal Machine Unlearning: A Systematic Analysis and Principled Unified Score
May 4, 2026Abdullah Ahmad Khan, Hamid Laga, Ferdous SohelVLM Evaluation VLM Unlearning
Murdoch University
Divide and Conquer: Decoupled Representation Alignment for Multimodal World Models
May 3, 2026Junyuan Xiao, Dingkang Liang, Xin Zhou +9World Models Cross-Modal Alignment
Tsinghua University, China · Huazhong University of Science and Technology, China · CSU, China +4
Referring Multiple Regions with Large Multimodal Models via Contextual Latent Steering
May 3, 2026Yun Xing, Hanyuan Liu, Jiahao Nie +1Multimodal Large Language Models Vision-Language Grounding
Nanyang Technological University
OmniEncoder: See, Hear, and Feel Continuous Motion Like Humans With One Encoder
May 2, 2026Detao Bai, Shimin Yao, Weixuan Chen +4Cross-Modal Learning Video Understanding
1Tongyi Lab Alibaba Group · Shenzhen University of Advanced Technology
Valley3: Scaling Omni Foundation Models for E-commerce
May 2, 2026Zeyu Chen, Guanghao Zhou, Min Yang +6Unified Multimodal Models Multimodal Pretraining
Valley Team, ByteDance Group
WildTableBench: Benchmarking Multimodal Foundation Models on Table Understanding In the Wild
May 1, 2026Junzhe Huang, Xiaoxiao Sun, Yan Yang +6Visual Question Answering Table QA
The University of Queensland · Stanford University · The Australian National University +3
Let ViT Speak: Generative Language-Image Pre-training
May 1, 2026Yan Fang, Mengcheng Lan, Zilong Huang +7Cross-Modal Learning Vision Transformer
1Beijing Jiaotong University · 2ByteDance · 3Nanyang Technological University
REALM: An RGB- and Event-Aligned Latent Manifold for Cross-Modal Perception
Apr 30, 2026Vincenzo Polizzi, David B. Lindell, Jonathan KellyCross-Modal Learning Event-Based Vision
University of Toronto, Robotics Institute · University of Toronto, Department of Computer Science
Beyond SFT-to-RL: Pre-alignment via Black-Box On-Policy Distillation for Multimodal RL
Apr 30, 2026Sudong Wang, Weiquan Huang, Xiaomin Yu +9Multimodal Large Language Models Multimodal Reasoning
Hong Kong University of Science and Technology (Guangzhou) · Nanyang Technological University · Tsinghua University +3
A Pattern Language for Resilient Visual Agents
Apr 30, 2026Habtom Kahsay Gidey, Alexander Lenz, Alois KnollVision-Language-Action Models Multimodal Foundation Models
Technische Universität München, München, Germany
MiniCPM-o 4.5: Towards Real-Time Full-Duplex Omni-Modal Interaction
Apr 30, 2026Junbo Cui, Bokai Xu, Chongyi Wang +33Efficient Multimodal Inference Unified Multimodal Models
MiniCPM-o Team, OpenBMB

May 11, 2026, cs.CVJ/K move · Enter open · S save

Fashion Florence: Fine-Tuning Florence-2 for Structured Fashion Attribute Extraction

Anushree Berlia

VLM Adaptation Multimodal Foundation Models

Open paper AskPDF arXiv

We present Fashion Florence, a Florence-2 vision-language model fine-tuned with LoRA to extract structured fashion attributes from clothing images. Given a single photograph, the model generates a JSON object containing category, color, material, style tags, and occasion tags, structured output suitable for direct programmatic consumption by downstream recommendation and retrieval systems. Fine-tuning data is derived from the iMaterialist Fashion dataset (228 labels), where we collapse fine-grained annotations into a compact 6-category, 16-color, 19-style schema via rule-based label engineering. We apply LoRA (r=16, alpha=32) to all decoder linear layers, training for 3 epochs on 3,688 examples. On a held-out test set of 461 images, Fashion Florence achieves 94.6% category accuracy and 63.0% material accuracy, compared to 89.3% / 43.3% for GPT-4o-mini and 87.4% for Gemini 2.5 Flash. Fashion Florence produces valid JSON in 99.8% of outputs while running at 0.77B parameters on a single GPU at zero marginal inference cost. Style tag F1 reaches 0.753 vs. 0.612 (Gemini) and 0.398 (GPT-4o-mini). The model is deployed as a Hugging Face Space and integrated into Loom, an open-source outfit recommendation system.

Momentum

Related topics

Latest papers 303