Multimodal Semantic Segmentation

Latest papers

26 papers2 papers in the last four weeks, with none the four weeks before. 0.0% of all new papers.

FloodVLM-Pix: Language-Guided Pixel-Level Flood Understanding
Oct 16, 2026Chengyang Zhang, Zhihao Wang, Zhiyu Long +4Remote Sensing VLMs Text-Guided Image Segmentation
Tsinghua University · National Disaster Reduction Center
Xiaomi-Lens-0 Technical Report
Oct 16, 2026Pei Fu, Yike Liu, Yue Sun +12Large Vision-Language Models Referring Image Segmentation
Xiaoai Plus, SeerRay Team
FuseAdapt: Parameter-Efficient Multimodal Fusion for Semantic Segmentation with Missing Modalities
Oct 15, 2026Xin Zhang, Robby T. TanMultimodal Semantic Segmentation Multimodal Fusion
National University of Singapore
Multimodal Fusion of RGB and Complementary Modalities for Semantic Segmentation
Oct 13, 2026Hu Cao, Yifeng Cheng, Mu He +3Multimodal Semantic Segmentation Multimodal Fusion
School of Automation, Southeast University, Sipailou 2, Nanjing, 210096, China. · Chair of Robotics, Artificial Intelligence and Real-time Systems, Technical University of Munich, Boltzmannstraße 15, Munich, 85748, Germany.
MultiFly: A Real-World Multimodal Aerial Dataset with Annotation-Efficient Label Transfer and Cross-Modal Semantic Consistency
Oct 7, 2026Markus Gross, Andreas Greiner, Taehyoung Kim +11Multimodal Semantic Segmentation Remote Sensing Benchmarks
Autonomous Aerial Systems, Fraunhofer Institute IVI · Computer Vision Group, Technical University of Munich · Munich Center for Machine Learning (MCML) +2
DTFormer: Text-Guided Semantic Alignment for RGB-D Segmentation
Oct 4, 2026Ziang Wei, Yinlong Liu, Yan Xia +2Semantic Segmentation Multimodal Semantic Segmentation
Chair of Robotics, Artificial Intelligence and Real-time Systems, Technical University of Munich, Germany · City University of Macau, China · University of Science and Technology of China, China +1
Better, Stronger, Faster, and Broader: Structured All-Mask Prediction for MLLM-Based Segmentation
Aug 3, 2026Jiazhen Liu, Mingkuan Feng, Long ChenInstance Segmentation Image Segmentation
The Hong Kong University of Science and Technology (HKUST), Hong Kong.
UMSS: Towards Unsupervised Multi-modal Semantic Segmentation
Jul 14, 2026Haitian Zhang, Thai Duy Nguyen, Xiangyuan Wang +2Cross-Modal Representation Learning Multimodal Semantic Segmentation
EmPACT Lab, School of EEE, Nanyang Technological University, Singapore · The University of Hong Kong, Hong Kong SAR, China
DifferSeg: Towards Diverse Multimodal Binary Segmentation via Differential Perception and Frequency Guidance
Jun 8, 2026Qiangqiang Zhou, Jiawei Xu, Yong Chen +3Image Segmentation Multimodal Semantic Segmentation
School of Artificial Intelligence, Jiangxi Normal University, Nanchang, China · Institute of AI Education, East China Normal University, Shanghai, China · Yale School of Medicine, Yale University, New Haven, CT, USA
Latent Space Guided Scenario Sampling for Multimodal Segmentation Under Missing Modalities
May 19, 2026Irem Ulku, Ö. Özgür Tanrıöver, Erdem AkagündüzMissing-Modality Learning Remote Sensing Image Segmentation
Department of Computer Engineering, Ankara University, Ankara, Türkiye · Department of Modeling and Simulation, Graduate School of Informatics, METU, Ankara, Türkiye
MPerS: Dynamic MLLM MixExperts Perception-Guided Remote Sensing Scene Segmentation
May 11, 2026Ziyi Wang, Xianping Ma, Ziyao Wang +2Remote Sensing Image Understanding Remote Sensing Image Captioning
The Chinese University of Hong Kong (Shenzhen) · Southwest Jiaotong University
UniD-Shift: Towards Unified Semantic Segmentation via Interpretable Share-Private Multimodal Decomposition
May 8, 2026Shuai Zhang, Zhecheng Shi, Zhuxiao Li +4Disentangled Representation Learning Multimodal Semantic Segmentation
1HKUST(GZ) · 2HKUST
Open-Vocabulary Semantic Segmentation Network Integrating Object-Level Label and Scene-Level Semantic Features for Multimodal Remote Sensing Images
Apr 27, 2026Jinkun Dai, Yuanxin Ye, Peng Tang +4Remote Sensing Image Segmentation Text-Guided Image Segmentation
Faculty of Geosciences and Engineering, Southwest Jiaotong University, Chengdu 611756, China · State-Province Joint Engineering Laboratory of Spatial Information Technology for High-Speed Railway Safety, Southwest Jiaotong University, Chengdu 611756, China · School of Artificial Intelligence, Wuhan University, Wuhan 430072, China +1
Robust Multispectral Semantic Segmentation under Missing or Full Modalities via Structured Latent Projection
Apr 17, 2026Irem Ulku, Erdem Akagündüz, Ömer Özgür TanrıöverMissing-Modality Learning Remote Sensing Image Segmentation
Department of Computer Engineering, Ankara University, Ankara, Türkiye · Department of Modeling and Simulation, Graduate School of Informatics, METU, Ankara, Türkiye
RSGMamba: Reliability-Aware Self-Gated State Space Model for Multimodal Semantic Segmentation
Apr 14, 2026Guoan Xu, Yang Xiao, Guangwei Gao +3Multimodal Semantic Segmentation Semantic Segmentation
Faculty of Engineering and Information Technology, University of Technology Sydney, Sydney, NSW 2007, Australia · PCA Lab, Key Laboratory of Intelligent Perception and Systems for High-Dimensional Information of Ministry of Education, School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing 210094, China · Bionic Vision Systems Laboratory, Shanghai Institute of Microsystem and Information Technology, Chinese Academy of Sciences, Shanghai 200050, China +2
EDFNet: Early Fusion of Edge and Depth for Thin-Obstacle Segmentation in UAV Navigation
Apr 6, 2026Negar FathiThin-Structure Segmentation Multimodal Semantic Segmentation
University of Nebraska–Lincoln, Lincoln, NE, USA
Moondream Segmentation: From Words to Masks
Apr 3, 2026Ethan ReidReferring Image Segmentation Image Segmentation
M87 Labs, San Francisco, CA, USA.
SegRGB-X: General RGB-X Semantic Segmentation Model
Mar 30, 2026Jiong Liu, Yingjie Xu, Xingcheng Zhou +4Multimodal Semantic Segmentation Semantic Segmentation
Chair of Robotics, Artificial Intelligence and Real-time Systems, Technical University of Munich, Munich, Germany
Weakly supervised multimodal segmentation of acoustic borehole images with depth-aware cross-attention
Mar 21, 2026Jose Luis Lima de Jesus SilvaMultimodal Semantic Segmentation Image Segmentation
Federal University of Bahia, Institute of Geosciences, Department of Geophysics, Salvador, BA 40170-115, Brazil · Grupo de Estudos e Aplicac¸ ˜ao de Inteligˆencia Artificial em Geof´ısica (GAIA), Federal University of Bahia, Salvador, BA 40170-115, Brazil
MM-OVSeg:Multimodal Optical-SAR Fusion for Open-Vocabulary Segmentation in Remote Sensing
Mar 18, 2026Yimin Wei, Aoran Xiao, Hongruixuan Chen +2Remote Sensing Image Segmentation Open-Vocabulary Semantic Segmentation
The University of Tokyo · RIKEN AIP
DualSwinFusionSeg: Multimodal Martian Landslide Segmentation via Dual Swin Transformer with Multi-Scale Fusion and UNet++
Mar 14, 2026Shahriar Kabir, Abdullah Muhammed Amimul Ehsan, Istiak Ahmmed Rifti +1Multimodal Semantic Segmentation Image Segmentation
Bangladesh University of Engineering and Technology, Bangladesh · University of California, Riverside, USA
Bidirectional Cross-Attention Fusion of High-Resolution RGB and Low-Resolution Hyperspectral Inputs for Multimodal Semantic Segmentation
Mar 14, 2026Jonas V. Funk, Lukas Roming, Andreas Michel +4Multimodal Semantic Segmentation Semantic Segmentation
KIT, Karlsruhe Institute of Technology, 76131 Karlsruhe, Germany · Fraunhofer IOSB, Fraunhofer Institute of Optronics, System Technologies and Image Exploitation, 76131 Karlsruhe, Germany
SGMA: Semantic-Guided Modality-Aware Segmentation for Remote Sensing with Incomplete Multimodal Data
Mar 3, 2026Lekang Wen, Liang Liao, Jing Xiao +1Multimodal Semantic Segmentation Remote Sensing Image Segmentation
State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University · Hangzhou Institute of Technology, Xidian University · School of Artificial Intelligence, Wuhan University
Enhancing 3D LiDAR Segmentation by Shaping Dense and Accurate 2D Semantic Predictions
Feb 21, 2026Xiaoyu Dong, Tiankui Xian, Wanshui Gan +1Multimodal Semantic Segmentation 3D Semantic Segmentation
The University of Tokyo, Tokyo, Japan · RIKEN AIP, Tokyo, Japan
From Pixels to Images: A Structural Survey of Deep Learning Paradigms in Remote Sensing Image Semantic Segmentation
May 21, 2025Quanwei Liu, Tao Huang, Jiaqi Yang +1Remote Sensing Image Segmentation Multimodal Semantic Segmentation
College of Science and Engineering, James Cook University, Cairns QLD 4878, Australia · Center for AI and Data Science Innovation, James Cook University, Cairns QLD 4878, Australia · Department of Forest and Wildlife Ecology, University of Wisconsin-Madison, Madison, WI 53706 USA +1
SARTM: Segment Any RGB Thermal Model with Language aided Distillation
May 4, 2025Dong Xing, Jinhe Zhang, Hang Yang +1Vision Foundation Model Adaptation Semantic Segmentation
Changchun Institute of Optics, Fine Mechanicsand Physics, Chinese Academy of Sciences University

Oct 16, 2026J/K move · Enter open · S save

FloodVLM-Pix: Language-Guided Pixel-Level Flood Understanding

Chengyang Zhang, Zhihao Wang, Zhiyu Long, Xueqian Wang+3

Tsinghua University · National Disaster Reduction Center

Remote Sensing VLMs Text-Guided Image Segmentation Remote Sensing Image Segmentation Multimodal Semantic Segmentation Remote Sensing Benchmarks

Open paper AskPDF arXiv

As climate change intensifies, floods are becoming more frequent and destructive, making accurate flood understanding an urgent priority. However, existing vision-language models perform poorly on flood-related tasks, suffering from attentional bias and inaccurate descriptions of flood magnitude. To bridge this gap, we construct the first large-scale remote sensing flood multi-modal benchmark that supports language-level flood understanding and pixel-level flood segmentation. The benchmark contains 131,200 flood image patches and corresponding 443,100 textual descriptions for flood severity and evolution. Based on this benchmark, we propose FloodVLM-Pix, a language-guided pixel-level flood understanding model that includes a flood prior module and a multi-temporal flood pixel decoder to achieve accurate flood understanding. The flood prior module automatically distills sub-region flood spatial priors and inputs priors into FloodVLM-Pix, solving the attentional bias problem in flood language understanding. The flood pixel decoder introduces a novel general-specific feature fusion mechanism to address performance degradation caused by interference (e.g., clouds or mountain shadows) during flood segmentation. Experiments on FloodVL-Pix demonstrate that FloodVLM-Pix improves both flood description quality and pixel-level flood segmentation performance compared with state-of-the-art VLMs and flood segmentation models. Our code and data will be released at https://github.com/Haomiaoyanmeng/FloodPix.