cs.CVMar 18, 2026

WeatherReasonSeg: A Benchmark for Weather-Aware Reasoning Segmentation in Visual Language Models

Authors: Wanjun Du, Zifeng Yuan, Tingting Chen, Fucai Ke, Beibei Lin, Shunli Zhang

Organizations: Beijing Jiaotong University · National University of Singapore · Monash University

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Existing vision-language models (VLMs) have demonstrated impressive performance in reasoning-based segmentation. However, current benchmarks are primarily constructed from high-quality images captured under idealized conditions. This raises a critical question: when visual cues are severely degraded by adverse weather conditions such as rain, snow, or fog, can VLMs sustain reliable reasoning segmentation capabilities? In response to this challenge, we introduce WeatherReasonSeg, a benchmark designed to evaluate VLM performance in reasoning-based segmentation under adverse weather conditions. It consists of two complementary components. First, we construct a controllable reasoning dataset by applying synthetic weather with varying severity levels to existing segmentation datasets, enabling fine-grained robustness analysis. Second, to capture real-world complexity, we curate a real-world adverse-weather reasoning segmentation dataset with semantically consistent queries generated via mask-guided LLM prompting. We further broaden the evaluation scope across five reasoning dimensions, including functionality, application scenarios, structural attributes, interactions, and requirement matching. Extensive experiments across diverse VLMs reveal two key findings: (1) VLM performance degrades monotonically with increasing weather severity, and (2) different weather types induce distinct vulnerability patterns. We hope WeatherReasonSeg will serve as a foundation for advancing robust, weather-aware reasoning.

Figures & tables

Figure 1: Adverse weather undermines VLM reasoning segmentation. Left: Under clean weather conditions, VLMs can correctly localize the target object and generate complete segmentation masks. However, when exposed to adverse weather, the model fails to accurately ground the reasoning query, resulting in incomplete segmentation masks and degraded pixel-level alignment and semantic reasoning performance. Right: WeatherReasonSeg addresses this limitation by introducing real-world degraded data with five structured reasoning dimensions—Function, Application, Structure, Relationship, and Requirement—to enable systematic evaluation of weather-robust reasoning.
Figure 2: Overview of WeatherReasonSeg. The framework consists of two complementary components. Top: A controllable synthetic weather generation process can generate weather type degradations (rain, snow, haze) of varying severity to construct synthetic image-query pairs for robustness evaluation. Bottom: A real-world adverse-weather reasoning dataset constructed via mask-guided large language model prompting, followed by human–model collaborative verification to ensure semantic alignment and reasoning validity.
Figure 3: Overview of WeatherReasonSeg. Left: Query word cloud. Right: Distribution of query–image pairs across weather conditions.
Figure 4: A schematic comparison of VLM reasoning segmentation results under different weather degradation levels. The VLM-based segmentation method used is Seg-Zero. The figure illustrates how segmentation performance progressively changes as weather severity increases..
Figure 5: Radar chart comparison of reasoning segmentation performance across five reasoning dimensions under different weather conditions (Fog, Rain, Snow, and Night). Results are reported using gIoU and cIoU for three representative VLM-based methods: Seg-Zero, LISA, and Seg-R1. The figure shows that visually grounded dimensions such as Structure and Function remain relatively stable, while context-dependent dimensions including Application and Requirement exhibit larger performance degradation under adverse weather conditions.
Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

WeatherLightModerateHeavy
FogA=0.8,β=0.32A{=}0.8,\beta{=}0.32A=0.8,β=1.2A{=}0.8,\beta{=}1.2A=0.8,β=2.0A{=}0.8,\beta{=}2.0
Rains=200,α=0.9s{=}200,\alpha{=}0.9s=700,α=0.85s{=}700,\alpha{=}0.85s=1000,α=0.8s{=}1000,\alpha{=}0.8
Snowρ=0.50,σ=3.0,γ=0.80\rho{=}0.50,\sigma{=}3.0,\gamma{=}0.80ρ=0.85,σ=4.0,γ=0.70\rho{=}0.85,\sigma{=}4.0,\gamma{=}0.70ρ=0.90,σ=4.5,γ=0.55\rho{=}0.90,\sigma{=}4.5,\gamma{=}0.55
Appendix
Table 5: Parameter settings for different adverse weather severities.
Figure 6: Prompt construction for reasoning query generation. The prompt consists of a system instruction defining the reasoning task and a user message containing the class label, RGB image, and corresponding binary mask to guide the generation of five reasoning oriented questions.
Figure 7: The figure outlines the verification rules used to ensure that generated questions satisfy object existence certainty, single target consistency, noun-based answers, reasoning diversity, and image grounded answer ability.
Figure 8: Given the same input image and target object, we construct queries from five semantic perspectives—Function, Application, Structure, Relationship, and Requirement. The reasoning segmentation method adopts Seg-Zero. The right panel illustrates the reasoning process and segmentation predictions of the VLM model, where green check marks denote correct predictions and the red cross indicates a failure case. This example highlights how different reasoning dimensions influence the model’s ability to correctly localize the target object.
Figure 9: Given the same input image and target object, we test the reasoning and segmentation prediction results of the VLM model from five queries, where a green check indicates correct prediction and a red cross indicates prediction failure. This example highlights how different reasoning dimensions affect the model’s ability to correctly locate the target object.
Figure 10: Qualitative results of reasoning segmentation under adverse weather conditions. For each example, we show the query, the original image, and the segmentation predictions produced by Seg-Zero, Seg-R1, and LISA, together with the ground truth (GT). The results highlight the different reasoning segmentation performance of VLM based methods in environments such as rain, snow, and nighttime.

Explore similar work

CardsList

  1. FloodReasonBench: Benchmarking VLM Reasoning Segmentation for Embodied Flood Response at the Edge

    Aug 15, 2026Rajat Bhattacharjya, Yoomee Jung, Minwoo Kim +4Reasoning SegmentationVLM Evaluation

    University of California, Irvine · Kookmin University, South Korea

  2. Bridging the Generalization Gap in Adverse Weather Segmentation: A Training Recipe Perspective

    May 27, 2026Cong Xu, Pu Luo, Yumei Li +1Image Corruption RobustnessDomain Adaptation

    1Xidian University

  3. VLM-RobustBench: A Comprehensive Benchmark for Robustness of Vision-Language Models

    Mar 6, 2026Rohit Saxena, Alessandro Suglia, Pasquale MinerviniVLM RobustnessVLM Evaluation

    University of Edinburgh, UK