Multimodal Large Language Models (MLLMs) are increasingly used to carry out visual workflows such as navigating GUIs, where the next step depends on verified visual compositional conditions (e.g., "if a permission dialog appears and the color of the interface is green, click Allow") and the process may branch or terminate early. Yet this capability remains under-evaluated: existing benchmarks focus on shallow-compositions or independent-constraints rather than deeply chained compositional conditionals. In this paper, we introduce MM-CondChain, a benchmark for visually grounded deep compositional reasoning. Each benchmark instance is organized as a multi-layer reasoning chain, where every layer contains a non-trivial compositional condition grounded in visual evidence and built from multiple objects, attributes, or relations. To answer correctly, an MLLM must perceive the image in detail, reason over multiple visual elements at each step, and follow the resulting execution path to the final outcome. To scalably construct such workflow-style data, we propose an agentic synthesis pipeline: a Planner orchestrates layer-by-layer generation of compositional conditions, while a Verifiable Programmatic Intermediate Representation (VPIR) ensures each layer's condition is mechanically verifiable. A Composer then assembles these verified layers into complete instructions. Using this pipeline, we construct benchmarks across three visual domains: natural images, data charts, and GUI trajectories. Experiments on a range of MLLMs show that even the strongest model attains only 53.33 Path F1, with sharp drops on hard negatives and as depth or predicate complexity grows, confirming that deep compositional reasoning remains a fundamental challenge.
Figures & tables
Figure 1: MM-CondChain targets visually grounded deep conditional reasoning beyond prior benchmarks. Top : existing benchmarks typically evaluate either shallow single-layer visual compositions or independent instruction constraints. Bottom left : MM-CondChain introduces nested, inter-layer conditional chains with rich intra-layer compositional predicates, where a minimally perturbed condition can create a hard negative that changes the execution path and causes early termination. Bottom right : experiments show that even advanced MLLMs achieve limited performance on this benchmark, highlighting visually grounded deep compositional reasoning as a fundamental challenge.
Benchmark
Compose
Nested
Visual
Hard Neg.
Prog. Verif.
Determ.
Auto.
Visual Reasoning
SugarCrepe Hsieh et al. (2023)
✓
✗
✓
✓
✗
✓
✓
Winoground Thrush et al. (2022)
✓
✗
✓
✓
✗
✓
✗
ARO Yuksekgonul et al. (2023)
✓
✗
✓
✓
✗
✓
✗
MMComposition Hua et al. (2024)
✓
✗
✓
✗
✗
✓
✗
Instruction Following
Table 1: Comparison with existing benchmarks. Compose : intra-layer multi-attribute composition; Nested : inter-layer chained conditions; Visual : conditions grounded in visual input; Hard Neg. : contrastive pairs with minimal perturbation; Prog. Verif. : ground truth verified via code execution; Determ. : deterministic evaluation without LLM-as-judge; Auto. : automated data construction.
Figure 2: Overview of the MM-CondChain agentic synthesis pipeline. Given a multimodal input, the Planner iteratively extends a conditional chain: at each layer, structured facts are extracted, a VPIR predicate pair is generated and verified via code execution, and the logic is rendered into natural language. The Composer then compiles the verified chain into paired True-path and False-path instances for evaluation.
Aspect
Natural
Chart
GUI
Input
Single image
Image + metadata
Image seq. + annotation
Preprocessing
None
CSV align + LLM repair
Completeness + CoAT parse
Fact Focus
Visual attributes
Numerical stats
Temporal actions
Table 2: Domain-specific adaptations within the unified VPIR framework.
Figure 3: Top attribute frequencies in extracted facts and VPIR variables across domains. (a,c,e) show the top 20 attributes in extracted facts for the Natural, Chart, and GUI domains, respectively; (b,d,f) show the top 20 variables used in VPIR predicates for the corresponding domains.
Figure 4: Logic pattern composition of VPIR expressions. Left: overall distribution of high-level VPIR logic families. Middle: top-20 dominant concrete VPIR templates. Right: an example showing how a VPIR template is instantiated into executable predicates and natural-language conditions.
Natural
Chart
GUI
Avg
Model
True
False
F1
True
False
F1
True
False
F1
F1
Open-Source MLLMs
Qwen3.5-0.8B
33.17
2.26
4.23
31.50
3.00
5.48
33.95
1.86
3.52
4.41
GLM-4.6V-Flash
83.92
9.55
17.14
81.91
5.53
10.36
87.53
0.53
1.05
9.52
InternVL3-8B
65.33
8.29
14.72
47.50
8.50
14.42
63.66
5.31
9.79
12.98
InternVL3.5-8B
82.41
10.30
18.31
76.00
19.50
31.04
82.23
1.33
2.61
17.32
Table 3: Main results on MM-CondChain across domains. All numbers are percentages (%). Path F1 is the harmonic mean of True- and False-path accuracy. Avg(F1) is the mean of the three domain F1 scores. Rows are sorted by Avg(F1) in ascending order within each category.
Model
D=2
D=4
D=6
Δ2→6
Gemini-3-Flash
70.68
53.85
47.19
− 33.2%
Qwen3-VL-Plus
61.51
52.56
43.58
− 29.1%
GPT-4o-1120
31.39
27.67
22.23
− 29.2%
Table 4: Effect of chain depth and predicate complexity on Path F1 (%). Left : Performance degrades as chain depth increases, with ∼ 30% relative drop from D=2 to D=6. Right : Increasing intra-layer predicate complexity ( Simple vs. Complex ) causes 28–36% degradation at fixed depth.
Visually grounded reasoning links visual evidence to multi-step conclusions and actions, aiming to make multimodal systems accurate, auditable, and robust.