Grounding Image Composition with Label-Conditioned Visual Evidence
Organizations: Computer Vision Lab, TU Wien, Vienna, Austria · UAS St. Pölten, St. Pölten, Austria
Abstract
Image composition labels such as center, diagonal, or rule of thirds describe visual structure rather than object categories. KU-PCP provides image-level labels and class-specific geometric elements, but does not show which visible regions people associate with each of its nine classes. We introduce KU-PCP Elements (KU-PCPE), a label-conditioned region-evidence benchmark for photographic composition. Given an image and a label already judged applicable, the task is to segment the visible regions that explain why the label applies. KU-PCPE contains 4,093 single-annotator training masks and a 1,327-task expert test split with three masks per image-label task. The annotations also include exploratory segment-versus-line ratings that describe whether region or semantic-guideline evidence better explains the label. A held-out expert reaches .867 Dice against the other experts, while the strongest deployable model reaches .664, a proposal oracle .672, and the best vision-language model (VLM) proposal selector .395. KU-PCPE exposes the gap between recognizing a composition label and grounding the human-aligned visual evidence that makes it plausible in a particular image.
Figures & tables
| Class | Train tasks | Expert-test tasks |
|---|---|---|
| Horizontal | 758 | 211 |
| Vertical | 195 | 66 |
| Diagonal | 471 | 189 |
| Curved | 235 | 92 |
| Triangle | 374 | 171 |
| Rule of thirds | 923 | 125 |
| Interface statistic | Train | Expert test |
|---|---|---|
| Annotations with records | 4,093 | 3,981 |
| No geometry action | 51.8% | 62.2% |
| Any geometry action | 48.2% | 37.8% |
| Manual tool use | 15.1% | 17.5% |
| Split operation | 5.7% | 6.1% |
| Median recorded actions | 15 | 8 |
| Class | Tasks | Pair D | Maj. D | Flip | |
|---|---|---|---|---|---|
| Horizontal | 211 | .6371 | .9539 | .8281 | .5118 |
| Vertical | 66 | .6557 | .8362 | .5782 | .1515 |
| Diagonal | 189 | .5973 | .8805 | .5433 | .4656 |
| Curved | 92 | .6105 | .8556 | .5240 | .3696 |
| Triangle | 171 | .8332 | .9351 | .7796 | .1170 |
| Rule of thirds | 125 | .9538 | .9782 | .9426 | .0080 |
| Method | Strict Dice | Ambig. Dice |
|---|---|---|
| Train class template | .451 | .462 |
| SAM2.1 Oracle@3 | .679 | .672 |
| SegFormer-B0 | .662 | .664 |
| Qwen3-VL-32B | .399 | .395 |
| Triangle comparison | Strict | Ambig. |
|---|---|---|
| Sides inside target | .374 | .398 |
| Sides inside target, 5-px tolerance | .465 | .489 |
| Enclosed region Dice | .572 | .585 |
| Enclosed region IoU | .451 | .462 |
| Annotation | Pair D | O@3 | T@3 | SegF. | Qwen | |
|---|---|---|---|---|---|---|
| No edit logged | 615 | .858 | .785 | .481 | .766 | .472 |
| Edit logged | 712 | .643 | .574 | .358 | .575 | .328 |
| Method | Dice | IoU | B-F1 | Over. | Hit |
|---|---|---|---|---|---|
| Human reference | .867 | .805 | .754 | .113 | .931 |
| Random matched area | .238 | .139 | .037 | .717 | .007 |
| Train class template | .462 | .325 | .169 | .481 | .494 |
| SAM2.1 Oracle@3 | .672 | .605 | .632 | .067 | .661 |
| SAM2.1 Template@3 | .415 | .334 | .447 | .510 | .399 |
| U-Net, no label | .496 | .379 | .307 | .458 | .534 |
| Same 12 candidates | Dice | IoU | B-F1 |
|---|---|---|---|
| Template@3 | .402 | .320 | .441 |
| Qwen3-VL-32B | .395 | .324 | .440 |
| Greedy Oracle@3 | .640 | .574 | .596 |
| Exact, exactly 3 | .640 | .575 | .596 |
| Exact, up to 3 | .645 | .582 | .610 |
| Mask | Blur | Fill | Blur flip | Fill flip | Dice |
|---|---|---|---|---|---|
| Human | 1.02 | 2.03 | .10 | .21 | 1.00 |
| Random area | .16 | .38 | .02 | .03 | .28 |
| Full image | 2.31 | 4.08 | .26 | .76 | .41 |
| U-Net + label | 1.16 | 2.80 | .12 | .28 | .60 |
| SAM O@3 | .55 | 1.31 | .06 | .14 | .67 |
| SAM T@3 | .71 | 1.75 | .07 | .19 | .41 |
Appendix figures & tables41 assets
Supplementary material from the paper’s appendix.
Appendix
| Class | Definition | What to look for |
|---|---|---|
| Rule-of-Thirds (RoT) | Main subject placed along a rule-of-thirds location, especially near a vertical trisecting line rather than dead center. | Off-center subject placement; alignment near thirds-grid intersections or vertical thirds lines; balance created by asymmetrical composition. |
| Vertical | Strong repeated or dominant vertical elements emphasizing height or upright structure. | Dominant upright lines or repeated vertical rhythm; height-emphasizing composition; vertical structure outweighs horizontal flow. |
| Horizontal | Strong horizontal structure or leading line, often horizon-like or calm lateral division. | Dominant left-to-right directional structure; horizon-like or layered horizontal bands; lateral organization more important than vertical cues. |
| Diagonal | Dominant slanted line or directional structure that creates diagonal flow. | Strong angled composition direction; motion or tension created by slanted alignment; diagonal path that guides gaze. |
| Curved | Dominant curved flow or path guiding the eye. | Arc-like or S-shaped compositional path; curve-driven eye guidance across the frame; smooth curvature dominates structural reading. |
| Triangle | Triangular or implied triangular arrangement, including vanishing-line based triangles. | Three-point or wedge-like structural arrangement; implied triangular balance in key regions; triangle formed by lines, edges, or grouped elements. |
| Class | How the mask can express the composition | Common ambiguity |
|---|---|---|
| Center | Centrally placed subject, part, or supporting central region. | Main subject alone versus a larger central structure. |
| Rule of thirds | Off-center subject or region whose placement creates the thirds relation. | Subject boundary versus surrounding context; left versus right placement. |
| Horizontal | Regions or elements forming a lateral division or arrangement. | Upper versus lower side of a complementary partition. |
| Vertical | Dominant or repeated upright supports. | Complete structures versus their most salient vertical parts. |
| Diagonal | One elongated carrier or several aligned elements creating slanted flow. | One side of a diagonal partition versus the other. |
| Symmetric | Corresponding reflected regions and, where relevant, their central support. | One side, both sides, or the region joining them. |
| Class | Representational issue illustrated by the ambiguous row |
|---|---|
| Horizontal | Complementary sides of a lateral partition and an intersecting subject. |
| Vertical | Central upright passage versus surrounding upright walls. |
| Diagonal | Opposite image regions separated by the same slanted boundary. |
| Curved | Water, land, or sky regions that jointly make the curved path visible. |
| Triangle | Triangle-forming architectural regions versus the broader enclosed shape. |
| Rule of thirds | Main subject alone versus subject with nearby supporting context. |
| Metric | Mean | 95% task-bootstrap CI |
|---|---|---|
| Dice | .8666 | [.8577, .8750] |
| IoU | .8054 | [.7939, .8167] |
| Boundary F1 | .7541 | [.7411, .7669] |
| Overcoverage | .1131 | [.1058, .1203] |
| [email protected] | .9312 | [.9209, .9410] |
| Class | Area | Pair D | Pair I | B-F1 | Maj. D | Flip | ||
|---|---|---|---|---|---|---|---|---|
| Horizontal | 211 | .3882 | .6371 | .6241 | .7320 | .9539 | .8281 | .5118 |
| Vertical | 66 | .2426 | .6557 | .5599 | .6619 | .8362 | .5782 | .1515 |
| Diagonal | 189 | .3741 | .5973 | .5566 | .6311 | .8805 | .5433 | .4656 |
| Curved | 92 | .2961 | .6105 | .5593 | .6103 | .8556 | .5240 | .3696 |
| Triangle | 171 | .3126 | .8332 | .7860 | .7611 | .9351 | .7796 | .1170 |
| Rule of thirds | 125 | .0939 | .9538 | .9326 | .9473 | .9782 | .9426 | .0080 |
| Method | Dice | IoU | B-F1 | Overcoverage | [email protected] |
|---|---|---|---|---|---|
| Random matched area | .2250 [.2191, .2310] | .1312 [.1273, .1350] | .0357 [.0319, .0397] | .7463 [.7371, .7556] | .0015 [.0000, .0038] |
| Train class template | .4512 [.4387, .4635] | .3186 [.3085, .3287] | .1621 [.1555, .1689] | .5104 [.4943, .5268] | .4891 [.4627, .5170] |
| SAM2.1 Oracle@3 | .6786 [.6591, .6979] | .6159 [.5951, .6368] | .6486 [.6298, .6681] | .0838 [.0724, .0956] | .6714 [.6458, .6963] |
| SAM2.1 Template@3 | .4131 [.3941, .4318] | .3341 [.3160, .3521] | .4429 [.4268, .4593] | .5319 [.5112, .5527] | .3956 [.3693, .4213] |
| U-Net, no label | .4821 [.4662, .4980] | .3691 [.3545, .3835] | .3029 [.2921, .3138] | .4962 [.4787, .5140] | .5200 [.4928, .5463] |
| U-Net + label | .5870 [.5715, .6024] | .4725 [.4574, .4877] | .3864 [.3742, .3986] | .4191 [.4021, .4361] | .6511 [.6255, .6767] |
| Method | Dice | IoU | B-F1 | Overcoverage | [email protected] |
|---|---|---|---|---|---|
| Random matched area | .2375 [.2315, .2436] | .1394 [.1354, .1433] | .0370 [.0332, .0409] | .7173 [.7068, .7275] | .0068 [.0030, .0113] |
| Train class template | .4617 [.4502, .4734] | .3254 [.3157, .3351] | .1691 [.1624, .1760] | .4815 [.4655, .4975] | .4936 [.4665, .5200] |
| SAM2.1 Oracle@3 | .6718 [.6533, .6903] | .6054 [.5846, .6263] | .6317 [.6124, .6505] | .0674 [.0575, .0776] | .6609 [.6353, .6865] |
| SAM2.1 Template@3 | .4147 [.3963, .4337] | .3337 [.3160, .3517] | .4475 [.4317, .4629] | .5104 [.4895, .5310] | .3986 [.3730, .4258] |
| U-Net, no label | .4958 [.4803, .5115] | .3790 [.3650, .3930] | .3071 [.2964, .3183] | .4585 [.4410, .4757] | .5335 [.5079, .5607] |
| U-Net + label | .5970 [.5820, .6120] | .4791 [.4645, .4939] | .3904 [.3788, .4025] | .3903 [.3739, .4070] | .6609 [.6353, .6850] |
| Source | Area fraction | Components | Boundary complexity | Modality score |
|---|---|---|---|---|
| Train annotator | .2436 [.2381, .2490] | 10.27 [9.55, 11.05] | 9.71 [9.50, 9.91] | .1955 [.1880, .2031] |
| Expert 1 | .2462 [.2368, .2555] | 7.96 [6.98, 9.05] | 8.69 [8.40, 9.00] | .3373 [.3234, .3512] |
| Expert 2 | .2520 [.2426, .2615] | 8.61 [7.26, 10.38] | 8.38 [8.12, 8.65] | .2499 [.2355, .2646] |
| Expert 3 | .2863 [.2767, .2960] | 7.92 [7.09, 8.82] | 9.15 [8.85, 9.47] | .4949 [.4793, .5104] |
| Strict consensus | .2537 [.2444, .2632] | 9.05 [8.12, 10.05] | 8.94 [8.65, 9.23] | .3607 [.3480, .3733] |
| Ambiguity-aware | .2827 [.2722, .2928] | 8.77 [7.83, 9.81] | 8.82 [8.54, 9.12] | .3607 [.3477, .3736] |
| Method | Expert 1 | Expert 2 | Expert 3 | Strict | Ambig. |
|---|---|---|---|---|---|
| Random matched area | .2199 | .2220 | .2419 | .2250 | .2375 |
| Train class template | .4433 | .4471 | .4444 | .4512 | .4617 |
| SAM2.1 Oracle@3 | .6742 | .6948 | .6688 | .6786 | .6718 |
| SAM2.1 Template@3 | .3957 | .4242 | .4006 | .4131 | .4147 |
| U-Net, no label | .4658 | .4545 | .4951 | .4821 | .4958 |
| U-Net + label | .5649 | .5770 | .5867 | .5870 | .5970 |
| Property | KU-PCP | KU-PCPE |
|---|---|---|
| Primary supervision | Image-level composition labels | Image-label region evidence masks |
| Localized output | Class-specific boxes, lines or axes, triangles, and regions produced by automatic methods | Binary regions selected by people |
| Representation across classes | Different geometric forms for different classes | One mask representation for all nine classes |
| Test interpretations | One output for each supported detector | Three expert masks and two consensus views |
| Intended use | Composition classification and element detection | Conditional grounding of human-selected evidence |
| Target | Line in target | +5 px | Target covered | Line D | Filled D | Filled I | |
|---|---|---|---|---|---|---|---|
| Ambiguity-aware | 171 | .398 [.354, .442] | .489 [.441, .536] | .036 | .064 | .585 [.544, .624] | .462 [.422, .501] |
| Strict | 171 | .374 [.332, .415] | .465 [.419, .511] | .036 | .065 | .572 [.530, .612] | .451 [.411, .490] |
| Annotation | Raw pair D | Comp. pair D | B-F1 | Pair | |
|---|---|---|---|---|---|
| No edit logged | 615 | .8579 [.8384, .8773] | .9132 [.9004, .9257] | .8858 [.8700, .9012] | .8072 [.7792, .8345] |
| Edit logged | 712 | .6434 [.6218, .6652] | .7467 [.7295, .7634] | .5822 [.5631, .6011] | .5292 [.4993, .5589] |
| Method | Dice | IoU | B-F1 | Overcoverage | [email protected] |
|---|---|---|---|---|---|
| Random matched area | .215 [.207, .224] | .125 [.120, .131] | .032 [.026, .037] | .754 [.739, .767] | .008 [.002, .016] |
| Train class template | .485 [.468, .501] | .346 [.331, .360] | .167 [.157, .178] | .478 [.456, .500] | .535 [.496, .574] |
| SAM2.1 Oracle@3 | .785 [.759, .810] | .739 [.710, .767] | .769 [.743, .793] | .025 [.017, .034] | .769 [.735, .802] |
| SAM2.1 Template@3 | .481 [.453, .509] | .397 [.369, .426] | .535 [.513, .557] | .482 [.452, .512] | .465 [.426, .504] |
| U-Net, no label | .558 [.535, .581] | .444 [.422, .466] | .361 [.343, .379] | .431 [.405, .456] | .620 [.582, .659] |
| U-Net + label | .671 [.652, .692] | .556 [.535, .576] | .457 [.439, .475] | .330 [.308, .353] | .767 [.733, .800] |
| Method | Dice | IoU | B-F1 | Overcoverage | [email protected] |
|---|---|---|---|---|---|
| Random matched area | .257 [.249, .265] | .152 [.146, .157] | .042 [.036, .047] | .686 [.671, .701] | .006 [.001, .011] |
| Train class template | .442 [.426, .457] | .308 [.294, .321] | .171 [.162, .180] | .484 [.461, .507] | .458 [.420, .494] |
| SAM2.1 Oracle@3 | .574 [.548, .599] | .490 [.464, .516] | .513 [.489, .539] | .104 [.088, .121] | .567 [.529, .604] |
| SAM2.1 Template@3 | .358 [.333, .382] | .279 [.256, .301] | .372 [.352, .393] | .535 [.506, .564] | .341 [.306, .375] |
| U-Net, no label | .442 [.423, .461] | .323 [.305, .339] | .261 [.249, .273] | .482 [.459, .505] | .459 [.423, .496] |
| U-Net + label | .533 [.512, .552] | .413 [.393, .432] | .333 [.319, .348] | .442 [.419, .465] | .569 [.532, .605] |
| Annotation | Class | Pair D | O@3 | T@3 | SegF. | Qwen | |
|---|---|---|---|---|---|---|---|
| No edit logged | Center | 203 | .912 | .776 | .548 | .828 | .509 |
| Curved | 24 | .822 | .593 | .330 | .552 | .413 | |
| Diagonal | 61 | .760 | .780 | .477 | .534 | .512 | |
| Horizontal | 94 | .684 | .917 | .580 | .705 | .613 | |
| Pattern | 26 | .613 | .319 | .245 | .593 | .215 | |
| Rule of thirds | 103 | .977 | .915 | .368 | .896 | .393 |
| Rater | Annotation | Pair D | O@3 | T@3 | SegF. | Qwen | |
|---|---|---|---|---|---|---|---|
| rater_01 | No edit logged | 859 | .793 [.774, .813] | .753 [.731, .775] | .467 [.443, .491] | .709 [.688, .729] | .453 [.428, .478] |
| rater_01 | Edit logged | 468 | .650 [.624, .676] | .523 [.494, .552] | .319 [.291, .347] | .580 [.550, .610] | .287 [.259, .316] |
| rater_03 | No edit logged | 770 | .817 [.797, .836] | .767 [.744, .791] | .482 [.456, .507] | .718 [.696, .740] | .462 [.435, .489] |
| rater_03 | Edit logged | 557 | .640 [.615, .664] | .540 [.512, .568] | .322 [.296, .349] | .588 [.560, .615] | .302 [.275, .329] |
| rater_04 | No edit logged | 846 | .807 [.788, .825] | .773 [.751, .795] | .484 [.459, .507] | .718 [.697, .739] | .469 [.444, .494] |
| rater_04 | Edit logged | 481 | .631 [.603, .657] | .493 [.465, .521] | .294 [.267, .320] | .568 [.538, .598] | .264 [.238, .291] |
| Rater | Split | Annotations | No edit | Edit | Manual | Split | Dynamic | Median actions |
|---|---|---|---|---|---|---|---|---|
| rater_02 | Train | 4,093 | 51.8% | 48.2% | 15.1% | 5.7% | 31.9% | 15 |
| rater_01 | Test | 1,327 | 64.7% | 35.3% | 16.5% | 7.5% | 27.4% | 12 |
| rater_03 | Test | 1,327 | 58.0% | 42.0% | 20.0% | 6.2% | 30.7% | 8 |
| rater_04 | Test | 1,327 | 63.8% | 36.2% | 16.1% | 4.7% | 28.3% | 6 |
| Diagnostic | Value |
|---|---|
| Multi-label images | 244 |
| Different-label task pairs | 264 |
| Expert same-image, different-label Dice | .3655 |
| Expert same-image, different-label IoU | .3247 |
| Train-template same-image, different-label Dice | .3095 |
| Expert/template distance correlation | .4508 |
| Label pair | Pairs | Expert D | Expert I | Template D |
|---|---|---|---|---|
| Center / Triangle | 54 | .8106 | .7674 | .5308 |
| Horizontal / Rule of thirds | 40 | .0218 | .0123 | .0000 |
| Diagonal / Horizontal | 37 | .1540 | .1224 | .3202 |
| Curved / Horizontal | 22 | .2301 | .1739 | .1401 |
| Curved / Diagonal | 18 | .3442 | .2846 | .6668 |
| Horizontal / Triangle | 12 | .1591 | .1214 | .2282 |
| Proposal method | Dice | IoU | B-F1 | Over. | Hit |
|---|---|---|---|---|---|
| Random@3 | .0895 | .0684 | .1224 | .6449 | .0622 |
| Largest | .2491 | .2294 | .3372 | .6950 | .2419 |
| Area@1 | .3571 | .3287 | .3946 | .5352 | .3519 |
| Area@3 | .4133 | .3259 | .4250 | .5676 | .4190 |
| Template@1 | .4099 | .3724 | .4325 | .3901 | .3873 |
| Template@3 | .4147 | .3337 | .4475 | .5104 | .3986 |
| Method | Label input | Dice | IoU | B-F1 | Over. | Hit |
|---|---|---|---|---|---|---|
| U-Net | No | .4958 | .3790 | .3071 | .4585 | .5335 |
| U-Net | Yes | .5970 | .4791 | .3904 | .3903 | .6609 |
| SegFormer-B0 | Yes | .6636 | .5762 | .4368 | .3000 | .7340 |
| Class | U-Net, no label | U-Net + label | SegFormer-B0 |
|---|---|---|---|
| Horizontal | .2838 | .4705 | .6459 |
| Vertical | .4465 | .5264 | .5281 |
| Diagonal | .3719 | .4596 | .4531 |
| Curved | .3789 | .4452 | .5052 |
| Triangle | .5889 | .6595 | .7841 |
| Rule of thirds | .6397 | .7840 | .8704 |
| Model | Dice | IoU | B-F1 | Over. | Hit | Invalid |
|---|---|---|---|---|---|---|
| Qwen3-VL-32B | .3946 | .3243 | .4396 | .5231 | .3760 | .0008 |
| Qwen3-VL-8B | .3770 | .3111 | .4309 | .5622 | .3519 | .0008 |
| MiniCPM-V 4.5 | .3693 | .3081 | .4107 | .5435 | .3429 | .0000 |
| InternVL3.5-38B | .3634 | .3003 | .4114 | .5408 | .3384 | .0211 |
| LLaVA-OneVision-7B | .3312 | .2579 | .3971 | .6608 | .3120 | .0000 |
| Molmo2-8B | .3286 | .2747 | .3909 | .6071 | .3044 | .0000 |
| Class | Train | Mean | S / M / L (%) | Test | Mean | S / M / L (%) |
|---|---|---|---|---|---|---|
| Horizontal | 758 | .47 | 13 / 83 / 4 | 633 | .55 | 9 / 64 / 27 |
| Vertical | 195 | .11 | 84 / 15 / 1 | 198 | .22 | 70 / 24 / 7 |
| Diagonal | 471 | .44 | 28 / 60 / 12 | 567 | .59 | 12 / 45 / 43 |
| Curved | 235 | .01 | 99 / 1 / 0 | 276 | .14 | 84 / 13 / 3 |
| Triangle | 374 | .34 | 45 / 52 / 4 | 513 | .54 | 22 / 38 / 40 |
| Rule of thirds | 923 | .02 | 98 / 1 / 0 | 375 | .20 | 72 / 22 / 5 |
| Rater | Split | Ratings | Mean | Segment | Mixed | Line |
|---|---|---|---|---|---|---|
| rater_01 | Test | 1327 | .250 | 58.9% | 33.2% | 7.9% |
| rater_02 | Train | 4093 | .196 | 67.5% | 29.8% | 2.8% |
| rater_03 | Test | 1327 | .495 | 34.5% | 27.4% | 38.1% |
| rater_04 | Test | 1327 | .337 | 45.9% | 44.6% | 9.5% |
| Modality predictor | Accuracy | Macro-F1 | MAE | Invalid |
|---|---|---|---|---|
| Always segment | .4943 | .2205 | .3082 | – |
| Training-set class prior | .7204 | .5008 | .1398 | – |
| Molmo2-8B | .4348 | .3522 | .2833 | .0000 |
| Qwen3-VL-32B | .3338 | .2832 | .4668 | .0000 |
| Qwen3-VL-8B | .3157 | .3048 | .4672 | .0000 |
| LLaVA-OneVision-7B | .3263 | .2615 | .4759 | .0000 |
| Method | Dice | IoU | B-F1 | Over. | Hit |
|---|---|---|---|---|---|
| DINO, area matched | .2102 | .1432 | .1413 | .7898 | .1515 |
| DINO, best area sweep | .3404 | .2314 | .2018 | .7307 | .2524 |
| Grad-CAM, area matched | .5176 | .3793 | .1371 | .4907 | .6074 |
| Grad-CAM, best area sweep | .5833 | .4328 | .1947 | .4927 | .7242 |
| Mask source | Blur drop | Fill drop | Blur flip | Fill flip | Mask D | Area |
|---|---|---|---|---|---|---|
| Human consensus | 1.0192 | 2.0311 | .0965 | .2095 | 1.0000 | .2827 |
| Random matched area | .1645 | .3754 | .0173 | .0347 | .2827 | .2827 |
| Edge matched area | 1.0981 | 2.2527 | .0942 | .2306 | .4499 | .3246 |
| Full image | 2.3069 | 4.0795 | .2622 | .7611 | .4085 | 1.0000 |
| U-Net + label | 1.1649 | 2.8026 | .1168 | .2833 | .5970 | .3160 |
| SAM2.1 Area@3 | .5675 | 1.5486 | .0603 | .1703 | .4133 | .3294 |
| Path | Contents |
|---|---|
| annotations/single_annotator_train/ | 4,093 records from 3,169 train images and their binary masks |
| annotations/expert_test/expert_01..03/ | Three pseudonymous bundles of 1,327 records and masks |
| consensus/strict/ | 1,327 strict-majority masks |
| consensus/ambiguity_aware/ | 1,327 ambiguity-aware masks |
| consensus/manifest.csv | Task-to-consensus-mask index |
| metadata/preference_ratings.csv | Exploratory expert modality ratings |
| Method | Dice | IoU | B-F1 |
|---|---|---|---|
| Template@3 | 0.4016 [0.3831, 0.4200] | 0.3204 [0.3030, 0.3382] | 0.4412 [0.4255, 0.4567] |
| Qwen3-VL-32B | 0.3946 [0.3752, 0.4139] | 0.3243 [0.3057, 0.3431] | 0.4396 [0.4232, 0.4556] |
| Greedy Oracle@3 | 0.6400 [0.6202, 0.6595] | 0.5745 [0.5539, 0.5952] | 0.5963 [0.5777, 0.6154] |
| Exact, exactly 3 | 0.6401 [0.6202, 0.6598] | 0.5746 [0.5534, 0.5958] | 0.5964 [0.5771, 0.6152] |
| Exact, up to 3 | 0.6448 [0.6248, 0.6644] | 0.5820 [0.5604, 0.6030] | 0.6096 [0.5896, 0.6292] |
| InternVL3.5-38B | 0.3634 [0.3438, 0.3832] | 0.3003 [0.2817, 0.3194] | 0.4114 [0.3951, 0.4282] |
| Method | Overcoverage | [email protected] | Invalid |
|---|---|---|---|
| Template@3 | 0.5401 [0.5195, 0.5605] | 0.3828 [0.3572, 0.4092] | 0.0000 [0.0000, 0.0000] |
| Qwen3-VL-32B | 0.5231 [0.5015, 0.5446] | 0.3760 [0.3497, 0.4024] | 0.0008 [0.0000, 0.0023] |
| Greedy Oracle@3 | 0.1226 [0.1087, 0.1371] | 0.6330 [0.6074, 0.6594] | 0.0000 [0.0000, 0.0000] |
| Exact, exactly 3 | 0.1214 [0.1074, 0.1361] | 0.6330 [0.6066, 0.6586] | 0.0000 [0.0000, 0.0000] |
| Exact, up to 3 | 0.0636 [0.0535, 0.0742] | 0.6368 [0.6104, 0.6631] | 0.0000 [0.0000, 0.0000] |
| InternVL3.5-38B | 0.5408 [0.5191, 0.5627] | 0.3384 [0.3127, 0.3640] | 0.0211 [0.0136, 0.0294] |
| Method | Dice | IoU | B-F1 |
|---|---|---|---|
| Template@3 | 0.3989 [0.3801, 0.4176] | 0.3201 [0.3023, 0.3380] | 0.4348 [0.4190, 0.4506] |
| Qwen3-VL-32B | 0.3988 [0.3792, 0.4185] | 0.3291 [0.3103, 0.3481] | 0.4383 [0.4219, 0.4549] |
| Greedy Oracle@3 | 0.6431 [0.6232, 0.6630] | 0.5816 [0.5605, 0.6028] | 0.6080 [0.5886, 0.6270] |
| Exact, exactly 3 | 0.6432 [0.6228, 0.6628] | 0.5818 [0.5602, 0.6026] | 0.6085 [0.5890, 0.6275] |
| Exact, up to 3 | 0.6498 [0.6295, 0.6703] | 0.5913 [0.5696, 0.6132] | 0.6253 [0.6052, 0.6456] |
| InternVL3.5-38B | 0.3674 [0.3476, 0.3872] | 0.3059 [0.2871, 0.3249] | 0.4085 [0.3915, 0.4256] |
| Method | Overcoverage | [email protected] | Invalid |
|---|---|---|---|
| Template@3 | 0.5613 [0.5410, 0.5813] | 0.3791 [0.3527, 0.4047] | 0.0000 [0.0000, 0.0000] |
| Qwen3-VL-32B | 0.5380 [0.5166, 0.5592] | 0.3828 [0.3564, 0.4099] | 0.0008 [0.0000, 0.0023] |
| Greedy Oracle@3 | 0.1492 [0.1334, 0.1650] | 0.6383 [0.6127, 0.6639] | 0.0000 [0.0000, 0.0000] |
| Exact, exactly 3 | 0.1486 [0.1325, 0.1646] | 0.6383 [0.6119, 0.6639] | 0.0000 [0.0000, 0.0000] |
| Exact, up to 3 | 0.0779 [0.0663, 0.0898] | 0.6436 [0.6172, 0.6692] | 0.0000 [0.0000, 0.0000] |
| InternVL3.5-38B | 0.5522 [0.5305, 0.5736] | 0.3482 [0.3225, 0.3730] | 0.0211 [0.0136, 0.0294] |
| Method | H | V | D | C | T | R | Ce | S | P |
|---|---|---|---|---|---|---|---|---|---|
| Template@3 | 0.566 | 0.340 | 0.354 | 0.291 | 0.408 | 0.334 | 0.457 | 0.307 | 0.213 |
| Qwen3-VL-32B | 0.571 | 0.544 | 0.356 | 0.281 | 0.301 | 0.363 | 0.437 | 0.302 | 0.184 |
| Greedy Oracle@3 | 0.839 | 0.716 | 0.581 | 0.509 | 0.563 | 0.858 | 0.655 | 0.447 | 0.266 |
| Exact, exactly 3 | 0.840 | 0.716 | 0.581 | 0.509 | 0.563 | 0.858 | 0.655 | 0.447 | 0.266 |
| Exact, up to 3 | 0.842 | 0.717 | 0.585 | 0.514 | 0.567 | 0.865 | 0.664 | 0.447 | 0.267 |
| InternVL3.5-38B | 0.552 | 0.376 | 0.339 | 0.302 | 0.282 | 0.236 | 0.429 | 0.281 | 0.161 |
| Method | H | V | D | C | T | R | Ce | S | P |
|---|---|---|---|---|---|---|---|---|---|
| Template@3 | 0.566 | 0.357 | 0.327 | 0.277 | 0.406 | 0.339 | 0.460 | 0.309 | 0.217 |
| Qwen3-VL-32B | 0.568 | 0.557 | 0.366 | 0.279 | 0.302 | 0.368 | 0.443 | 0.298 | 0.207 |
| Greedy Oracle@3 | 0.840 | 0.718 | 0.588 | 0.474 | 0.569 | 0.865 | 0.660 | 0.448 | 0.296 |
| Exact, exactly 3 | 0.840 | 0.718 | 0.588 | 0.474 | 0.570 | 0.866 | 0.661 | 0.448 | 0.296 |
| Exact, up to 3 | 0.843 | 0.721 | 0.599 | 0.490 | 0.576 | 0.873 | 0.669 | 0.448 | 0.296 |
| InternVL3.5-38B | 0.553 | 0.382 | 0.348 | 0.291 | 0.285 | 0.241 | 0.436 | 0.278 | 0.177 |
Visual Grounding
Visual grounding connects language to the image regions, objects, or pixels that support a description, question, or action.
See all 167 papers