Open Vision Banana: A Fully Open Recipe for Generative Visual Perception
Organizations: Fundamental AI Lab, University of Technology Nuremberg · DLR · AIST · University of Tübingen, Tübingen AI Center ·
Abstract
Recent work suggests that pretrained image generators encode real-world understanding beyond image synthesis. Vision Banana exploits this by framing vision tasks as conditional image editing. However, its data, weights, code, and implementation details are not public. We present Open Vision Banana (OVB), a fully open and reproducible recipe for converting pretrained generators into unified multi-task perception systems. Our analyses identify the design choices that matter most for generalization, including color encoding, prompt design, parameter adaptation, and task balancing. These choices interact, so individually helpful choices can hurt when combined with others. Starting from 4B and 9B parameter open-weight base models and using 2.1M-3.8M training pairs from public datasets, OVB surpasses Vision Banana's reported Cityscapes zero-shot performance (73.9 vs. 69.9 mIoU) and rivals state-of-the-art affine-invariant depth estimators. Its non-zero-shot variants come within five points of Vision Banana on referring and reasoning segmentation but remain further behind on instance segmentation. All models largely retain the base model's image-editing abilities. With legend ensembling, we introduce a test-time scaling method for generator-based perception. It segments an image under several class-color legends and fuses the results by majority vote, correcting unstable color-class bindings and improving mIoU by up to 8.7 points on Cityscapes and 15.0 points on COCO-Object without retraining. We release the dataset construction and processing pipeline, the training and evaluation code, as well as the final, intermediate, and ablation checkpoints. Project website: https://openvisionbanana.github.io
Figures & tables
| Model | ZS | SemSeg mIoU | RefSeg cIoU | ReaSeg gIoU | InstSeg cgF1 | Depth | Normal err. |
| Open generator-based generalists | |||||||
| OVB 9B | ✓ | 73.9 ⋄ (65.6) | 57.4 | 67.1 ⋆ | 37.4 ⋄ (36.0) | 97.4 | 19.85 |
| OVB 9B | ✗ | 75.7 ⋄ (69.7) | 69.2 | 68.6 ⋆ | 39.6 ⋄ (38.1) | 99.0 | 13.51 |
| OVB 4B | ✓ | 71.3 ⋄ (65.1) | 57.0 | 67.9 ⋆ | 34.0 ⋄ (33.0) | 97.5 | 19.87 |
| OVB 4B | ✗ | 76.4 ⋄ (69.2) | 65.6 | 74.6 ⋆ | 35.9 ⋄ (34.8) | 98.7 | 13.21 |
| ZS Vision Learners ‡ [ 54 ] | ✓ | 25.7 † | n/a | n/a | n/a | — | 20.5 A |
| Method | DiffSegm. [ 93 ] | iSeg [ 87 ] | Talk2DINO ∗ [ 5 ] | RF-CLIP [ 48 ] | FSeg-LLaVA [ 121 ] | OV-Stitcher [ 65 ] | CorrCLIP [ 122 ] | OVB 9B (ZS) | OVB 9B (NZS) |
| mIoU | 37.9 | 38.4 | 41.0 | 41.8 | 42.0 | 46.5 | 49.4 | 53.0 ⋄ (38.7) | 59.3 ⋄ (45.0) |
| Model | ZS | Raw | Qwen | Gemma4 | Gemini |
| F2K4B Base (pretrained) | ✓ | 18.5 | 24.2 | 24.1 | 23.6 |
| OVB 4B | ✓ | 41.2 | 68.0 | 67.9 | 69.3 |
| OVB 4B | ✗ | 51.2 | 71.9 | 74.6 | 74.6 |
| Model | ZS | cgF1 | MCC | pmF1 |
| SenseNova-Vision † [ 31 ] | ✗ | 30.0 | 0.72 | 39.8 |
| SAM 3 [ 12 ] | ✗ | 54.1 | 0.82 | 66.1 |
| SAM 3 + Llama 3.2 (ft) | ✗ | 61.2 | 0.86 | 70.8 |
| gDINO-T [ 52 ] | ✓ | 3.3 | 0.15 | 16.2 |
| LLMDet-L [ 21 ] | ✓ | 6.5 | 0.21 | 27.3 |
| Gemini 2.5 [ 26 ] | ✓ | 13.0 | 0.29 | 46.1 |
| IoU | (a) Training progression ( ) | ||||
| 12.5K | 22.5K | 56.3K | 74.3K | 93K | |
| Standard readout | 51.1 | 56.7 | 61.2 | 65.5 | 68.1 |
| Oracle relabeling | 83.6 | 84.7 | 85.9 | 85.2 | 85.9 |
| Aligned depth (%) | ||||||
| Method | ZS | NYU | KITTI | ETH3D | ScanNet | DIODE |
| Marigold V2 † | ✓ | 98.0 | 97.4 | 99.2 | 97.9 | 97.1 |
| Lotus-2 † | ✓ | 97.6 | 94.1 | 98.6 | 97.2 | 95.5 |
| FE2E † | ✓ | 97.6 | 96.0 | 98.7 | 97.1 | 96.4 |
| Vision Banana | ✓ | 94.8 ‡ | 91.5 ‡ | 93.5 ‡ | – | – |
| OVB 9B | ✓ | 97.4 | 95.3 | 98.2 | 93.4 | 95.4 |
| Model | Basic avg. | UGE avg. |
| F2K4B Base | 3.66 (3.65) | 3.69 (3.67) |
| OVB 4B (zero-shot mix) | 3.78 | 3.75 |
| OVB 4B (non-zero-shot mix) | 3.82 | 3.79 |
| F2K9B Base | 3.89 (3.89) | 4.03 (3.99) |
| OVB 9B (zero-shot mix) | 3.86 | 3.95 |
| OVB 9B (non-zero-shot mix) | 3.89 | 3.68 |
| Model | VQAScore | |
| F2K4B Base | 0.7911 | – |
| OVB 4B (zero-shot mix) | 0.7412 | |
| OVB 4B (non-zero-shot mix) | 0.7418 | |
| F2K9B Base | 0.8123 | – |
| OVB 9B (zero-shot mix) | 0.7823 | |
| OVB 9B (non-zero-shot mix) | 0.7857 |
| Model | COCO20k | ScanNet | SPair |
| DINOv3 7B/16 (published) | 55.1 | – | 58.7 † |
| DINOv3 7B/16 (our run) | 54.8 | 40.7 | 56.1 |
| F2K4B Base (pretrained) | 39.3 | 27.6 | 57.8 |
| OVB 4B | 47.6 | 40.7 | 59.4 |
| F2K9B Base (pretrained) | 52.5 | 25.6 | 59.3 |
| OVB 9B | 56.3 | 30.5 | 61.9 |
| Considered classes & class–color mapping | Cityscapes | COCO- Stuff | COCO- Things | ||
| fixed | rd | set | rd | rd | |
| (#1) all classes & canonical colors ( fixed ) | 53.9 | 7.1 | 21.2 | 4.4 | 3.5 |
| (#2) all classes & random colors ( rd ) | 31.5 | 51.0 | 53.6 | 19.8 | 13.0 |
| (#3) random subset & random colors ( set ) | 24.1 | 36.0 | 66.5 | 14.7 | 10.5 |
| Palette | Cityscapes | COCO-Stuff | COCO-Things | |||
| set | rd | set | rd | set | rd | |
| All 200 colors | 75.1 | 57.9 | 54.8 | 32.6 | 43.1 | 18.8 |
| Top 100 | 73.3 | 59.6 | 57.1 | 36.6 | 55.4 | 35.4 |
| Top 64 | 74.4 | 60.3 | 60.5 | 38.7 | 59.4 | 33.6 |
| Top 32 | 75.8 | 60.8 | 51.3 | 36.5 | 43.9 | 31.4 |
| Top (# classes per img.) | 75.2 | 57.9 | 56.1 | 38.2 | 53.9 | 35.0 |
| Training fraction | Depth | Normals | Guardrails | ||||||
| AbsRel | (%) | Mean | Med. | CS | CO | RS | IS | IE | |
| (a) Image generation | |||||||||
| 0% | 0.064 | 96.2 | 22.2 | 12.9 | 25.9 | 15.1 | 40.1 | 17.6 | 4.1 |
| 7.5% | 0.074 | 93.5 | 26.0 | 17.0 | 30.6 | 19.5 | 43.9 | 19.0 | 3.8 |
| 15% default | 0.061 | 96.5 | 24.9 | 16.5 | 27.4 | 21.2 | 45.8 | 17.5 | 4.0 |
| 30% | 0.063 | 96.6 | 26.7 | 18.2 | 30.8 | 14.6 | 45.7 | 15.0 | 3.7 |
| Depth | Normals | Other tasks | |||||||
| Updated scope | AbsRel | (%) | Mean ( ∘ ) | Med. ( ∘ ) | CS | CO | RS | IS | IE |
| Last 4 | 0.120 | 85.3 | 35.2 | 30.6 | 4.5 | 3.5 | 12.5 | 3.2 | 3.60 |
| Last 12 | 0.090 | 91.8 | 22.8 | 17.9 | 6.4 | 4.1 | 25.6 | 11.5 | 3.96 |
| All 24 | 0.069 | 95.1 | 20.2 | 14.7 | 12.2 | 5.0 | 32.0 | 16.7 | 3.56 |
| Full except embeddings | 0.068 | 95.3 | 19.3 | 13.4 | 21.6 | 12.8 | 41.1 | 18.3 | 3.66 |
| Full transformer † | 0.068 | 95.1 | 18.9 | 13.0 | 26.0 | 15.1 | 44.4 | 17.5 | 3.74 |
Appendix figures & tables19 assets
Supplementary material from the paper’s appendix.
Appendix
| Task adaptation / evaluation | Reproducibility | ||||||||||||||
| Model | Base Model / Paradigm | Sem. seg. | Ref. seg. | Inst. seg. | Prompt seg. | Depth | Normals | Class. | Det./pose | Gen./edit kept | Arch. | Weights | Train code | Train data | Data pipeline |
| Open Vision Banana | open I2I diffusion / flow model | ✓ | ✓ | ✓ | ✗ | ✓ | ✓ | ✓ | ✗ | ✓ | |||||
| Generalist / multi-task generative perception models | |||||||||||||||
| Vision Banana [ 23 ] | Nano Banana Pro | ✓ | ✓ | ✓ | ✗ | ✓ | ✓ | ✗ | ✗ | ✓ | |||||
| Z.S. Vision Learners [ 54 ] | frozen open image editors | ❄ | ✗ | ✗ | ✗ | ❄ | ❄ | ✗ | ✗ | n/a | n/a | ||||
| RINO-Zero [ 111 ] | frozen open image editors / RGB interface | ❄ | ❄ | ❄ | ✗ | ❄ | ❄ | ✗ | ❄ | ❄ | n/a | n/a | |||
| Setting | OVB 4B | OVB 9B |
| Optimizer | AdamW, , , , weight decay , gradient-norm clipping at 1.0 | |
| Schedule | constant learning rate after linear warm-up | |
| Training length | 100K optimizer steps ( 19 epochs) | |
| Sample plan | 1M rows per epoch, drawn with replacement, seed 42 | |
| Batch | global batch 192 | |
| FLUX guidance embedding | 3.5 during training and inference | |
| Setting | Segmentation | Geometry |
| Optimizer | AdamW, , , , weight decay , gradient-norm clipping at 1.0 | |
| Learning rate | ||
| Schedule | constant with 1,000-step warm-up | |
| Training length | 37.5K optimizer steps | |
| Evaluated checkpoint | final 37.5K-step checkpoint | |
| FLUX guidance embedding | 3.5 during training and inference | |
| Benchmark | Metric | Averaged | 100K | |
| Cityscapes | mIoU | 65.4 | 64.4 | |
| RefCOCOg | cIoU | 57.4 | 56.8 | |
| ReasonSeg | gIoU | 66.9 | 66.3 | |
| SA-Co/Gold | cgF1 | 36.2 | 35.6 | |
| ImgEdit Basic | score | 3.86 | 3.88 | |
| ImgEdit UGE | score | 3.95 | 3.89 |
| Task type | Segm. | Geom. | OVB |
| Semantic segmentation | 24.8% | 14% | 14.3% |
| Referring segmentation | 18.8% | 8% | 13.2% |
| Instance segmentation | 24.8% | 8% | 14.3% |
| Depth estimation | 0% | 20% | 10.0% |
| Surface normals | 0% | 10% | 20.0% |
| Image editing | 11.9% | 15% | 11.3% |
| Task type | Training sources | Evaluation sources |
| Semantic segmentation 14.3% of samples | ACDC [ 80 ] 1.0 0.6 , ADE20K [ 127 ] 11.0 6.9 , BDD100K [ 115 ] 4.0 2.5 , FoodSeg 103 [ 103 ] 3.5 2.2 , GOOSE [ 66 ] 4.4 2.7 , Hypersim [ 76 ] 7.3 4.6 , Mapillary Vistas [ 69 ] 9.9 6.2 , MUSES [ 7 ] 0.9 0.5 , PASCAL VOC [ 20 ] 1.4 0.9 , PASCAL-Context [ 67 ] 5.0 3.1 , RailSem19 [ 119 ] 4.2 2.6 , Trans10K-v2 [ 105 ] 3.0 1.9 , VIPSeg [ 62 ] 37.3 23.3 , Waymo PVPS [ 60 ] 7.0 4.4 , Cityscapes [ 14 ] 0.9 , COCONut-XL (COCO images) [ 17 , 49 ] 15.0 , COCONut-XL (Objects365 images) [ 17 , 82 ] 5.0 , IDD [ 88 ] 2.5 , nuImages [ 11 ] 5.1 , Synscapes [ 100 ] 7.8 , WildDash2 [ 120 ] 1.3 | Cityscapes [ 14 ] , COCO thing/stuff [ 49 , 10 ] (w/ 12/15 super-categories and 500 image subsets for ablations), COCO-Object (80 Categories) [ 49 ] |
| Referring segmentation 13.2% of samples | Visual Genome [ 43 ] (w/o COCO-derived samples) 87.6 20.5 and Mapillary Vistas [ 69 ] 12.4 9.5 (annotations from [ 79 ] ), RefCOCO [ 37 ] 8.6 , RefCOCO+ [ 116 ] 8.6 , RefCOCOg [ 59 ] 5.6 , gRefCOCO [ 51 ] 11.7 , PhraseCut [ 101 ] 11.5 , LVIS [ 30 ] (annotations [ 79 ] ) 23.1 , ReasonSeg [ 45 ] 1.0 | RefCOCOg UMD [ 59 , 68 ] , ReasonSeg [ 45 ] |
| Instance segmentation 14.3% of samples | ACDC [ 80 ] 1.3 0.5 , ADE20K [ 127 ] 12.4 5.1 , GOOSE [ 66 ] 5.6 2.3 , Hypersim [ 76 ] 8.7 3.6 , Mapillary Vistas [ 69 ] 13.1 5.4 , MUSES [ 7 ] 1.1 0.4 , OVIS [ 72 ] 5.6 2.3 , PASCAL VOC [ 20 ] 1.9 0.8 , VIPSeg [ 62 ] 42.8 17.6 , Waymo PVPS [ 60 ] 7.5 3.1 , COCONut-XL (COCO images) [ 17 , 49 ] 24.0 , COCONut-XL (Objects365 images) [ 17 , 82 ] 6.0 , IDD [ 88 ] 2.1 , nuImages [ 11 ] 4.1 , PACO-LVIS [ 74 ] 14.7 , Synscapes [ 100 ] 6.7 , WildDash2 [ 120 ] 1.1 | SA-Co/Gold [ 12 ] (Qwen3-VL-8B presence gate) Ablations – 14K-query seed-0 subset (7 subsets 2K queries) Main result comparisons – all 168K queries |
| Depth estimation 10.0% of samples | Synscapes [ 100 ] 20.0 11.1 , Hypersim [ 76 ] 20.0 11.1 , Taskonomy [ 118 ] 20.0 11.1 , Matterport3D [ 13 ] 20.0 11.1 , Replica [ 86 ] 20.0 11.1 , NYUv2 [ 84 ] 11.1 , DIODE [ 89 ] 11.1 , ScanNet v2 [ 15 ] 11.1 , Virtual KITTI 2 [ 9 ] 11.1 | Final benchmarks: NYUv2 [ 84 ] , DIODE [ 89 ] , KITTI Eigen [ 25 ] , ETH3D [ 81 ] , ScanNet v2 [ 15 ] . Development diagnostics (Sec. C.4 ): iBims-1 [ 41 ] , nuScenes [ 11 ] , Virtual KITTI 2 [ 9 ] |
| Surface normals 20.0% of samples | Synscapes [ 100 ] 20.0 11.1 , Hypersim [ 76 ] 20.0 11.1 , Taskonomy [ 118 ] 20.0 11.1 , Matterport3D [ 13 ] 20.0 11.1 , Replica [ 86 ] 20.0 11.1 , NYUv2 [ 84 ] 11.1 , DIODE [ 89 ] 11.1 , ScanNet v2 [ 15 ] 11.1 , Virtual KITTI 2 [ 9 ] 11.1 | Final benchmarks: NYUv2 [ 84 ] , DIODE [ 89 ] (official normals), iBims-1 [ 41 ] and Sintel (reference normal targets), Virtual KITTI 2 [ 9 ] , ScanNet v2 [ 15 ] . Development diagnostics: depth-derived normals for ETH3D [ 81 ] , KITTI [ 25 ] , nuScenes [ 11 ] , iBims-1, Virtual KITTI 2 and ScanNet v2 |
| Image editing 11.3% of samples | Pico-Banana-400K [ 73 ] 100 100 | ImgEdit-Bench [ 114 ] |
| Dataset | Depth | Normals | Depth range |
| NYUv2 [ 84 ] | 654 | 654 | |
| KITTI Eigen [ 25 ] | 652 | – | |
| ETH3D [ 81 ] | 454 | – | |
| ScanNet [ 15 ] | 800 | 300 | |
| DIODE [ 89 ] | 771 | 325 | |
| iBims-1 [ 41 ] | – | 100 | – |
| Method | mIoU |
| Task-specialist models | |
| CorrCLIP [ 122 ] | 49.4 |
| OV-Stitcher [ 65 ] | 46.5 |
| FSeg-LLaVA [ 121 ] | 42.0 |
| Ours with legend pruning (Qwen3-VL-8B-Instruct) | |
| OVB 9B (zero-shot) | 63.6 ⋄ (62.2) |
| Method | NYU | ScanNet | iBims-1 | Sintel |
| Marigold V2 [ 70 ] † | 16.6 / 61.2 | 14.1 / 67.4 | 15.9 / 70.9 | 28.7 / 27.6 |
| DSINE [ 2 ] † | 16.4 / 59.6 | 16.2 / 61.0 | 17.1 / 67.4 | 34.9 / 21.5 |
| Lotus-2 [ 33 ] † | 16.9 / 59.0 | 14.2 / 66.8 | 15.4 / 70.4 | 30.3 / 27.6 |
| FE2E [ 94 ] † | 16.2 / 59.6 | 13.8 / 67.2 | 15.1 / 70.6 | 31.2 / 22.3 |
| GeoWizard [ 22 ] † | 18.9 / 50.7 | 17.4 / 53.8 | 19.3 / 63.0 | 40.3 / 12.3 |
| Vision Banana [ 23 ] | 17.78 / – | 15.05 / – | – | – |
| Model | adjust | style | backgr. | extract | remove | replace | add | compose | action | Basic avg. | UGE avg. |
| FLUX.2 Klein 4B backbone | |||||||||||
| FLUX.2 Klein 4B Base | 3.43 | 2.97 | 4.13 | 4.00 | 2.97 | 3.29 | 4.64 | 3.38 | 3.98 | 3.66 | 3.69 |
| OVB 4B (zero-shot mix) | 3.82 | 2.93 | 4.05 | 4.05 | 3.81 | 3.19 | 4.60 | 3.51 | 3.91 | 3.78 | 3.75 |
| OVB 4B (non-zero-shot mix) | 3.81 | 2.70 | 4.06 | 4.01 | 4.16 | 3.37 | 4.67 | 3.49 | 3.90 | 3.82 | 3.79 |
| FLUX.2 Klein 9B backbone | |||||||||||
| FLUX.2 Klein 9B Base | 4.06 | 3.02 | 4.22 | 4.03 | 3.71 | 3.45 | 4.75 | 3.49 | 4.00 | 3.89 | 4.03 |
| Map | Scan | List | cgF1 | MCC | pmF1 |
| Random | Top Bottom | Stated | 30.82 | 0.80 | 38.18 |
| Random | Unordered | Unstated | 33.94 | 0.81 | 41.59 |
| Fixed | Top Bottom | Stated | 32.43 | 0.82 | 39.27 |
| Fixed | Left Right | Stated | 34.32 | 0.81 | 41.74 |
| Fixed | Top Bottom | Implicit | 34.72 | 0.81 | 42.54 |
| Depth | Normal | |||
| Configuration | AbsRel | (%) | Mean | Med. |
| Concise inverse / normal | 0.060 | 96.2 | 23.95 | 14.75 |
| Pixel-explicit inverse / normal | 0.059 | 96.4 | 23.64 | 14.34 |
| Scene-preserving inverse / normal | 0.061 | 96.4 | 23.98 | 14.91 |
| Scene-preserving metric RGB depth | 0.062 | 96.0 | – | – |
| Depth | Normals | Retention | |||||||
| Variant | AbsRel | (%) | Mean ( ∘ ) | Med. ( ∘ ) | CS | CO | RS | IS | IE |
| (a) Supervision and data mixture, 37.5K | |||||||||
| Masked reference (20/10) | 0.0645 | 96.35 | 23.10 | 14.29 | 15.85 | 18.51 | 36.71 | 18.28 | 3.56 |
| Training masks | |||||||||
| No mask | 0.0719 | 95.72 | 27.27 | 18.37 | 19.58 | 11.93 | 43.40 | 18.20 | 3.54 |
| Erosion, 8 px | 0.0656 | 96.28 | 23.65 | 15.11 | 15.51 | 16.56 | 41.05 | 17.76 | 3.64 |
| Table 30 : ( E.2.5 ) . Effect of replay and perception-data diversity. The 40% replay reference uses the geometry-ablation mixture, not the full-scale OVB mixture. Depth scores average the full NYUv2 and KITTI Eigen splits with disparity-affine alignment; normal scores average NYUv2 and DIODE indoor. Each dataset has equal weight within its task. Other-task evaluations follow Sec. 4 . Depth Normals Retention Training data AbsRel (%) Mean Med. CS CO RS IS IE (a) Replay fraction 1% 0.0670 96.01 21.91 16.62 28.77 11.71 41.58 16.32 2.74 5% 0.0643 95.79 19.52 14.47 23.56 17.54 40.50 17.26 3.84 10% 0.0650 95.71 18.96 13.58 28.69 18.79 43.36 16.54 3.36 40% reference 0.0676 95.31 19.30 13.41 21.64 12.80 41.14 18.33 3.66 (b) Unique perception examples 10% 0.0704 95.34 21.06 15.94 22.71 17.51 41.09 21.42 3.76 30% 0.0709 95.24 21.12 15.97 22.06 12.40 30.43 16.47 3.32 100% default 0.0676 95.31 19.30 13.41 21.64 12.80 41.14 18.33 3.66 | Table 31 : Generation retention for the geometry ablations. IG-C is the GenAI-Bench CLIP–FlanT5 score, higher is better. Panels (a)–(c) use 37.5K checkpoints; (d) uses the 50K generation/editing-data runs from Table 14 . Compare rows within each panel, using its study’s reference run. Setting IG-C (a) Adaptation scope Last 2 0.8118 Last 4 0.8095 Last 8 0.8078 Last 12 0.8036 All 24 0.7997 Full except embeddings 0.7962 Full transformer 0.7893 (b) Replay fraction 1% 0.7954 5% 0.7787 10% 0.7919 40% 0.7962 Setting IG-C (c) Unique perception examples 10% 0.7919 30% 0.7747 100% 0.7962 (d) Generation/editing allocation Generation 0% 0.8074 Generation 7.5% 0.7639 Generation 15% 0.7871 Generation 30% 0.7840 Editing/identity 0% 0.7635 Editing/identity 12.5% 0.7673 Editing/identity 25% 0.7871 Editing/identity 50% 0.7731 |
| Semantic seg. (mIoU ) | Ref. seg. | Reas. seg. | Inst. seg. | ImgEdit (score ) | Depth | Normals | |||||
| Adaptation | Steps | Trainable | Cityscapes | COCO-Object | RefCOCOg cIoU | ReasonSeg gIoU | SA-Co/Gold cgF1 | Basic | UGE | NYUv2 (%) | NYUv2 Mean ( ∘ ) |
| Full fine-tuning | 37.5K | 3.9B | 63.0 | 24.6 | 53.0 | 66.4 | 30.2 | 3.75 | 3.59 | 95.1 | 19.76 |
| Full fine-tuning | 100K | 3.9B | 65.8 | 30.6 | 55.9 | 69.2 | 26.8 | 3.80 | 3.51 | 97.2 | 19.34 |
| LoRA | 25K | 102M | 42.7 | 11.2 | 46.3 | 63.3 | 28.5 | 3.79 | 3.88 | 93.6 | 19.96 |
| 37.5K | 50.3 | 11.4 | 46.3 | 63.6 | 27.5 | 3.78 | 3.63 | 95.9 | 18.95 | ||
| 50K | 53.5 | 12.2 | 42.2 | 62.2 | 28.8 | 3.75 | 3.74 | 96.1 | 19.56 | ||
Open-Vocabulary Semantic Segmentation
Open-vocabulary semantic segmentation labels image pixels using text-defined categories beyond a fixed training vocabulary.
See all 93 papers