Skip to main content
Back to timeline
arXivSource publication:

Open Vision Banana releases an open generative-perception recipe, hitting 73.9 mIoU zero-shot on Cityscapes against Vision Banana's 69.9

Synopsis

Open Vision Banana releases a reproducible recipe that converts open-weight FLUX.2-Klein image generators into a unified multi-task perception model, trained only on public data to output segmentation, depth and surface normals. The zero-shot 9B model reaches 73.9 mIoU on Cityscapes, above the 69.9 reported for Vision Banana; its non-zero-shot variants come within five points on referring and reasoning segmentation but remain further behind on instance segmentation. The data pipeline, training code and checkpoints are released.

Source-provided article image: Open Vision Banana: A Fully Open Recipe for Generative Visual Perception
arXiv

Interpretation

An image generator can be turned into a unified segmentation and geometry model without adding a separate prediction head per task. Vision Banana showed the same idea, but its base model, data and code are proprietary, so which design choices produced those results was unclear. Starting from open-weight FLUX.2-Klein at 4B and 9B and 2.1M to 3.8M public training pairs, the zero-shot 9B model reaches 73.9 mIoU on Cityscapes, where Vision Banana reports 69.9.

Each task is written as an RGB image: given an input image and an instruction, the generator paints a colored result that a deterministic readout converts into a prediction. Segmentation prompts carry a randomly drawn class-color legend, depth uses per-image normalized inverse depth, and surface normals map camera-space components onto three color channels. Training keeps the base generator's rectified-flow objective and masks invalid geometry regions; the non-zero-shot runs took about 73 hours on 16 GH200 GPUs for 4B and 79 hours on 32 for 9B.

Legend ensembling, a training-free test-time scaling method, segments the same image under several class-color legends and fuses the passes by per-pixel majority vote. A single pass often paints the right region in the wrong class color, and voting mainly corrects these color-class binding errors. The zero-shot 9B model gains up to 8.7 mIoU on Cityscapes and 15.0 mIoU on COCO-Object; on Cityscapes it rises from 65.4 mIoU in one pass and saturates near 74 mIoU after about 10 to 15 passes.

Perspective

The demonstrated scope is four FLUX.2-Klein models fine-tuned on public data (4B and 9B, under zero-shot and non-zero-shot data regimes), covering semantic, referring, reasoning and instance segmentation plus depth and surface normals. Image-editing ability is largely retained, but text-to-image generation is not: perception fine-tuning lowers VQAScore for all four models on GenAI-Bench. Zero-shot means the evaluation benchmarks were excluded from training, while the tasks themselves were seen during training.

The authors state that the weaker perception of open base models relative to proprietary generators such as Nano Banana Pro is the main limitation, and the non-zero-shot models still trail Vision Banana and SAM 3 on instance segmentation. Legend-ensembling gains depend on the number of classes, saturating after about ten passes on Cityscapes while COCO-Object keeps improving over more passes; vote agreement ranks pixels by reliability and is not a calibrated probability of correctness.