Image Generators are Generalist Vision Learners (Vision Banana)

TL;DR

Google DeepMind’s Vision Banana takes Nano Banana Pro — a frontier image generator — and lightly instruction-tunes it on a mixture of its original generation data plus a small slug of vision-task data, with all vision outputs (segmentation masks, depth maps, etc.) re-parameterized as RGB images. The single resulting model beats or rivals the Segment Anything series on segmentation and the Depth Anything series on metric depth, while preserving generation quality. The framing — and the empirical bet — is that image generation pretraining is to vision what next-token prediction is to language: a single generative objective that yields a generalist learner whose output space (RGB) doubles as a universal task interface. The paper positions this as a paradigm shift toward “Foundational Vision Models” built from generative pretraining rather than discriminative or contrastive objectives.

Key claims

Image generation pretraining produces general visual representations that, after lightweight instruction-tuning, hit SOTA on 2D and 3D vision tasks — beating or rivaling domain-specific specialists [Abstract].
Re-parameterizing vision-task outputs (masks, depth, etc.) as RGB images lets the base generator’s existing capacity be reused as a task interface, mirroring text generation’s role in language [Abstract].
Vision Banana matches or beats the Segment Anything series on segmentation tasks and the Depth Anything series on metric depth estimation [Abstract].
The recipe is “lightweight instruction-tuning without sacrificing the base model’s image generation capabilities” — i.e. generation quality is preserved post-tuning [Abstract].
The base model is Nano Banana Pro (NBP); the vision-task data added is small relative to the original generation training mixture [Abstract].

Method

Vision Banana is built by instruction-tuning the Nano Banana Pro image generator on a mixture of its original training data and a small amount of vision-task data. The critical reparameterization is that every vision task — semantic segmentation, instance segmentation, metric depth estimation, and the rest of the 2D/3D suite — has its output expressed as an RGB image. Masks become colored RGB overlays; depth becomes a colormapped image; the model emits these the same way it emits any other generated picture. Because the output space is already what the base model was trained to produce, no new heads, projections, or task-specific decoders are needed; the instruction-tune is genuinely a tune rather than an adapter-and-decoder graft.

Mixing original generation data in alongside the vision-task data is the mechanism for preserving generation quality — a familiar replay recipe (cf. Replaying pre-training data improves fine-tuning). The published material does not provide a per-task data scale, ablations on the mixture ratio, or comparison against larger task-specific datasets at parity.

Results

Beats or rivals the Segment Anything series on “various segmentation tasks” [Abstract] — exact numbers per benchmark not in the publication page summary.
Beats or rivals the Depth Anything series on metric depth estimation [Abstract].
Base image-generation capability is reported as preserved post-tune [Abstract].
The model is a single generalist, not a per-task fine-tune; results above are from one set of weights with task selection via instruction [Abstract].

The published abstract does not include numerical tables; deeper evaluation (per-dataset mIoU, AbsRel, RMSE, generation-side FID/GenEval) would need the full paper.

Why it’s interesting

This is the strongest filed argument so far that the “generation improves understanding” arrow on Unified Multimodal Models is not just a small-scale auxiliary-loss trick but the right organizing principle for a foundational vision model. It complements Generation Enhances Understanding in Unified Multimodal Models via Multi-Representation Generation — which showed +7.2 VSR / +3.7 Hallusion gains on a 3.6B UMM by auxiliary generation of depth and segmentation — by taking the opposite extreme: instead of bolting generation supervision onto an understanding-first model, start from a frontier generator and bolt understanding-as-generation onto it. The contrast with UniG2U-Bench: Do Unified Models Advance Multimodal Understanding? is also sharp: UniG2U found that inference-time “generate-then-answer” usually hurts unified models, but the Vision Banana recipe makes the generation itself the answer, sidestepping the brittle intermediate-image step. Worth pairing with the author list — Saining Xie’s “flavor of the bitter lesson” essay (The flavor of the bitter lesson for computer vision) reads as a manifesto for exactly this kind of move, and Kaiming He on the byline plus the explicit “deconstructing diffusion as SSL” lineage situates this as the empirical follow-through to that argument.