Introducing Agentic Vision in Gemini 3 Flash

TL;DR

Google announces “Agentic Vision” in Gemini 3 Flash: a productized Think-Act-Observe loop that lets the model write and execute Python during visual reasoning — cropping, rotating, annotating, plotting, counting bounding boxes — and append the transformed image back into its own context before answering. Enabling code execution on Gemini 3 Flash is reported to give a consistent 5-10% quality boost across most vision benchmarks, and a customer (PlanCheckSolver) reports a 5% accuracy improvement on building-plan validation by enabling the same feature. The post is a deployment note rather than a research paper — no architecture, no training details, no ablations — but it’s the first frontier-lab framing of “image understanding as an active investigation” as a shipped API capability, and it makes the Think-Act-Observe loop the agentic-vision contract going forward.

Key claims

Agentic Vision turns image understanding from a single static glance into a Think-Act-Observe loop: the model formulates a multi-step plan, writes Python to manipulate or analyze the image (crop, rotate, annotate, count, plot), and appends the transformed image back to its context window before generating a final response [§“Agentic Vision: a new frontier AI capability”].
Enabling code execution with Gemini 3 Flash gives a consistent 5-10% quality boost across most vision benchmarks (no benchmarks or numbers cited in the post) [§intro].
Gemini 3 Flash is trained to implicitly zoom when fine-grained details are required — the zoom behavior fires without an explicit prompt nudge; other behaviors (rotating, visual math) currently still require explicit prompting and Google says they’re working to make them implicit [§“Zooming and inspecting”, §“What’s next”].
A customer case (PlanCheckSolver, building-plan validation) improved accuracy by 5% by enabling code execution with Gemini 3 Flash to iteratively inspect high-resolution architectural inputs — the model generates Python to crop specific regions (roof edges, building sections) and feeds the crops back into its context [§“Zooming and inspecting”].
For counting tasks, the model uses Python to draw bounding boxes and numeric labels directly onto the canvas as a “visual scratchpad” before committing to a final count, grounding the answer in pixel-perfect annotations rather than probabilistic estimation [§“Image annotation”].
For visual math / dense tables, the model offloads computation to a deterministic Python environment — parsing high-density tables, normalizing values, and emitting Matplotlib charts — to bypass the multi-step arithmetic hallucination failure mode of standard LLMs [§“Visual math and plotting”].
The feature ships today via Gemini API in AI Studio and Vertex AI, and is rolling out in the Gemini app (Thinking mode in the model dropdown); the announced roadmap adds (a) more implicit code-driven behaviors beyond zoom, (b) more tools including web and reverse image search, (c) the same capability on other Gemini model sizes beyond Flash [§“What’s next”, §“How to get started”].

Method

Agentic Vision is described as adding a Think-Act-Observe loop on top of Gemini 3 Flash’s existing image understanding, with code execution as the first supported tool. The Think step is plan formulation given the user query + initial image; the Act step generates and executes Python that either manipulates the image (e.g. cropping, rotating, annotating) or computes over it (e.g. running calculations, counting bounding boxes); the Observe step appends the transformed image back into the model’s context window so the next reasoning turn can inspect the new data. The post does not disclose how the model was trained to do this (no RL recipe, no SFT data, no architecture changes mentioned), and it does not specify which exact Python libraries or sandbox the code runs in beyond noting that it parallels Gemini’s existing code-execution tool. The cited 5-10% benchmark boost is reported as a single aggregate number, with no per-benchmark breakdown or comparison against prior Gemini versions. Three product surfaces expose the feature: the Gemini API (turn on “Code Execution” under Tools in AI Studio Playground), Vertex AI (separate Vertex code-execution docs), and the Gemini app under the Thinking model dropdown.

Results

The only quantitative claims in the post are: (a) “a consistent 5-10% quality boost across most vision benchmarks” from enabling code execution with Gemini 3 Flash, with no per-benchmark numbers or named evaluations; and (b) a partner-reported 5% accuracy improvement on PlanCheckSolver’s building-plan validation product. No comparison against prior Gemini models, no head-to-head with GPT-5 / Claude / Qwen3-VL, no ablation isolating Agentic Vision from the rest of the Gemini 3 Flash upgrade. The three qualitative case studies (zooming on fine-grained details, counting fingers via annotated bounding boxes, parsing dense tables and emitting Matplotlib plots) are presented as demo-app screenshots rather than measured wins.

Why it’s interesting

This is the first frontier-lab productization of the “thinking with images via code execution” loop that the wiki has been tracking on the research side — it gives a name (Think-Act-Observe), a shipping API, and an implicit benchmark headline (5-10%) to a paradigm whose research-side instances are filed at Video-Thinker: Sparking "Thinking with Videos" via Reinforcement Learning (autoregressive MLLM invokes its own latent perception ops via tagged CoT), DiffThinker: Towards Generative Multimodal Reasoning with Diffusion Models (diffusion model treats the solution image as the CoT), and VisGym: Diverse, Customizable, Scalable Environments for Multimodal Agents (the same multi-turn loop benchmarked against frontier VLMs, where today’s top models top out at 46.6% Easy / 26.0% Hard). The Agentic Vision recipe sits closest to VisGym’s framing — same loop, same action space (perception/manipulation primitives expressed as code), and the same bet that interaction with the image beats static one-shot perception. Two things to watch: (i) the “implicit zoom is trained, other behaviors still need a prompt nudge” detail is a partial answer to the open question on Thinking with Modalities about whether tool-invocation should be trained vs. prompted, and Google is publicly committing to training all of them in; (ii) the productization signal — Google sees code-execution-as-vision-tool as worth a launch and a 5-10% headline number — strongly suggests the next batch of vision benchmark scores from frontier labs will be code-execution-enabled by default, which changes the apples-to-apples comparison surface for any open Luma evaluation.