PaperBanana: Automating Academic Illustration for AI Scientists

TL;DR

PaperBanana is an agentic pipeline that takes a paper’s source context + figure caption and produces a publication-ready methodology diagram, by orchestrating five specialized VLM/image-gen agents (Retriever, Planner, Stylist, Visualizer, Critic) in an iterative self-critique loop. The team releases PaperBananaBench — 292 test + 292 reference cases curated from NeurIPS 2025 methodology diagrams — and reports their system beats single-pass Nano-Banana-Pro and other baselines on faithfulness, conciseness, readability, and aesthetics. The framing positions academic-figure generation as a self-contained research workflow distinct from generic text-to-image generation, with VLM critique as the central refinement signal.

Key claims

The framework is a five-agent loop — Retriever, Planner, Stylist, Visualizer, Critic — that transforms scientific source context + caption into a methodology diagram via iterative self-critique [§Framework].
PaperBananaBench is curated from NeurIPS 2025 methodology diagrams through a four-stage pipeline (Collection & Parsing → Filtering → Categorization → Human Curation), yielding 584 valid samples split into 292 test + 292 reference cases [§Benchmark].
The test set has an average source-context length of 3,020.1 words and an average figure-caption length of 70.4 words [§Benchmark statistics].
PaperBanana outperforms leading baselines (including Nano-Banana-Pro) across four evaluation dimensions: faithfulness, conciseness, readability, aesthetics [§Results].
Auto-summarized aesthetic style guidelines from PaperBananaBench reference cases can also be applied as a post-hoc polish step on human-drawn diagrams, improving color schemes, typography, and graphical-element consistency [§Enhancing Aesthetics].
Image-generation models and code-based plotting trade off differently on statistical plots: image generation wins on aesthetics, code wins on faithfulness (numerical-hallucination and element-repetition failures dominate image-gen) [§Statistical Plots case study].
The primary failure mode is connection errors (redundant connections, mismatched source/target nodes), and the Critic agent often fails to identify them — suggesting the bottleneck is the foundation model’s perceptual ability to read diagram topology, not the iteration loop itself [§Failure Cases].

Method

PaperBanana decomposes the diagram-generation task into five role-specialized agents executed in sequence with feedback loops. The Retriever pulls reference diagrams from a corpus (PaperBananaBench reference cases). The Planner decides what to depict from the source context. The Stylist selects/summarizes aesthetic guidelines from retrieved references. The Visualizer invokes an image-generation backbone (Nano-Banana-Pro-style) to render. The Critic is a VLM that compares the rendered image against source context + caption + style brief and emits refinement instructions; the loop iterates Visualizer ↔ Critic until convergence or budget exhaustion.

PaperBananaBench is constructed by scraping NeurIPS 2025 methodology diagrams, filtering for visual quality and source-context alignment, categorizing by diagram type, and applying human curation. Each test case ships with the original source paragraph, the original figure caption, and the ground-truth diagram for evaluation. Evaluation uses four VLM-judged axes — faithfulness (does the diagram match the source context?), conciseness (is it free of clutter?), readability (typography, layout), and aesthetics (color, polish).

Results

The post does not publish numerical tables, only qualitative case studies and an aggregate claim of consistent dominance over baselines (Nano-Banana-Pro vanilla, other unspecified leading baselines) across all four axes. The released benchmark — 292 test + 292 reference cases, average source-context length 3,020.1 words, caption 70.4 words — is the main quantitative artifact.

Case studies highlight: (a) PaperBanana produces more concise, less verbose diagrams than vanilla Nano-Banana-Pro on the same input; (b) the aesthetic-style-guideline transfer can re-polish human-drawn diagrams to a higher visual standard; (c) on statistical plots, image-generation outputs are more visually appealing but suffer numerical hallucination — code-based plotting remains the faithfulness winner; (d) the principal failure mode is connection errors that the VLM Critic fails to detect.

Why it’s interesting

This is the wiki’s first datapoint on agentic image-generation pipelines aimed at structured, content-faithful outputs rather than general aesthetic synthesis — the central design choice (a VLM Critic in the iterative loop) is the same architectural pattern as the tool-use agents on Tool-Use Agents but applied to a generative rather than search/code domain. The reported failure mode — Critic fails to detect connection-topology errors — is a concrete instance of the broader “VLMs are weak at fine-grained structural perception” finding from VisGym (VisGym: Diverse, Customizable, Scalable Environments for Multimodal Agents §3.2), and suggests that the self-critique loop’s ceiling is set by the Critic’s perceptual acuity, not by the generator. The Image-vs-Code finding on statistical plots is also a clean data point: for any pipeline where numerical fidelity matters, image generation is the wrong primitive even if outputs look better. For Luma, the most reusable piece is the reference-driven style-guideline extraction step — a recipe for capturing aesthetic priors from a curated corpus and conditioning a generator on them, separate from the surrounding agent loop.