Ideogram 4.0 Technical Details: Open model at the forefront of design

TL;DR

Ideogram 4.0 is a 9.3B-parameter open-weight text-to-image foundation model from Ideogram (Toronto), released June 3 2026 alongside a parallel press release, a model card, day-0 ComfyUI support, GitHub code (Apache-2.0), and a separate non-commercial weight license with a paid commercial tier. Architecturally it is a 34-layer single-stream DiT where text and image tokens share projections at every layer; the text encoder is Qwen3-VL-8B-Instruct and the DiT consumes hidden states from 13 of its intermediate layers concatenated along the feature dimension, rather than a single hidden slice. The headline design bet is data: training uses only structured JSON captions with per-element styling, optional bounding boxes, and color palettes, and the reference inference pipeline parses every prompt as JSON and validates it against the schema before generation. It is positioned as the best open-weight model on text rendering, layout control, and designer-preference ELO (#1 open / #2 overall on a third-party DesignArena leaderboard), at a parameter count well below larger open competitors (FLUX.2, HunyuanImage 3.0 80B-A13B).

Key claims

9.3B-parameter single-stream DiT in which text and image tokens share the same projections across 34 transformer blocks — each block uses self-attention with QK-RMSNorm, 3D multimodal RoPE placing text and image tokens in one positional frame, and a SwiGLU MLP, with AdaLN modulating attention and MLP residual paths from the flow-matching timestep [§Architecture].
The text encoder is Qwen3-VL-8B-Instruct (a vision-language model), and the DiT consumes hidden states from 13 of its intermediate layers concatenated along the feature dimension, rather than a single hidden state or no external encoder [§Architecture].
The model is trained exclusively on structured JSON captions with per-element styling and optional bounding boxes and color palettes; the reference inference pipeline parses every prompt as JSON and validates against the schema before generation [§Architecture, §Training data].
The training objective is flow matching: the DiT predicts a velocity field v(z_t, t) defining an ODE from noise to a clean latent; at inference an Euler sampler integrates over a logit-normal noise schedule whose mean auto-adjusts with resolution [§Training objective].
Classifier-free guidance is made asymmetric: the unconditional pass drops the text tokens entirely instead of replacing them with a learned null embedding [§Inference, “Classifier-free guidance”].
The JSON surface exposes three primitives that a flat text prompt cannot express directly: color palettes up to 16 hex colors per image (5 per element), bounding-box layout in [y_min, x_min, y_max, x_max] normalized to 0–1000, and typed text elements with a literal string plus a separate styling description [Fig. 1; model card §JSON surface].
On the authors’ five-axis capability radar (layout control via 7Bench mIoU, spatial reasoning via SpatialGenEval judged by Gemini-2.5-Flash, object fidelity, prompt alignment via Prism-Bench, text rendering via X-Omni English OCR), Ideogram 4.0 is presented as competitive with closed-source models (GPT Image 2, Nano Banana 2) and ahead of every open-weight release evaluated [Fig. 2].
On X-Omni English OCR for text-rendering vs parameter count, the 9.3B model is positioned in the small-but-strong corner ahead of every other open-weight model, against Hunyuan Image v3 reported at 80B total / ~13B active per token [Fig. 3].
In an internal designer-preference arena where graphic designers blind-pick the better of two generations, Ideogram 4.0 ranks second overall and first among open-weight models on ELO [Fig. 4].

Method

The model is one of the cleanest exemplars to date of the single-stream DiT family — text and image tokens are concatenated into a single sequence and pass through the same self-attention and MLP projections at every one of 34 layers, with a 3D multimodal RoPE giving text and image tokens a shared positional frame. The text encoder choice is the more unusual half of the architecture: rather than a CLIP/T5-style encoder or a frozen LLM read at one layer, the DiT pulls hidden states from 13 intermediate layers of Qwen3-VL-8B-Instruct and concatenates them along the feature axis, so the conditioning carries a multi-scale slice of the encoder’s representation. AdaLN modulates attention and MLP residual paths from the flow-matching timestep; QK-RMSNorm and SwiGLU MLPs round out the standard modern-DiT toolkit.

The data side is the bet. Every training caption is a JSON document with per-element styling, optional bounding boxes (0–1000 normalized), and color palettes (up to 16 hex colors per image, 5 per element); a “describe-to-structure-to-recreate” loop on the press-release page summarizes the recipe — first parse scenes/text/objects into structured data, then learn to rebuild images from that representation. At inference, Ideogram’s reference pipeline parses every prompt as JSON and validates it against the schema before generation; the trick that makes plain-text prompts usable is that a separate “magic prompt” step expands them into JSON first. CFG drops text tokens entirely on the unconditional pass — an asymmetric variant rather than the standard learned-null-embedding swap.

Results

The blog reports two evaluation axes. On the five-axis capability radar — layout control (7Bench mIoU), spatial reasoning (SpatialGenEval combined splits, judged uniformly by Gemini-2.5-Flash instead of the leaderboard’s Qwen2.5-VL-72B), object fidelity, prompt alignment (Prism-Bench alignment track), and text rendering (X-Omni English OCR accuracy on in-image text) — Ideogram 4.0 is presented as competitive with closed-source GPT Image 2 and Nano Banana 2 and ahead of every open-weight release evaluated [Fig. 2]. On parameter efficiency for text rendering, the 9.3B model sits alone in the “small-but-strong” corner ahead of every other open-weight release, against Hunyuan Image v3 at 80B total / ~13B active [Fig. 3]. In a blind internal designer-preference arena, Ideogram 4.0 ELO ranks #2 overall and #1 among open-weight models [Fig. 4]. The companion DesignArena third-party leaderboard, cited in launch coverage (gigazine, buildfastwithai, promptslove), corroborates the open-weight #1 / overall #4 (or #2 by some counts) ranking; designer-survey usability is reported at 3.55 / 5 vs Nano Banana 2 (2.84), Grok Imagine 1.0 (2.61), and FLUX.2 max (2.49). Numbers in launch coverage that should be taken as press-release-grade rather than peer-reviewed.

Why it’s interesting

Ideogram 4.0 is the strongest open-weight test yet of the hypothesis FIBO (Generating an Image From 1,000 Words: Enhancing Text-to-Image With Structured Captions) put on the wiki — that the bottleneck on controllable text-to-image is caption structure rather than model capacity. The two share the same design move (train exclusively on structured captions, validate against a schema at inference, use a VLM to expand short user prompts), but Ideogram 4.0 is a much larger and more production-oriented instance: at 9.3B vs FIBO’s 8B it claims to outperform open-weight competitors >5× its size on text rendering, and it ships with a commercial-license tier and day-0 ComfyUI integration. The architecture also shifts the bet on the encoder side relative to Qwen-Image Technical Report (Qwen-Image): both use a Qwen-family VLM as the text encoder, but Ideogram 4.0 reads from 13 intermediate layers concatenated rather than a single hidden state, doubling down on the “the LM’s representation, not its top-layer summary, is the conditioning signal” intuition. The single-stream-DiT-with-asymmetric-CFG architecture parallels Z-Image: An Efficient Image Generation Foundation Model with Single-Stream Diffusion Transformer (Z-Image’s S3-DiT) and Wan-Image: Pushing the Boundaries of Generative Visual Intelligence (Wan-Image), making this a useful three-way contrast at the same point in the design space. The press release also signals a forthcoming 4.0 follow-up that returns alpha channels and editable text layers directly from inference — if delivered, this would be a much stronger native-layered-output datapoint for Layered Image/Video Decomposition than the current background-removal-as-a-post-processing route (Bria AI Image Editing — Precision AI Image Editing Built for Production Scale, Stable-Layers: Fine-Tuning Image Layer Decomposition Models with VLM-Scored Reinforcement Learning).