Generating an Image From 1,000 Words: Enhancing Text-to-Image With Structured Captions

TL;DR

FIBO is an 8B-parameter DiT-based, flow-matching text-to-image model from BRIA AI trained exclusively on long, structured JSON captions (~1,000 words each) on licensed-only data. The structured supervision schema covers explicit fields for lighting, camera, composition, color, and depth-of-field, and the authors report that this yields native disentanglement — single attributes can be tweaked without prompt drift, enabling iterative refinement even though the model was never trained on image-pairs. The recipe pairs SmolLM3-3B as a frozen text encoder with a new conditioning mechanism (DimFusion) that fuses intermediate LLM token features into the DiT without lengthening the conditioning sequence, and uses the Wan 2.2 VAE. The paper also introduces TaBR (Text-as-a-Bottleneck Reconstruction), an evaluation protocol that measures controllability by closing a caption→generation loop on real images, and ships a fine-tuned Qwen-2.5 VLM (or Gemini 2.5 Flash) that expands short user prompts into the structured JSON schema.

Key claims

FIBO is the first open-source text-to-image model trained entirely on long, structured JSON captions averaging ~1,000 words each, covering disentangled visual fields (lighting, camera, composition, color, DoF) [§1, Fig. 1].
The structured-caption schema enables disentangled control: a single attribute can be edited (e.g. camera angle, skin tone) without unintended changes elsewhere, despite the model never being trained on image-editing pairs [§1, “Disentangled control”].
DimFusion is a conditioning mechanism that fuses intermediate tokens from a lightweight LLM (SmolLM3-3B) into the DiT without increasing the conditioning token length, enabling efficient long-caption training [§Method, abstract].
TaBR (Text-as-a-Bottleneck Reconstruction) is a new evaluation protocol that measures how well real images can be reconstructed through a captioning→generation loop, providing a controllability metric that remains meaningful at >1,000-word caption lengths where standard t2i benchmarks saturate [§Evaluation].
The 8B model is reported to achieve state-of-the-art prompt alignment among open-source baselines on a licensed-data subset of PRISM-Bench (image-text alignment and aesthetics) [§Evaluation; HF model card].
The system ships in three modes — Generate (short prompt → VLM expansion → image), Refine (existing JSON + short instruction → only-the-named-attributes change), Inspire (image → VLM extraction → variation) — all backed by the same DiT [HF model card §Work with FIBO in Three Simple Modes].
A fine-tuned Qwen-2.5-based VLM is released alongside FIBO as an open alternative to the default Gemini 2.5 Flash for the prompt→JSON expansion step [§1; HF model card §Training and Architecture].
The dataset is described as ~1B image candidates filtered for licensing, attribution traceability, and GDPR/EU-AI-Act compliance — positioned as an enterprise-grade, “legally safe” alternative to scraped-web training [HF model card §Data Distribution].

Method

FIBO is a standard 8B DiT trained with flow-matching, but the entire input pipeline is built around a structured caption schema rather than free-form text. At inference, a VLM (default: Gemini 2.5 Flash; open alternative: a fine-tuned Qwen-2.5) takes the user’s short prompt and emits a JSON object whose keys are visual factors — lighting, camera, composition, style_medium, artistic_style, depth-of-field, color palette, etc. — each filled with verbose values. The resulting JSON, often >1,000 words, is fed to SmolLM3-3B (frozen), and DimFusion injects intermediate-layer features from SmolLM3 into the DiT at multiple depths without growing the token count, which is what makes training on 1,000-word conditioning sequences tractable. The image-side VAE is Wan 2.2.

Two design bets do the heavy lifting. First, schema-as-supervision: because every training caption is structured into the same fields, the model learns to map each JSON field to a roughly independent direction in image space — this is what the authors mean by “native disentanglement.” A refinement at inference time is then just a JSON diff: the VLM rewrites only the relevant field (e.g. swap lighting: "golden hour" for lighting: "soft backlit at dusk") and re-generates, and only that aspect of the image changes. No image-pair editing data was used.

Second, TaBR for evaluation: standard t2i metrics (CLIPScore, GenEval, DPG, PRISM) were built around short prompts and don’t meaningfully distinguish models at the 1000-word regime. TaBR closes a loop on real images — caption an existing image with the VLM, regenerate from that caption, and measure reconstruction fidelity — turning controllability and expressiveness into a measurable axis even when the prompt is essentially a full scene specification.

Results

The HF model card reports that FIBO outperforms “comparable open-source baselines” on a licensed-data subset of PRISM-Bench across image-text alignment and aesthetics, attributing the gain to structured-caption training. Specific competitor numbers and ablation tables are in the paper but not surfaced in the materials inspected at filing time. The release ships day-0 API endpoints (Bria Platform, fal.ai, Replicate), ComfyUI nodes, and a BriaFiboPipeline integrated into HuggingFace Diffusers, all under CC-BY-NC-4.0 for non-commercial use [HF model card §Quick Start].

Why it’s interesting

FIBO is the cleanest test yet of a hypothesis that’s been quietly accumulating in the wiki — that the bottleneck on controllable image generation is caption structure, not model capacity. Two complementary precedents: Qwen-Image Technical Report (Qwen-Image) replaced the CLIP/T5 text encoder with a frozen multimodal LLM (Qwen2.5-VL) on the grounds that “the vision-language space is already aligned” and that LM capacity lets longer/more compositional prompts survive — but its training prompts were still free-form. FIBO takes the same architectural direction (decoder-only LM as text encoder, here SmolLM3-3B) but goes further on the data side: every training caption is a 1000-word JSON schema, which makes the disentanglement effect a property of the supervision rather than a model-side regularizer. The closest contrast on the wiki is PhotoFramer: Multi-modal Image Composition Instruction (PhotoFramer), which also uses long VLM-expanded prompts but for compositional control rather than disentangled attribute editing; FIBO’s claim is stronger — that the editing behavior of InstructX: Towards Unified Visual Editing with MLLM Guidance / Kontinuous Kontext: Continuous Strength Control for Instruction-based Image Editing-style instruction-tuned editors emerges for free from schema-structured pretraining, with no pair data. If that holds up, it sharpens the question of what dedicated editing recipes are actually buying over a single DiT with a richer caption distribution. The licensed-only data positioning is also notable as a deliberate counter-bet to the scraped-web open-foundation lineage (Qwen-Image Technical Report, HunyuanImage 3.0 Technical Report, Z-Image: An Efficient Image Generation Foundation Model with Single-Stream Diffusion Transformer).