There and Back Again: Bidirectional Diffusion Bridges for Multimodality Translation

TL;DR

BIT (Bidirectional Image-Text Diffusion Bridges) is a Stanford (Ermon lab) generative framework that recasts multimodal translation as an SDE-based bridge between the source and target modalities, rather than as a denoising diffusion / flow from Gaussian noise conditioned on the source. The bridge interpolates directly from a text embedding endpoint to an image endpoint (or vice versa), so the same trained SDE can be simulated in either direction — one model handles both text→image and image→text without a separate captioner. Reports competitive quality against denoising-diffusion and deterministic-flow baselines on vision-language and natural-science evaluations.

Key claims

Existing text→image generative paths do not directly represent the source modality (source only enters as a conditioning signal on a Gaussian-noise → image path), limiting sampler flexibility and preventing inversion [Abstract].
BIT reframes the generative trajectory as a bidirectional bridge whose two endpoints are the source and target modality samples, providing a source-aware generative path amenable to diverse samplers [Abstract].
The endpoint-conditioned process admits reverse-direction traversal (image → text) with the same trained model, giving one unified bidirectional framework in place of separate forward/inverse networks [Abstract].
BIT is derived through stochastic calculus, yielding SDE forms that are simulable and loss functions that remain tractable at high dimensions [Abstract].
Reports competitive results against denoising-diffusion and deterministic-flow baselines on vision-language tasks and outperforms them on several natural-science evaluations [Abstract].

Method

BIT sets up a stochastic differential equation whose two boundary conditions are (i) a sample from the source modality (text embedding) and (ii) a sample from the target modality (image), and trains a network to match the drift of this bridge. Because both endpoints are real data (rather than noise), the path is “source-aware”: every intermediate state carries information about where the trajectory started. The construction is derived from stochastic calculus to yield a scalable regression loss over the drift, and the same trained SDE can be simulated forward (text→image) or reverse (image→text). Details of the specific parameterization, noise schedule, tokenizers, and training corpus were not extracted from the abstract-level fetch.

Results

Headline claim: BIT is competitive with denoising-diffusion and deterministic-flow baselines on vision-language tasks (e.g. text-to-image), and outperforms them on several natural-science evaluations [Abstract]. Specific benchmark numbers, model scale, and training compute were not extracted from the abstract-level fetch.

Why it’s interesting

Sits in an unusual corner of the Unified Multimodal Models design space: rather than the dominant AR / AR+Diffusion / MoT taxonomy tracked on that concept page, BIT proposes a single SDE bridge whose bidirectionality is architectural rather than a training-time recipe — one model, two directions. Contrast this with the DuoGen: Towards General Purpose Interleaved Multimodal Generation decoupled MLLM+DiT recipe or the encoder-free MoT approach in NEO-unify: Building Native Multimodal Unified Models End to End, both of which keep separate machinery for understanding and generation. Also of note as a pixel/latent-space diffusion alternative to text-conditioned CFG samplers tracked on Classifier-Free Guidance variants — the source-aware path removes the need for a two-pass conditional/unconditional mix. Whether the natural-science outperformance points at a broader “bridges beat noise-to-target flows when the source is structured” pattern is worth watching.