Qwen3-VL-Embedding-8B

TL;DR

Open-weights multimodal embedding model built on the Qwen3-VL-8B-Instruct VLM, designed to produce a single dense vector for arbitrary mixtures of text, images, screenshots, and video for retrieval and clustering. Outputs up to 4096-dim embeddings with MRL support (user-selectable 64–4096), 32k context, 30+ languages, instruction-aware encoding via a system-prompt instruction. On MMEB-V2 the 8B variant scores 77.9 overall (78 datasets, image+video+VisDoc), beating IFM-TTE (74.1), RzenEmbed (72.9), and Seed-1.6-embedding (76.9); on MMTEB text-only it reaches 67.88 mean (vs. 70.58 for the text-only Qwen3-Embedding-8B), confirming a small text-quality tax for going multimodal. Released as a sibling pair with Qwen3-VL-Reranker-8B (also 2B variant); intended to be used together in a recall→rerank pipeline.

Key claims

Built on Qwen3-VL-8B-Instruct as the backbone, producing 4096-dim embeddings with Matryoshka Representation Learning (MRL) so dimensions 64–4096 can be selected per use case [§Model Overview].
Accepts text, images, screenshots, video, and arbitrary multimodal combinations (e.g. text+image, text+video) in a unified encoder, with 32k context length [§Model Overview].
On MMEB-V2 (78 datasets across image/video/visual-doc), 8B reaches 77.9 overall vs. 76.9 for Seed-1.6-embedding-1215 and 74.1 for IFM-TTE; 2B variant reaches 73.4 [§Evaluation Results on MMEB-V2].
On image-only MMEB-V2, 8B = 80.1 overall (CLS 74.4 / QA 81.0 / RET 80.0 / GD 92.2); on video subset, 8B = 66.1 (CLS 79.1 / QA 70.1 / RET 57.0 / MRET 53.2) [§Evaluation Results on MMEB-V2].
On visual-document retrieval (VisDoc), 8B = 83.3 overall (ViDoRe v1 88.2 / v2 69.9 / VisRAG 88.8 / OOD 78.3), with the OOD split being the largest gap above prior SOTA [§Evaluation Results on MMEB-V2].
On MMTEB text-only benchmark, multimodal Qwen3-VL-Embedding-8B scores 67.88 mean (task), trailing the text-only Qwen3-Embedding-8B at 70.58 — a ~2.7 point text-quality cost for adding multimodal coverage [§Evaluation Results on MMTEB].
Encoding is instruction-aware (default system prompt "Represent the user's input."); the authors report 1–5% gains from task-specific instructions, and recommend English instructions even in multilingual settings [§Note, §Sentence Transformers usage].
Supports vLLM (runner="pooling") and SGLang (is_embedding=True) inference paths beyond HF Transformers and sentence-transformers [§vLLM / §SGLang Basic Usage Example].
The release includes a 2B variant (Qwen3-VL-Embedding-2B) with the same architecture and feature set; arxiv:2601.04720 is the technical report covering both embedding and reranker series [§Citation].

Method

The model is a fine-tuned Qwen3-VL-8B-Instruct (an MMDiT-style VLM, see HF base-model link) repurposed as a sentence-encoder. The backbone is loaded as AutoModelForImageTextToText; the model’s representations are pooled into a single vector per input. Inputs are formatted as ChatML conversations with a system message containing the task instruction ("Represent the user's input." by default) and a user message containing any combination of {"type": "text", ...} and {"type": "image", ...} content blocks — so an image-only document, a text-only query, and a mixed text+image document all go through the same conversational template. Embedding dimension is configurable from 64 to 4096 via MRL: the model is trained so that truncated prefixes of the 4096-dim embedding remain useful, letting downstream systems trade recall quality for index size. Quantization-aware training is claimed to preserve quality at int8/int4 (described in technical report). The released code paths include sentence-transformers (SentenceTransformer("Qwen/Qwen3-VL-Embedding-8B")), HF Transformers via a Qwen3VLEmbedder wrapper, and vLLM/SGLang in pooling mode for high-throughput serving.

Results

Headline benchmarks (8B):

MMEB-V2 overall (78 datasets): 77.9 (vs. 74.1 IFM-TTE, 72.9 RzenEmbed, 76.9 Seed-1.6-embedding, 68.9 Ops-MM-embedding-v1).
MMEB-V2 image only (36 datasets): 80.1 (vs. 77.9 IFM-TTE, 75.9 RzenEmbed, 78.0 Seed-1.6-embedding).
MMEB-V2 video only (18 datasets): 66.1 (vs. 59.2 IFM-TTE, 55.7 RzenEmbed, 67.7 Seed-1.6-embedding) — Seed wins on video by 1.6.
MMEB-V2 visual-doc (24 datasets): 83.3 (vs. 79.5 IFM-TTE, 81.3 RzenEmbed, 82.2 Seed-1.6-embedding) — VisDoc OOD 78.3 vs. Seed 70.7 is the largest single-split gap.
MMTEB text-only mean (task): 67.88 (vs. 70.58 text-only Qwen3-Embedding-8B, 68.37 Gemini Embedding). Multimodal training costs ~2.7 points relative to the pure-text sibling.
2B variant (Qwen3-VL-Embedding-2B): MMEB-V2 overall 73.4, beating VLM2Vec-V2 (59.2) and GME-7B (59.1) at smaller size.

Why it’s interesting

For a generative-models team this matters in two concrete ways. First, it’s an open, downloadable checkpoint that can index Luma’s own assets — image renders, video clips, screenshots of UIs — in a single shared vector space with text queries, without paying per-call API costs (Seed-1.6-embedding and Gemini Embedding are both closed). Second, the architecture is a direct repurposing of a Qwen3-VL VLM as an encoder, which lines up with the broader pattern Bud has been seeing this week: Qwen-family VLMs being fine-tuned into specialized roles (Video-Thinker on Qwen2.5-VL for grounded reasoning, VLM-Gym on Qwen-VL for RL gym, DiffThinker on Qwen-Image-Edit for reasoning). Worth tracking the text-vs-multimodal trade-off — the 2.7-point MMTEB tax suggests a single multimodal encoder isn’t yet a free upgrade over text-only Qwen3-Embedding-8B if the workload is text-dominant.