Qwen3-VL-Reranker-8B

TL;DR

Open-weights multimodal cross-encoder reranker built on Qwen3-VL-8B-Instruct, designed as the second stage of a recall→rerank retrieval pipeline paired with Qwen3-VL-Embedding-8B. Takes a (query, document) pair where either side can be arbitrary text/image/screenshot/video mixtures and outputs a single relevance score. On the MMEB-V2 retrieval subset the 8B variant reaches 79.2 average (vs. 73.4 for its embedding sibling and 75.1 for the 2B reranker), a +5.8 point lift over single-stage dense retrieval and consistent gains on JinaVDR (83.6) and ViDoRe v3 (66.7) over jina-reranker-m0. Instruction-aware and multilingual (30+ languages), with sentence-transformers CrossEncoder support, HF Transformers, and a vLLM pooling/scoring path.

Key claims

• Built on Qwen3-VL-8B-Instruct as a cross-encoder reranker: input is a (query, document) pair where each can contain arbitrary text/image/video mixtures, output is a scalar relevance score [§Highlights, §Sentence Transformers usage].
• Supports 30+ languages and 32k context, mirroring the embedding model [§Qwen3-VL-Reranker-8B features].
• On MMEB-V2 retrieval (avg over Image/Video/VisDoc), 8B reranker reaches 79.2 vs. 75.1 for the 2B reranker and 73.4 for the 8B embedding alone — a +5.8 point lift from adding the rerank stage [§Model Performance].
• The 2B reranker (75.1) beats jina-reranker-m0 (2B) on image-only (73.8 vs 68.2 — note: jina-reranker-m0 has a partial entry) and visual-doc (83.4 vs 85.2 — jina wins here), and is competitive on retrieval more broadly [§Model Performance].
• On JinaVDR and ViDoRe v3 (visual-document retrieval benchmarks), 8B reaches 83.6 / 66.7 respectively, vs. jina-reranker-m0 82.2 / 57.8 [§Model Performance].
• The reranker also lifts MMTEB-Retrieval from 68.1 (embedding-only) to 74.9 (8B reranker) — +6.8 points on text-only retrieval [§Model Performance].
• Implements the standard “two-stage retrieval” recipe: the embedding model performs efficient initial recall, the reranker refines results in a second stage, with the explicit framing that this two-stage approach significantly boosts retrieval accuracy [§Highlights].
• Instruction-aware: default prompt is "Retrieve text relevant to the user's query.", customizable per task; same 1–5% gain from tuned instructions is reported [§Note, §Using Sentence Transformers].
• A 2B variant (Qwen3-VL-Reranker-2B) ships alongside the 8B; arxiv:2601.04720 is the joint technical report for the embedding+reranker series [§Citation].

Method

The reranker is the same Qwen3-VL-8B-Instruct backbone as the embedding model, but trained with a classification head (Qwen3VLForSequenceClassification with classifier_from_token=["no", "yes"]) — i.e. it’s posed as a binary relevance LM that scores P(yes | query, document) for the (query, document) pair as a chat conversation. Inputs flow through the standard chat template: a system message carries the task instruction (e.g. "Retrieve images or text relevant to the user's query."), the user message contains the query, and a separate document block contains text and/or an image. The score is read out of the model head; a sigmoid maps it to (0, 1) when calibrated probabilities are wanted. Released usage paths: sentence-transformers CrossEncoder.predict(pairs, prompt=...) and .rank(query, documents); HF Transformers via a Qwen3VLReranker wrapper; vLLM via LLM.score(...) with runner="pooling" and a jinja chat template. Trained on retrieval datasets drawn from MMEB-v2 retrieval subtasks plus MMTEB retrieval; visual-document benchmarks evaluated include JinaVDR and ViDoRe v3.

Results

Headline benchmarks (8B reranker, on top of Qwen3-VL-Embedding-8B as first stage; comparison to embedding-only and to 2B variants and jina-reranker-m0):

Model	Size	MMEB-v2(Ret) avg	Image	Video	VisDoc	MMTEB(Ret)	JinaVDR	ViDoRe v3
Qwen3-VL-Embedding-2B (no rerank)	2B	73.4	74.8	53.6	79.2	68.1	71.0	52.9
jina-reranker-m0	2B	—	68.2	—	85.2	—	82.2	57.8
Qwen3-VL-Reranker-2B	2B	75.1	73.8	52.1	83.4	70.0	80.9	60.8
Qwen3-VL-Reranker-8B	8B	79.2	80.7	55.8	86.3	74.9	83.6	66.7

Adding the reranker lifts MMEB-v2 retrieval average by 5.8 points (73.4 → 79.2) over the embedding-only baseline, MMTEB retrieval by 6.8 points (68.1 → 74.9), JinaVDR by 12.6 points (71.0 → 83.6), and ViDoRe v3 by 13.8 points (52.9 → 66.7). The 8B beats the 2B by ~4 average points across benchmarks, suggesting scale still pays off at the rerank stage and a 2B recall + 8B rerank pipeline is a sensible default.

Why it’s interesting

The interesting datapoint is the size of the rerank lift on visual-document retrieval — +13.8 on ViDoRe v3, +12.6 on JinaVDR — which is much larger than the gains usually reported for text-only cross-encoder reranking (typically 2–5 points). That suggests current multimodal embeddings still leave a lot of signal on the table that a cross-encoder can recover, and that for VisDoc workloads (the closest proxy to “search Luma’s PDFs / UI screenshots / training-doc archive”) the recall→rerank split is structurally important rather than optional. Practically, this gives Luma a fully open recipe for in-house multimodal search at any modality mix — a use case where Cohere Rerank Multimodal v3 has been the convenient closed option. Worth noting the 2B reranker is the practical sweet spot if rerank latency is dominant.

See also

• Qwen3-VL-Embedding-8B — the sibling embedding model; designed to be paired as the first-stage recall component.
• Technical report (arXiv:2601.04720) — joint report covering both embedding and reranker series.
• Qwen3-VL-Embedding GitHub — training code and additional usage examples (same repo for both models).
• Qwen3-VL-Reranker-2B — smaller variant of the same model.
• JinaVDR and ViDoRe v3 — visual-document retrieval benchmarks where the rerank lift is largest.
• NVIDIA Unveils New Open Models, Data and Tools to Advance AI Across Every Industry — NVIDIA Nemotron RAG ships an embed+rerank pair for multimodal/multilingual document retrieval in the same release window.