Qwen3-VL compact dense — 4B and 8B (Instruct + Thinking)

TL;DR

Alibaba’s Qwen team shipped dense Qwen3-VL checkpoints at 4B and 8B scales, each in both Instruct and Thinking post-training variants, plus FP8-quantized siblings. They round out the Qwen3-VL family below the previously released 30B-A3B and 235B-A22B MoE tiers while keeping the same architectural deltas (Interleaved-MRoPE, DeepStack ViT fusion, Text–Timestamp Alignment), 256K-native context (expandable to 1M), 32-language OCR, and 2D/3D grounding. The dense, single-GPU footprint is what’s novel here: 4.44B and ~8.77B parameter checkpoints under Apache 2.0, runnable on a consumer GPU.

Key claims

• Compact dense Qwen3-VL released in 4B and 8B sizes, each with both Instruct and Thinking variants [tweet body].
• Released on 2025-10-15 per the upstream Qwen3-VL GitHub “News” section, following the 30B-A3B MoE release on 2025-10-04 [tweet timestamp].
• Tweet’s headline pitch is “lower VRAM usage, full Qwen3-VL capabilities retained, strong performance across the board” — i.e. capability surface preserved at smaller scale [tweet body].
• Per the linked tweet text, the dense variants are claimed to outperform larger models on some benchmarks (the comparison detail is cut off in the fetched text) [tweet body].

Method

Tweet-only artifact; no method content in the source. For the underlying architecture, the parent model card describes three Qwen3-VL upgrades over Qwen2.5-VL: Interleaved-MRoPE (positional embedding allocated across time/width/height), DeepStack (multi-level ViT feature fusion), and Text–Timestamp Alignment (timestamp-grounded event localization replacing T-RoPE). The training stack inherits from the Qwen3 base series, with FP8 checkpoints provided alongside BF16 for low-VRAM deployment.

Results

No benchmark numbers in the tweet. Downstream reporting (Qwen3-VL Technical Report, November 2025) places the full dense family at 2B / 4B / 8B / 32B alongside the 30B-A3B and 235B-A22B MoE checkpoints; all variants share the 256K context window.

Why it’s interesting

The 4B and 8B dense variants are the natural base models for ablation studies that need an existing strong VLM but can’t afford a 30B-A3B serving budget. Ayaan flagged exactly this use case in the originating Slack message — running Exploring Diffusion Transformer Designs via Grafting-style grafting ablations against base Qwen3 versus Qwen3-VL to see whether grafting transfers cleanly from the LM backbone to the VL backbone. This release also continues the Qwen3-VL family’s coordinated Instruct + Thinking pairing — the same matrix used by the Qwen3-VL-30B-A3B-Instruct / Qwen3-VL-30B-A3B-Thinking siblings — and slots in below them as the smallest “full capability surface” entry under Open foundation-model releases. With 4B at ~4.44B params and 8B at ~8.77B params under Apache 2.0, the family is now competitive with FastVLM: Efficient Vision Encoding for Vision Language Models for single-GPU multimodal use.

See also

• Qwen3-VL-30B-A3B-Instruct — the larger MoE sibling Instruct variant; same architectural stack, 30B total / 3B active.
• Qwen3-VL-30B-A3B-Thinking — the larger MoE sibling Thinking variant; the Instruct/Thinking pairing pattern repeats here at 4B and 8B.
• Exploring Diffusion Transformer Designs via Grafting — the grafting paper Ayaan plans to ablate against this checkpoint.
• Qwen3-VL-Embedding-8B — downstream specialization of Qwen3-VL-8B-Instruct as a multimodal embedding model.
• Open foundation-model releases — broader pattern of coordinated open-weights releases this fits into.