MinerU2.5: A Decoupled Vision-Language Model for Efficient High-Resolution Document Parsing

TL;DR

MinerU2.5 is a 1.2B-parameter document-parsing VLM from Shanghai AI Lab built around a coarse-to-fine, two-stage strategy: a single model first runs layout analysis on a downsampled thumbnail of the page, then crops each detected region at native resolution and feeds it back to itself for content recognition (text, formula, table). The architecture is a Qwen2-VL skeleton — 0.5B Qwen2-Instruct decoder + 675M NaViT vision encoder + a 2×2 pixel-unshuffle patch merger — trained in three stages (modality alignment → pretraining on 6.9M model-labeled samples → fine-tuning on 630K human-curated hard cases mined by inference-consistency). At 1.2B params and ~2.12 pages/s on A100 80G, it claims to outperform 3B-class specialists (dots.ocr, MonkeyOCR-pro-3B, Nanonets-OCR-s) on OmniDocBench while running 4-7× faster end-to-end.

Key claims

A single VLM split into a two-stage inference pipeline — thumbnail layout in stage I, native-resolution crop recognition in stage II — avoids the quadratic token cost of feeding a full-page native-resolution image to an end-to-end VLM, while still preserving fine-grained detail in the crops [§1, §3.2].
The 1.2B-parameter model is composed of a 0.5B Qwen2-Instruct decoder with M-RoPE replacing 1D-RoPE, a 675M NaViT-style native-resolution vision encoder initialized from Qwen2-VL-2B-Instruct, and a 2×2 pixel-unshuffle patch merger [§3.1].
Replacing Qwen2.5-VL’s window attention with Qwen2-VL’s full NaViT attention is reported necessary — window attention degraded document-parsing accuracy in their setting [§3.1, Vision Encoder].
Three-stage training: stage 0 modality alignment on LLaVA-Pretrain (frozen encoder + decoder, only MLP adapter trained) then LLaVA-Instruct VQA (full unfreeze); stage 1 pretraining on 6.9M model-labeled samples (2.3M layout, 2.4M text, 1.1M formula, 1.1M table) for 2 epochs; stage 2 fine-tuning on 630K samples for 3 epochs [§3.3, Table 1].
The pretraining corpus is generated by a closed-loop data engine — raw documents are first labeled by the prior MinerU2-pipeline, then refined per content type by larger expert models (Qwen2.5-VL-72B for text, an in-house retrained UniMERNet for formulas, an in-house table parser) [§4.1.2].
Hard-case mining (Iterative Mining via Inference Consistency, IMIC) selects high-difficulty samples from the unlabeled pool for fine-tuning; these are pre-annotated by Gemini-2.5-Pro for complex tables and human-corrected [§4.1.3].
A unified hierarchical tagging system covers non-body elements (headers/footers/page numbers), subdivides figures into image/chart/chemical_structure with separate caption/footnote tags, and assigns distinct categories to code, algorithms, references, and lists [§4.2.1, Table 4].
On OmniDocBench, MinerU2.5 achieves 2.12 pages/s and 2337 tokens/s on A100 80G with vLLM, vs 0.28 pages/s for dots.ocr (3B), 0.47 pages/s for MonkeyOCR-pro-3B, and 0.55 pages/s for Nanonets-OCR-s (3.7B) [Table 3].
Without deployment optimizations the baseline is 0.95 pages/s / 1045 tokens/s, still above all 3B-class baselines under their default configurations [§3.4, Table 3].
vLLM-level deployment tricks: asynchronous backend for page-level batching, decoupled scheduling of stage I vs stage II, and layout-type-conditioned frequency_penalty/presence_penalty to suppress degenerate token repetition without penalizing legitimate repetitive content like tables [§3.4].
Robustness augmentations are layout-type-aware — spatial transformations (scaling, grid distortion, rotation) are not applied to layout-analysis samples, only to content-recognition crops [§3.3.4, Table 2].

Method

The model is a vanilla Qwen2-VL stack: Qwen2-Instruct-0.5B decoder, NaViT-675M encoder initialized from Qwen2-VL-2B-Instruct, and a 2×2 pixel-unshuffle patch merger between them. The decoder’s positional encoding is swapped from 1D-RoPE to M-RoPE to accommodate the wide range of crop aspect ratios that arrive in stage II. The interesting move is at inference time: rather than feeding a full document page at native resolution (the dots.ocr / MonkeyOCR strategy) or losing detail through aggressive resizing, the same model is called twice — first on a fixed-size thumbnail with prompt Layout Detection: to emit bounding boxes plus a category from the unified tag schema, then once per detected region with prompts Text Recognition:, Formula Recognition:, or Table Recognition: at native crop resolution capped at a maximum pixel budget. Most of the document’s blank space never reaches the expensive native-resolution forward pass.

Training is staged to match: stage 0 borrows LLaVA-Pretrain and LLaVA-Instruct to bootstrap vision-language alignment; stage 1 pretrains on 6.9M layout/text/formula/table samples generated by the data engine (initial labels from MinerU2-pipeline, then per-content-type refinement by Qwen2.5-VL-72B for text, an in-house retrained UniMERNet for formulas, and an in-house table parser); stage 2 fine-tunes on 630K hard cases mined by IMIC and human-corrected, mixed with high-quality re-samples from stage 1 to avoid forgetting.

Results

OmniDocBench end-to-end throughput on A100 80G with vLLM is 2.12 pages/s at 2337 tokens/s of valid output, against 0.28 pages/s (dots.ocr 3B), 0.47 (MonkeyOCR-pro-3B), 0.53 (MonkeyOCR-pro-1.2B at 1.9B params), and 0.55 (Nanonets-OCR-s 3.7B) — a 4-7× pages-per-second advantage [Table 3]. On H200 141G the throughput climbs to 4.47 pages/s / 4938 tokens/s. Without deployment optimization the baseline is still 0.95 pages/s / 1045 tokens/s, which the paper notes already beats all listed 3B-class baselines under their default configurations. The paper claims SOTA on OmniDocBench across general-purpose and domain-specific models, and specifically calls out improvements in long mixed-language formula parsing and difficult tables (rotated, borderless, partial-border).

Why it’s interesting

This is the third document-parsing VLM filed in close succession — after DeepSeek-OCR: Contexts Optical Compression (DeepSeek-OCR, 380M+570M-active) and PaddleOCR-VL: Boosting Multilingual Document Parsing via a 0.9B Ultra-Compact Vision-Language Model (PaddleOCR-VL, 0.9B) — and it adopts a fundamentally different efficiency strategy: instead of compressing the entire page into a small bag of vision tokens (DeepSeek-OCR’s optical compression) or feeding the whole page at dynamic native resolution (PaddleOCR-VL’s NaViT), it routes the page through a layout-first decoupling where the expensive native-resolution forward pass only ever sees individual layout regions, not the full page. The cluster is now large enough that the differences are sharp: optical compression (DeepSeek-OCR) trades resolution for token count, native-resolution-everywhere (PaddleOCR-VL) trades token count for latency, and two-stage decoupling (MinerU2.5) trades a second forward pass for keeping both detail and token count bounded — and the paper’s throughput numbers suggest decoupling wins at scale for long documents. The data-engine framing (IMIC hard-case mining with VLM-assisted pre-annotation + human review) is also worth tracking against the broader Synthetic Training Data thread.