Ovis2.5 Technical Report

TL;DR

Ovis2.5 is Alibaba AIDC’s successor to Ovis2, a perception/reasoning-focused MLLM family (2B and 9B) that drops a native-resolution ViT (NaViT, initialized from siglip2-so400m-patch16-512) into the Ovis visual-tokenizer + visual-embedding-table architecture, and adds an optional “thinking mode” cultivated via DPO + GRPO post-training on reflection-style data. Ovis2.5-9B averages 78.3 on OpenCompass — claimed open-source SOTA in the sub-40B class — with leading numbers on chart/OCR (OCRBench 87.9, ChartQA Pro 63.8) and math reasoning (MathVista 81.4, WeMath top rank). A five-phase curriculum (VET pretraining → multimodal pretraining → instruction tuning → DPO → GRPO/RLVR) plus data packing + Megatron-based DP/TP/CP hybrid parallelism yields a claimed 3–4× end-to-end training speedup.

Key claims

Replacing fixed-resolution ViT + tiling with NaViT (native variable resolution, RoPE in every ViT block) eliminates tile-induced loss of global structure and is the primary driver of the chart/document gains [§2, §3.1.2 P2].
Training on reflection-style “thinking” data — <think>…</think> tags supervising self-checking and revision, not just linear CoT — is exposed at inference as an optional “thinking mode” with a controllable thinking-token budget, allowing latency↔accuracy tradeoff per query [§1, §3.1.2 P3].
Ovis2.5-9B averages 78.3 on the 8-benchmark OpenCompass suite (MMBench, MMStar, MMMU, MathVista, HallusionBench, AI2D, OCRBench, MMVet), claimed SOTA among open-source MLLMs in the sub-40B parameter class; Ovis2.5-2B averages 73.9 [§4.1, Table 2, Table 3].
On chart/OCR, Ovis2.5-9B reaches OCRBench 87.9, OCRBench v2 63.4 (EN) / 58.0 (CN), and ChartQA Pro 63.8 — beating the proprietary GPT-4o on OCRBench v2 in the reported numbers [§4.3, Table 6].
On visual grounding, Ovis2.5-9B reaches an average of 90.1 across RefCOCO/+/g splits, with particular strength on RefCOCOg (non-salient objects with complex descriptions) [§4.4, Table 7].
On video/multi-image, Ovis2.5-9B scores BLINK 67.3, MMT 69.3, VideoMME-with-subs 76.4, VideoMME-without-subs 72.6 [§4.5, Table 8].
A five-phase training curriculum — VET pre-training (frozen ViT, final-layer + visual head + VET only) → full-parameter multimodal pretraining → instruction tuning → DPO (with auxiliary NLL stabilization) → GRPO with vision modules frozen, LLM updated — is the pipeline shape [§3, Table 1].
Data packing + Megatron-based DP+TP+CP hybrid parallelism yields a claimed 3–4× end-to-end training speedup over their previous framework [§3.3].
During GRPO post-training, only the LLM parameters are updated; vision modules are frozen — claimed to focus optimization on high-level reasoning without degrading general multimodal abilities [§3.2.2 P2].
A fill-in-the-blank conversion of multiple-choice RLVR data is used to reduce guess-the-letter reward hacking during GRPO [§3.2.1 RLVR Data].

Method

Ovis2.5 retains the Ovis architecture’s three-module split: a Visual Tokenizer (ViT + visual head producing a probability distribution over a visual vocabulary), a Visual Embedding Table (one embedding per visual word, summed under the VT distribution to produce the final visual embedding), and an LLM. The architectural change vs. Ovis2 is replacing the fixed-resolution ViT with a NaViT (variable native resolution + RoPE in every block, initialized from siglip2-so400m-patch16-512); in pre-training Stage P1 the VET is trained first with RoPE disabled and dynamic position-embedding interpolation, and in P2 onwards RoPE is activated as the supported resolution range expands to a total pixel count up to 1792² (~3.2M pixels).

Post-training has two phases. DPO is run over a multimodal preference corpus that mixes (a) verifiable reasoning preferences (vanilla CoT vs thinking-style rollouts scored against ground truth) with (b) general-task MLLM-as-judge preferences, with the entire model — vision modules + LLM — trained full-parameter under a DPO + NLL composite loss. GRPO follows, on an RLVR corpus of math + science + visual logic problems, where multiple-choice items are partly converted to fill-in-the-blank to limit reward hacking; vision modules are frozen and only the LLM updates. Multimodal data packing (combining shorter samples into a single longer sequence to reduce padding waste) is paired with DP/TP/CP hybrid parallelism in Megatron to enable the wide-resolution training.

Results

OpenCompass averages: Ovis2.5-9B 78.3, Ovis2.5-2B 73.9, both reported as open-source SOTA at their scale ([§4.1] Table 2/3). Component scores for Ovis2.5-9B: MMMU 71.2, MathVista 81.4, OCRBench 87.9, HallusionBench 65.1, AI2D 85.5. On the more challenging MMMU-Pro reasoning suite Ovis2.5-9B reaches 54.4. On math: top rank on MathVista (81.4) and WeMath, second-rank on MathVerse, MathVision, LogicVista, DynaMath [§4.2, Table 4/5]. On chart/OCR: OCRBench v2 63.4 EN / 58.0 CN, ChartQA Pro 63.8 (top open-source score; ChartQA 77.6) [§4.3, Table 6]. On grounding: avg 90.1 over RefCOCO/+/g splits, with the largest margin on RefCOCOg val/test [§4.4, Table 7]. On video: BLINK 67.3, MMT 69.3, VideoMME-w/sub 76.4, VideoMME-w/o-sub 72.6, MVBench / MLVU / TempCompass numbers competitive with GLM-4.1V-9B-Thinking and Keye-VL-8B-Thinking [§4.5, Table 8].

Why it’s interesting

Ovis2.5 is a deliberate refocus inside the Ovis family. The previous filed entry Ovis-U1 Technical Report was a unified 3B model — understanding + T2I + image editing in one stack via a diffusion decoder and a bidirectional token refiner — and slotted into the Unified Multimodal Models taxonomy as an End-to-End joint-training instance. Ovis2.5 drops generation entirely and pours scale (2B and 9B) into perception + reasoning, with NaViT for the perception side and a DPO+GRPO “thinking mode” recipe for the reasoning side. It’s the same team taking the bet that perception-side and reasoning-side scaling pay independently of the unified-generation track, which is useful to the wiki as a control: if Ovis2.5-9B’s chart/math numbers cleanly beat unified counterparts at similar scale, the wiki’s ongoing question — does unified gen+understanding training help understanding? (UniG2U-Bench: Do Unified Models Advance Multimodal Understanding?, Generation Enhances Understanding in Unified Multimodal Models via Multi-Representation Generation) — gets a non-unified upper-bound datapoint.

The thinking-mode + GRPO recipe also lands cleanly in the Reasoning RL cluster. Unlike Video-Thinker: Sparking "Thinking with Videos" via Reinforcement Learning which uses a SFT→GRPO two-stage pipeline with structured-CoT tag rewards, or MiroThinker: Pushing the Performance Boundaries of Open-Source Research Agents via Model, Context, and Interactive Scaling which uses SFT→DPO→online GRPO with format-penalty rewards, Ovis2.5’s reasoning stage is a three-step SFT→DPO→GRPO with auxiliary NLL on DPO and the vision modules frozen at the GRPO step — a more conservative recipe that may explain the reported gains on chart-reasoning without obvious regression on general benchmarks. The fill-in-the-blank conversion to defeat MCQ guess-rewarding is a small but novel reward-hacking countermeasure that didn’t appear in earlier filed reasoning-RL papers.

The NaViT choice — variable native resolution with per-block RoPE — is also a counter-example to the NEO-unify: Building Native Multimodal Unified Models End to End direction of going encoder-free native pixels; Ovis2.5 keeps a pretrained ViT prior (siglip2) and pushes native-resolution through the existing encoder rather than removing it. That contrast becomes more pointed alongside Tuna-2: Pixel Embeddings Beat Vision Encoders for Multimodal Understanding and Generation which argues pixel embeddings can beat vision encoders entirely — Ovis2.5’s strong chart numbers are an existing-encoder + native-resolution counterargument at 9B scale.