MegaFlow: Zero-Shot Large Displacement Optical Flow

TL;DR

MegaFlow is a unified optical-flow / point-tracking model built on a frozen DINO backbone plus a trainable CNN, that handles variable-length inputs and large displacements zero-shot — without any architectural change between dense-flow and point-tracking deployment. Alternating frame-local and global attention over DINO patch tokens produces a globally consistent feature representation; pairwise global matching then proposes initial flows, and a recurrent refiner (spatial convolutions + temporal attention) tightens them to sub-pixel accuracy. The same forward pass supports tracking through occlusions as a zero-shot application — point visibility is not predicted explicitly, the trajectory just continues. Notable for being the second filed paper to cleanly fuse a frozen self-supervised perception foundation model (DINO here, VGGT in Track4World) into a downstream dense-tracking head.

Key claims

A single architecture, with no per-task modifications, handles both dense optical flow and point tracking — the latter is a zero-shot application of the same flow predictor [§Method].
Frozen DINO features supply the global-semantic backbone, while a trainable CNN provides local structural features; the two are fused before alternating frame + global attention [§Method].
Alternating frame attention (within-frame) and global attention (across the sequence) is what gives the representation its variable-length, large-displacement robustness — pair-wise global matching can then act on a globally consistent feature volume [§Method].
The recurrent refinement stage uses spatial convolutions plus temporal attention to upgrade the initial coarse flow to sub-pixel accuracy [§Method].
Tracking through occlusions falls out for free because the point-tracking deployment doesn’t predict visibility — the model continues to emit a position even when the point is unseen [project page caption].

Method

Given a video sequence, MegaFlow extracts dense patch tokens from each frame using a frozen DINO encoder and complements them with local structural features from a trainable CNN. These tokens flow through alternating frame attention (mixing within a frame) and global attention (mixing across the sequence), followed by a feature-fusion step that produces a globally consistent representation. From this representation, pair-wise global matching computes initial dense flows between frame pairs. A recurrent refinement module then iteratively upgrades each initial flow using spatial convolutions for local consistency and temporal attention across iterations for sub-pixel accuracy. The same network handles variable-length input without architectural changes, and is run zero-shot for point tracking by querying it for the trajectory of a chosen pixel across frames — visibility is not modeled, so occluded points are tracked through the occlusion rather than masked out.

Results

The project page reports zero-shot dense flow and point-tracking demonstrations qualitatively, but does not surface a numbers table beyond the displayed (1 / 16 points) tracking visualization caption. Quantitative benchmarks are not extracted on this page; the arxiv preprint (2603.25739) is the place to look for them.

Why it’s interesting

MegaFlow and Track4World: Feedforward World-Centric Dense 3D Tracking of All Pixels are a tight pair: both wrap a frozen perception foundation model (DINO here, VGGT there) with a small trainable head that turns the foundation into a dense tracker. Track4World goes further — it predicts joint 2D + 3D flow and world-coordinate tracks — while MegaFlow stays in image-space but explicitly targets the large-displacement regime and reuses the same head for point tracking with no retraining. The shared bet is that the frozen-backbone + thin-head recipe is now the default way to build perception trackers, and that the right backbone choice (DINO vs VGGT) is what determines whether you stay 2D or go fully 3D. Worth tracking against The flavor of the bitter lesson for computer vision‘s position that such intermediate perception stacks will be dissolved by end-to-end generative-rollout world models — MegaFlow is a clean example of the recipe that essay critiques, and its zero-shot generalization is one of the stronger empirical arguments for keeping the recipe alive.