Motion4Motion: Motion Transfer Across Subjects at Inference

TL;DR

Motion4Motion is a training-free motion-transfer framework that drives a target subject in a pre-trained WAN-T2V DiT from a source video, without ever computing or aligning a skeleton. Instead of kinematics, it operates on dense pixel-level motion flow — anchor points sampled from the source subject (via Grounded SAM-2), lifted to the target through diffusion-feature semantic matching, and tracked in time — and re-injects that flow into self-attention by re-embedding cached K/V under new RoPE positions (“TransPE”). The recipe transfers motion across arbitrary morphologies (human → goose, human → panda, human → table) that no skeleton-based method can bridge, and outperforms FlexiAct, MotionClone, MotionDirector, RoPECraft, Diffusion-As-Shader, and WAN-Move on both animal (33 pairs) and human (123 pairs) benchmarks. SIGGRAPH ‘26.

Key claims

Skeleton-based motion transfer breaks whenever source and target morphologies diverge (human vs goose vs table), because no shared skeletal template exists and labeled cross-topology data is scarce; a skeleton-free representation is required for cross-species generalization [§Why skeleton-free].
Dense pixel-level motion flow — spatio-temporal trajectories of anchor points sampled by Grounded SAM-2 on the source subject and matched to the target via diffusion-feature correspondence — is a topology-agnostic substitute for the skeleton [§Motion flow in, motion flow out].
TransPE injects motion into a frozen WAN-T2V DiT by caching the target’s K/V from inversion, replicating them along the time axis, and re-embedding them with RoPE positions taken from the retargeted motion flow — no training required [§How K and V are rewired].
The two-stage inference pipeline (motion-flow extraction + TransPE attention) is entirely inference-time; no weight updates, no adapter, no fine-tuning on the source/target pair [§Motion flow in, motion flow out].
On animal motion transfer (33 pairs) and human motion transfer (123 pairs), Motion4Motion ranks best across Textual Similarity, Motion Fidelity, Temporal Consistency, Appearance Consistency, and Pose Similarity vs FlexiAct, MotionClone, MotionDirector, RoPECraft, Diffusion-As-Shader, and WAN-Move [§Cross-species results].
Novel-concept composition (e.g. “a desk coming to life, running rapidly along a muddy riverside”) works via a light “bone-binding” trick that pairs SAM-2 masks (human legs ↔ table legs) and drives the table’s gait entirely through flow-based attention manipulation — no dedicated training on animate-desk data [§Teaching a table to walk].

Method

The core pipeline runs entirely at inference time on a frozen WAN-T2V DiT. Stage 1 — motion flow extraction: anchor points are sampled on the source subject in the first frame via Grounded SAM-2; a semantic matcher built on diffusion features establishes cross-image correspondence to the target subject; a point tracker then produces temporal trajectories of the source anchors across the source video. Both spatial (source→target) and temporal (across-frame) correspondence are handled outside the DiT. Stage 2 — TransPE attention: during denoising, the target subject’s K and V (obtained by inverting the target frame) are cached and replicated along the time axis to match the source clip length; each replicated K/V is then re-embedded with a new RoPE position taken from the retargeted motion flow. The DiT’s query “looks for” the target’s appearance at the coordinates dictated by the source flow, so appearance follows kinematics without either being learned end-to-end. For hard cases where morphology mapping is ambiguous (human → table), an additional bone-binding step pairs SAM-2 masks between source parts and target parts so that flow chunks are routed to the right target regions.

Results

Ranks best (mean of ranks, or first-place per metric — the blog does not report scalars) on five metrics across two benchmarks: animal transfer (33 pairs) and human transfer (123 pairs) — Textual Similarity, Motion Fidelity, Temporal Consistency, Appearance Consistency, Pose Similarity [§Cross-species results].
Beats seven prior methods head-to-head: FlexiAct, MotionClone, MotionDirector, RoPECraft, Diffusion-As-Shader, WAN-Move [§Cross-species results].
Concrete novel-concept demo: transfers a walking human’s gait to an animate desk / table (a case where WAN-T2V alone produces a static or sliding desk) via the bone-binding + TransPE combination [§Teaching a table to walk].
Full quantitative tables and ablation deltas are not surfaced in the project-page text at filing; SIGGRAPH camera-ready paper (DOI 10.1145/3799902.3811062) is the primary reference.

Why it’s interesting

Motion4Motion is the third filed instance of inference-time RoPE manipulation as a control primitive on the WAN video-DiT stack, joining Video Analysis and Generation via a Semantic Progress Function (ReTime/SPF warps temporal RoPE by an SPF derived from the model’s own SigLIP frame distances) and OmniTransfer: All-in-one Framework for Spatio-temporal Video Transfer (OmniTransfer uses per-task RoPE offsets to switch between camera/motion/ID/style/effect reference-transfer). The three papers together triangulate the “RoPE-as-control” design space: SPF measures a property of the model output and warps RoPE globally; OmniTransfer picks the offset axis (spatial-width vs temporal-frame) to select the task; Motion4Motion sets per-token RoPE positions from a separately estimated motion field rather than from a global schedule or a reference video. Unlike Wan-Animate: Unified Character Animation and Replacement with Holistic Replication and Kling-MotionControl Technical Report — the current skeleton-based SoTA — Motion4Motion covers the cross-topology regime that skeletons cannot bridge (human ↔ animal ↔ inanimate object), which is exactly the failure mode named in the Camera-Controlled Video Diffusion cluster’s open questions. The fact that it needs no training also complements Time-to-Move: Training-Free Motion Controlled Video Generation via Dual-Clock Denoising as another data point that training-free flow/attention manipulation is closing the gap on trained motion-control adapters.