MoRight: Motion Control Done Right

TL;DR

MoRight is NVIDIA’s unified motion-controllable video generation model that disentangles two axes that prior work entangled: camera viewpoint and object motion. It uses a dual-stream architecture — one stream models object motion on a canonical frame, the other models camera motion — with cross-view self-attention transferring object motion across viewpoints. Motion is further decomposed into active (driving actions) and passive (responses), so the model can perform forward causality reasoning (predict consequences from an input action) and inverse causality reasoning (recover plausible actions from a desired outcome). At inference, users supply either active or passive tracks plus a target camera trajectory and the model fills in the rest, beating ATI and WanMove in the qualitative comparison while using only active tracks (no privileged depth or full-track input).

Key claims

Camera and object motion can be disentangled into two separate streams in a single video diffusion model: an object-motion stream over the canonical frame and a camera-motion stream, coupled by cross-view self-attention that transfers object motion across viewpoints [Method].
Motion itself can be decomposed into active (actions a user wants to specify, e.g. hand motion) and passive (responses caused by the actions, e.g. cloth or butter moving), enabling forward causality (action → consequence) and inverse causality (outcome → action) at inference time [Forward / Inverse Motion Causality Reasoning].
The disentangled design supports independent control: the same active object motion can be re-rendered under different camera trajectories (orbit, zoom in/out, static), with consistency across viewpoints up to stochastic-sampling differences [Disentangled Camera–Object Control].
Qualitatively, MoRight matches or beats ATI and WanMove on combined object+camera control while using only active tracks, whereas the baselines require both active and passive tracks plus privileged depth [Qualitative Comparison].
Failure modes are enumerated: unnatural motion when input tracks become temporally sparse due to occlusion, incorrect interaction reasoning (e.g. kabobs merging), physically implausible dynamics (objects disappearing), and hallucinated content in later frames [Limitations].

Method

MoRight is a video diffusion model with a dual-stream architecture. One stream operates on the canonical frame and represents object motion (active and passive tracks); the other stream represents camera motion. Cross-view self-attention is the bridge: it transfers object motion learned on the canonical frame across different camera viewpoints, so the same action can be re-rendered under an orbit, zoom, or static camera with no per-viewpoint retraining. The active/passive decomposition of motion is the second axis of disentanglement — by training the model to consume either the active tracks (and predict the passive response) or the passive tracks (and recover the action), the same architecture supports forward and inverse motion-causality reasoning at inference. The qualitative comparison emphasizes that MoRight uses only active tracks without privileged depth, whereas ATI and WanMove are fed full active+passive tracks with depth [Qualitative Comparison].

Results

The project page reports only qualitative side-by-side comparisons against ATI and WanMove across multiple scenes (static / zoom in / zoom out / orbit camera variants) — no VBench numbers, no user-study win rates, no FVD. The headline qualitative claim is that MoRight handles combined object+camera control with only the active track, while the baselines need active+passive+depth and still show artifacts. Limitations are explicitly visualized for four cases: occlusion-induced track sparsity, incorrect interaction outcomes, vanishing objects, and hallucinated extra hands.

Why it’s interesting

MoRight slots into the Camera-Controlled Video Diffusion cluster as the first filed paper to explicitly factor motion itself into active/passive components rather than just decoupling camera from a single object-motion channel. The cluster’s open question — where does 3D Gaussian trajectory representation break, and whether body/face/hand are the right axes or a different factoring is warranted — gets a new answer here: factor along causality (active vs passive), not anatomy. This complements Kling-MotionControl Technical Report‘s three-channel body/face/hand decomposition and VerseCrafter: Dynamic Realistic Video World Model with 4D Geometric Control‘s 3D-Gaussian object trajectories with a third orthogonal axis. The cross-view self-attention bridge is also a contrast point with SpaceTimePilot: Generative Rendering of Dynamic Scenes Across Space and Time‘s source-aware camera conditioning and OmniTransfer: All-in-one Framework for Spatio-temporal Video Transfer‘s RoPE-offset reference branch — three different mechanisms for the same “render the same object motion under a different camera” capability.