SteadyDancer: Harmonized and Coherent Human Image Animation with First-Frame Preservation

TL;DR

SteadyDancer is a pose-driven human image animation framework on top of Wan-2.1-I2V-14B that flips the dominant Reference-to-Video (R2V) paradigm — which binds a reference image onto a driving pose and breaks under real-world spatio-temporal misalignments — back to Image-to-Video (I2V) so that the first frame of the generated clip robustly equals the reference image. The team argues the R2V relaxation of alignment is why prior animators show identity drift, structural artifacts, and abrupt start-frame jumps when the driver’s body proportions or initial pose differ from the reference; SteadyDancer fixes this with (a) a Condition-Reconciliation Mechanism that channel-concatenates rather than adds the pose and image conditions and uses LoRA-only adaptation, (b) Synergistic Pose Modulation Modules (spatial dynamic-conv refiner + temporal factorized-conv coherence module + frame-wise cross-attention alignment) to align the pose representation to the reference image, and (c) a three-stage decoupled-objective training pipeline. It reports SOTA FVD on RealisDance-Val (326.49 vs Wan-Animate 386.87) with only 14,500 total training steps on ~10.2 hours of data — substantially less than comparable DiT-based methods.

Key claims

The dominant Reference-to-Video paradigm “relaxes alignment constraints” between the reference image and the driving pose and so fails under real-world spatial misalignment (limb structure/proportion) and temporal misalignment (initial-frame pose gap), producing identity drift, artifacts, and abrupt start-jumps [§1, Fig. 2].
An Image-to-Video paradigm trained for human animation inherently guarantees first-frame preservation by generating the clip starting from the reference, but requires tighter image-pose alignment than R2V, which the paper argues is why I2V animation research has been scarce [§1, §3].
A naive I2V baseline that VAE-encodes the pose and element-wise adds it to the image condition latent biases the model toward motion control at the cost of appearance preservation [§3.2, Eq. 2; Fig. 6 Row 1].
Replacing the element-wise addition with channel-wise concatenation of the image and pose latents preserves the distinctness of each condition and significantly improves both appearance and motion control [§3.3 “Condition Fusion”, Eq. 3; Fig. 6].
LoRA-based fine-tuning is the chosen condition-injection strategy because adapter-based or decoupled cross-attention injection adds too many trainable parameters and risks degrading the pre-trained model’s generation capacity [§3.3 “Condition Injection”; Fig. 6 Row 2].
Temporally concatenating the channel-cat condition with the image latent and the first-frame pose latent, plus enriching the global context with the CLIP feature of the first pose frame, are two augmentations that together improve first-frame preservation [§3.3 “Condition Augmentation”, Eq. 4].
The Spatial Structure Adaptive Refiner uses dynamic convolution (kernel generated from the pose features themselves) to make the pose representation more compatible with the reference image’s feature space [§3.4 SPAE].
The Temporal Motion Coherence Module uses stacked factorized convolutions (depthwise spatial + pointwise temporal) to smooth erratic pose sequences and suppress jitter [§3.4 TMSM].
The Frame-wise Attention Alignment Unit applies cross-attention with the denoising latent as Query and the pose latent as Key/Value to calibrate the pose representation per frame for appearance alignment [§3.4 FAAU].
The Condition-Decoupled Distillation stage uses the original pretrained I2V model as a stationary unconditional teacher and decomposes the student’s velocity prediction into an unconditional component (aligned to the teacher) and a conditional component (regressed to ground-truth velocity), so the unconditional manifold is injected without biasing the pose conditioning [§3.5, Eq. 6].
Motion Discontinuity Mitigation: a Pose Simulation procedure linearly interpolates between two real pose pairs to synthesize plausible start-gap training pairs, claimed to resolve >80% of start-frame discontinuity in only a few hundred fine-tuning steps and with no architectural change [§3.5].
Total training is 12,000 + 2,000 + 500 = 14,500 steps on a proprietary 7,338-clip / 10.2-hour dataset on 8 H800s — significantly less than RealisDance ([200k, 100k]), VACE (200k), or DreamActor-M1 ([20k, 20k, 30k]) at comparable or larger data scale [Table 2, §4.1].
On RealisDance-Val with VBench-I2V, SteadyDancer beats Wan-Animate on FVD (326.49 vs 386.87) and on identity-related I2V Subject/Background metrics while remaining competitive on Sync metrics; it also beats Hyper-Motion, UniAnimate-DiT, VACE, MimicMotion, Animate-X, HumanVid, Moore-AnimateAnyone on FVD [Table 1 bottom].
On the same-source TikTok benchmark, SteadyDancer is competitive but not dominant (FVD 451.3 vs Realisdance-DiT 458.8 / Animate-X 508.6) — the authors flag that same-source benchmarks systematically favor methods using the middle frame as reference (shorter average spatio-temporal distance), while SteadyDancer uses the first frame [Table 1 top, §4.2].
The new X-Dance benchmark is introduced specifically to evaluate the spatio-temporal misalignment regime by pairing reference images and driving videos from different sources (male/female/cartoon × upper/full-body × complex motion with blur/occlusion) [§4.2, Fig. 7-8].
Ablations: channel-cat beats element-wise addition (Row 1) and beats adapter-based injection (Row 2) on identity preservation; removing Condition Augmentation degrades appearance fidelity (Row 3) [Fig. 6, §4.3].
ComfyUI integration via kijai’s ComfyUI-WanVideoWrapper landed within a week of arXiv release (Nov 30 2025) and a GGUF-quantized weight via city96 followed Dec 4 2025, with WanGP integration Dec 3 2025 — fast community adoption around the Wan-Animate ecosystem [GitHub README Community Works].

Method

The starting point is a pretrained Wan-2.1-I2V-14B latent video DiT. The base I2V model conditions on a static reference image (encoded by the VAE then temporally concatenated with zero-filled frames into a condition latent) plus a binary mask, channel-concatenated with the noisy latent and a global CLIP context. SteadyDancer adds a new control condition — a per-frame 2D pose sequence — by reusing the same VAE to encode the pose into a pose-condition latent, then channel-concatenating it (rather than adding it) with the image-condition latent. The combined condition is temporally concatenated with the image latent and the first-frame pose latent so the model has an explicit reference both for starting appearance and starting pose, and the global CLIP context is augmented with the CLIP feature of the first pose frame for richer pose-aware semantics.

The pose latent itself is not used raw — three Synergistic Pose Modulation Modules refine it. The Spatial Structure Adaptive Refiner generates a dynamic convolution kernel from the pose features themselves to refine pose into a representation more compatible with the image’s feature space. In parallel, the Temporal Motion Coherence Module applies stacked factorized convolutions (depthwise spatial then pointwise temporal) to smooth erratic poses. Their outputs combine with the base feature via residual connection, and the result is calibrated by the Frame-wise Attention Alignment Unit — cross-attention with the denoising latent as Query and the pose latent as Key/Value — to enforce frame-wise appearance/pose correspondence.

Training is split into three stages with decoupled objectives. Stage 1 (Action Supervision, 12,000 steps) uses LoRA-only adaptation to instill motion control without disturbing the I2V generative prior; the first frame of each training video is fixed as the reference, and the whole video is both the motion source and supervision target. Stage 2 (Condition-Decoupled Distillation, 2,000 steps) treats the original pretrained I2V model as a frozen unconditional teacher and the Stage-1 student as a conditional flow estimator; the student’s velocity prediction is split into an unconditional component (aligned to the teacher via MSE) and a conditional component (regressed to ground-truth velocity), so the unconditional manifold is injected without biasing pose conditioning. Stage 3 (Motion Discontinuity Mitigation, 500 steps) trains on Pose-Simulated pseudo samples that linearly interpolate between two real pose pairs to expose the model to plausible start-gap mismatches — claimed to resolve over 80% of start-frame discontinuity at no additional inference cost.

Results

RealisDance-Val (VBench-I2V): SteadyDancer leads on FVD (326.49) vs Wan-Animate (386.87), Hyper-Motion (568.14), UniAnimate-DiT (599.03), Animate-X (679.66), HumanVid (624.33), Moore-AnimateAnyone (748.38), MimicMotion (785.73), VACE (911.72) [Table 1 bottom].
TikTok same-source benchmark (low-level metrics): SteadyDancer FVD 451.3, beats Realisdance-DiT (458.8), Animate-X (508.6), HumanVid (691.8), Champ (736.1), MagicAnimate (876.0), Animate Anyone (935.6); SSIM 0.749 / PSNR 17.67 / LPIPS 0.263 / FID 30.65 — Champ leads on SSIM/PSNR/LPIPS, Realisdance-DiT leads on FID by a small margin [Table 1 top]. The paper flags this regime as systematically disadvantaging first-frame-as-reference methods because middle-frame baselines have shorter average distance to the rest of the clip [§4.2].
Training-resource efficiency: 14,500 total steps and 10.2 hours of training data — RealisDance uses [200k, 100k] steps on ~16k videos, VACE uses 200k steps on 1M videos, DreamActor-M1 uses [20k, 20k, 30k] steps on 500-hour videos, RealisDance-DiT uses 1M videos. SteadyDancer is the smallest training budget in Table 2 among DiT-based methods [Table 2].
Qualitative on X-Dance (different-source benchmark with intentional spatio-temporal misalignment): SteadyDancer reportedly preserves first-frame identity and follows the driving motion where Wan-Animate, RealisDance-DiT, and others fail to do both simultaneously, exhibiting “catastrophic dual failure” [§4.2, Figs. 7-8].
Ablations confirm channel-concatenation beats element-wise addition and adapter-based injection on identity preservation, and that the Condition Augmentation strategy (temporal concat of first-frame pose latent + CLIP-pose global context) is necessary for appearance fidelity [Fig. 6, §4.3]. SPAE/TMSM/FAAU each tackle a distinct misalignment failure mode (pose errors, missing/contradictory poses, structural disparities) [Fig. 11].
Community uptake: ComfyUI integration via kijai (Nov 30 2025), WanGP integration (Dec 3 2025), GGUF quantization (Dec 4 2025), ModelScope mirror (Dec 8 2025) — fast adoption within the Wan-Animate ecosystem.
14B Wan2.2-Animate is independently claimed to lead on RealisDance-Val FVD (386.87) in the Wan-Animate report, which SteadyDancer now beats — explicitly addressed in the existing Wan-Animate wiki entry (Wan-Animate: Unified Character Animation and Replacement with Holistic Replication).

Why it’s interesting

SteadyDancer is the latest entry in the “frozen Wan-2.1-I2V-14B + parameter-efficient adapter” recipe family tracked by Camera-Controlled Video Diffusion — specifically the character-animation branch alongside Wan-Animate: Unified Character Animation and Replacement with Holistic Replication — but it makes a sharp paradigm argument that prior R2V methods (Wan-Animate, VACE, UniAnimate-DiT, RealisDance-DiT, DreamActor-M1) overrelax the alignment between image and pose conditions, and that flipping back to I2V with a careful three-aspect Condition-Reconciliation Mechanism plus pose modulation modules wins on both first-frame preservation and FVD at >10× less training data than the R2V leaders. This complements Wan-Animate’s region-flagged input-paradigm approach (which kept I2V structure but added skeleton-on-noise-latent additive injection): both papers converge on “preserve the pretrained I2V structure, decouple conditioning channels,” but they disagree on whether to use additive injection on noise latents (Wan-Animate) or channel-concatenation with the image-condition latent (SteadyDancer). The Condition-Decoupled Distillation stage is also notable as a training-time knowledge-distillation that injects the unconditional manifold from the original I2V model into the post-trained student to recover generation quality — adjacent to but distinct from the score-distillation/CFG-distillation patterns tracked in Diffusion Distillation (which target inference-step reduction rather than quality recovery after conditioning).